Problems rendering some emoji characters such as green circle

[bacforsyth]

lnav version
v0.11.1

Describe the bug
Lnav is not rendering some emoji characters such as green circle U+1F7E2 🟢. My terminal (macOS 13.2) renders the same emoji fine when a text file containing emoji is given to cat. lnav on the same file renders some of the emoji but not all.
lnav:

TextEdit:

Terminal:

To Reproduce
Steps to reproduce the behavior:
create a plain text file with a variety of unicode emoji characters and open the file with lnav to verify all are rendered correctly.

[tstack]

Hmm, maybe it's an ncurses issue? This python program has the same issue for me on macos:

import time
import curses

scr = curses.initscr()

curses.start_color()

scr.addstr(0, 0, "Grinning \U0001f600")
scr.addstr(1, 0, "Green Sq \U0001F7E9")
scr.addstr(2, 0, "Green C  \U0001F7E2")
scr.addstr(3, 0, "White Fl \U0001F3F3")
scr.addstr(4, 0, "")
scr.getch()

curses.endwin()

[bacforsyth]

I'm still having problems rendering some emojis in lnav 0.12.2 but interestingly they do show up in lo-fi mode. I'm on macOS 15.0.1.

[hgezim]

I'm also having issues. ✨ will not appear. 💦 works fine, tho.

[plan44]

I also have issues with emoj. Some don't display at all, such as ✅ and ℹ️, but others do, for example 🛠️, but somehow mess up displaying the line where these are on in the following ways:

• some characters following in the line text get duplicated
• in the white space between actual log text message and the "scroll bar", extra characters debris appears, sparsely distributed in the white space:

  [2025-01-10 19:51:41.754 D] 🛠️ CHIP:IN  ➡️ SecureSession[0x778a7bc0]: Allocated Type:2 LSID:47572                                v                                                                             )                                │
  

  (the "v" and the ")" are such extra debris. Moving the cursor over such a line usually redraws it correctly.
• as a result, the "scroll bar" often gets drawn one or two positions shifted to the left, so appears jagged instead of a straight line.

I thought this might be a bug in macos terminal.app, but iTerm 2 shows exactly the same behaviour.

And same as @bacforsyth observed: in loFi view (^L), ✅ and ℹ️ do appear and lines with 🛠️ display cleanly.

[tstack]

@plan44

I thought this might be a bug in macos terminal.app, but iTerm 2 shows exactly the same behaviour.

It's an ncurses issue, which is why it shows up in both terminals. In the top-of-tree, I have replaced ncurses with notcurses and that gets the emojis to show up. However, it looks like there are still some glitches in the lnav display where the character widths are not computed correctly. I'll investigate more and try to get it fixed before doing a release.

If you would like to try out a binary build from the ToT, they are attached to the runs of the "bins" action. For example: https://github.com/tstack/lnav/actions/runs/12730891846

Thanks for trying lnav and sending feedback.

[plan44]

Thanks for your superfast reply 😄 !

I tried the ToT version, and yes, it does display the emojs but has problems with the character widths and thus still distorts the scroll bar.

As a lnav newbie - can I place a filter/replace regex somewhere to prevent the problematic characters to appear as such in the log text at all?

[tstack]

I think this is fixed now in the ToT, thanks for you patience. I'm going to close it now, feel free to reopen or open a new bug if there is some other issue.

Here's the green circle:

And the log messages:

You can try out one of the builds attached to this action run -- https://github.com/tstack/lnav/actions/runs/12774294394

As a lnav newbie - can I place a filter/replace regex somewhere to prevent the problematic characters to appear as such in the log text at all?

I don't think so.