Curly and colored underlines

[egmontkob]

This is originally a feature of Kitty, now also adopted by VTE (GNOME Terminal and friends). Technically two separate features, but they mostly make sense together, e.g. for spell checking.

Apparently vim and neovim have already / are about to support these, see e.g. vim undercurl, vim color, neovim.

---

The new SGR 4:3 (\e[4:3m) attribute, strictly with a colon as separator, was introduced to start a curly underline.

In the mean time, 4:0, 4:1 and 4:2 were also added as aliases for the standard 24 (turn off all kinds of underlining), 4 (single underline) and 21 (double underline), respectively.

At some point in the future, probably 4:4 and 4:5 could also stand for dotted and dashed underlines in some order (these are the five types of underlining supported by HTML/CSS).

---

The new SGR 58 and 59 sequences specify the color of the underline, following the pattern of 38 and 39. That is, 58;5;idx for an entry of the 256-color palette, or 58;2;r;g;b for direct RGB. There's no shortcut notation for the first 16 entries (corresponding to SGR 30-37 and 90-97), use the 256-color mode with indices of 0-15 instead.

59 reverts to the default, that is, the underline's color auto-following the text color.

In case you're short of bits, I believe it's okay to drop some precision, e.g. store only 4 bits per color channel. We were also considering this in the VTE bug.

Update: In VTE we ended up approximating the truecolor underline to 4+5+4 bits (R, G, B respectively). This means: 8+8+8+1 = 25 bits for foreground (in addition to truecolor, we have to be able to denote palette and default values too). Same for background. 4+5+4+1 = 14 bits for underline (again, an extra bit to denote palette and default values). That's a total of 64 bits for the colors, stored in an int64 (bold, italic etc. are additional bits are next to this in another variable). With multiplication instead of bitpacking, it could have been 5+5+5 bits for the underline. Of couse it's not a necessity to put all the color and no other attributes in the same integer, we just decided it's nice this way (and results in some performance improvement).

[egmontkob]

(As per kovidgoyal/kitty#226 (comment), mintty decided that 4:4 is dotted underline, and 4:5 is dashed.)

[Tyriar]

If someone wants to try put a PR together for this you can search the repo for "underline" to give you an idea of what has to happen. Basically support the escape sequence, enable storing it in the buffer and then support drawing it using the renderers.

[jwhitley]

As an aside, I note that both neovim and tmux now fully support undercurl in current releases as of this writing. It's pretty great when used with coc.nvim for LSP support, plus the appropriate undercurl highlight definitions.

[jerch]

We should implement this, its very useful for editors, debuggers etc. 😸
Any takers?

There are 3 gems hidden here though:

• extend sequence handlers to support those new sub parameters - easy one
• extend the buffer to hold needed information - tricky one, we have currently only 5 spare bits per cell (in total, not per channel lol), we either need to extend the buffer by another uint32 (increases memory usage from 12 to 16 bytes per cell) or some other trick to store that information.
• extend renderers to support those new line styles - not sure about the complexity of this one, the self drawing renderers (canvas, webgl) have different constraints than the DOM renderer. When I played with text-underline in different browsers the other day it was a nightmare - the support for this is mostly non-existent. The DOM renderer might be the hard part here.

[egmontkob]

the support for this is mostly non-existent

Really? Even two years ago, when I played with it, Firefox and Chromium supported curly and colored underlines.

(Chromium's one is much uglier, though. It can only draw entire periods, the last, partial period of the curly underline is completely omitted. It also stops and restart the curly underline on every DOM element boundary.)

Maybe even for browsers you could resort to some background image-like drawing? :)

Re bits: I've updated the original post.

[jerch]

@egmontkob

Really? Even two years ago, when I played with it, Firefox and Chromium supported curly and colored underlines.

Yeah, well its a mixed picture. I tested it with dashed/dotted, which leads to really ugly output or no underlines at all. Support for curly might be better. This needs some testing beforehand, still my guess is that we will have to resort to "drawing-by-hand" here to avoid box borders artefacts and such.

Re bits: I've updated the original post.

Thx for clarification. Yeah the attribute storage is an issue of its own, I tested several different approaches during the buffer rework. My favorite approach, storing them in a separate RB tree, led to lousy performance due to the needed additional tree lookup, while storing them in continous memory in the cells array itself is much much faster. Memory sacrifice for performance, as always. But with the underlines I wonder, if we should go that path again for a simple reason - underlines are very rare, thus the memory is wasted most of the time. Therefore a bit-notion in the cell and the additional lookup into a foreign storage might be sufficient here.

Edit: I wonder if the reduced underline palettes in VTE will create you color matching issues later on - the colors will slightly differ due to the reduced resolution, thus ppl cannot use FG/BG coloring tricks to temporarily hide those lines. Unlikely and rather an edge case, I know, but still a valid assumption from the sequence definitions.

[jwhitley]

FYI, there's a Chromium bug related to fixing the really poor rendering of text-decoration: underline wavy. That was blocked by the advent of "LayoutNG", which has apparently now landed.

So theoretically, it might be possible to avoid a custom renderer (for Chrome*, at least) if someone could be prodded into fixing that issue.

A cursory check via MDN's text-decoration page in FF vs Chrome shows that Firefox's wavy underline rendering is way better than Chromium's. Does FF also exhibit box boundary problems?

[jwhitley]

So theoretically, it might be possible to avoid a custom renderer (for Chrome*, at least) if someone could be prodded into fixing that issue.

Clarification: there's a good argument based on use cases like this one that wavy underline should render the wave angle based on a notion of absolute/viewport coordinates(?), not box-relative coordinates. If FF "does the right thing" on that point, it's additional ammunition to make that argument re: the Chromium issue.