On Fri, Apr 18, 2025 at 09:05:33AM +0300, Eli Zaretskii wrote:
> Cc: Johannes Altmanninger <aclopte@gmail.com>
> From: Johannes Altmanninger <aclopte@gmail.com>
> Date: Thu, 17 Apr 2025 20:46:32 +0200
>
> * lisp/term.el (term-handle-ansi-escape): Check if parameters contain
> subparameters (values separated by colon) and ignore those commands.
> This prevents incorrect interpretation of advanced terminal features
> that aren't yet properly supported, such as curly underlines (\e[4:3m).
>
> Consider ignoring whitespace changes when reviewing this patch.
> The essence of this change is the addition at the beginning of
> term-handle-ansi-escape, to skip all CSI commands that contain at
> least two subparameters (e.g. the ones that contain at least one ":").
>
> Some background / longer description:
>
> If I run
>
> emacs --eval '(ansi-term "/bin/bash")'
>
> and type
>
> printf '\e[4:3mHELLO\e[m\n'
>
> then I get text with a straight underline.
>
> This is not what I expect, because "\e[4:3m" is supposed to turn on
> curly/wavy underlines, see https://sw.kovidgoyal.net/kitty/underlines/.
>
> My program wants to use curly underlines to indicate errors. Rendering
> those as straight underlines is noisy and confusing because it suggests
> a different meaning (in my program).
Why cannot you solve this problem by a suitable configuration of faces
in Emacs?
I'm implementing the default behavior for a shell; it doesn't behoove
the shell to touch the user's (Emacs) configuration.
Now of course I could theoretically achieve my goal with this logic:
if the terminal supports underline styles (terminfo Smulx or Su):
use curly underlines
else:
use no underline at all
where the condition could be checked by:
1. reading the terminfo database based on $TERM
2. querying the terminal's embedded terminfo database via XTGETTCAP
Option 1 is not great because terminfo frequently has false negatives,
and many of those cannot easily be fixed because they use a generic
$TERM (that's the problem with user-agents). For example, GNOME
Terminal and KDE's Konsole both support styled underlines but their
TERM=xterm-256color says otherwise.
# False negative.
$ infocmp -x xterm-256color | grep -E 'Smulx|Su'
# Another false negative (here the terminfo db is "fixable").
$ echo $TERM
foot
$ infocmp -x $TERM | grep -E 'Smulx|Su'
# True positive.
$ infocmp -x kitty | grep -E 'Smulx|Su'
Smulx=\E[4:%p1%dm, Ss=\E[%p1%d q, TS=\E]2;,
$ infocmp -x xterm-kitty | grep -E 'Smulx|Su'
am, ccc, hs, km, mc5i, mir, msgr, npc, xenl, Su, Tc, XF, fullkbd,
Smulx=\E[4:%p1%dm, Ss=\E[%p1%d q, Sync=\EP=%p1%ds\E\\,
Note that only few terminals are "fixable" (by virtue of using a
non-impersonating $TERM), That comes with the added burden that one
may need to copy terminfo files to remote servers etc., which is
probably part of why few terminals do this.
Option 2 (XTGETTCAP) always gives the right answer because we are
asking the terminal directly -- both the request and response are
control sequences written to the pty. However it's only implemented
by xterm, kitty and foot so far, so I only use it if there is no
other good option.
But I think there *is* a better (long-term) option here: if we can
assume that a terminal ignores commands that it doesn't recognize,
we don't need to query the capability at all, thus making it much
simpler to use curly underlines in e.g. a bash script. In practice,
this is already the case for a lot of other sequences.
Now the case of «printf "\e[4:3m"» may not be as obvious.
One argument for ignoring the sequence it is that if a user really
wants ansi-term's current behavior of "use curly underlines if
supported, else use straight underlines", they can simply print both:
"\e[4m\e[4:3m", or equivalently "\e[4;4:3m".
XTerm ignores "\e[4:3m" and most the dozens of terminals I tested
follow suit. However I give you that there are some (important)
terminals render curly underline as straight underline:
- emacs ansi-term
- emacs-vterm
- GNU screen
- terminology
- tmux
- Vim
I plan to propose patches for all of them (ansi-term happened to be
the first) We'll see what other terminal developers think. I can
link the discussions here. I guess I'll try tmux next.
Notice that most of these are terminals that run inside another
terminal. So I guess the reason for this automatic fallback could
theoretically be that "tmux/screen/'emacs -nw' etc. cannot accurately
detect whether the underlying terminal supports curly underlines,
so we make the decision for them and give them straight underlines
when in doubt". But that would be somewhat odd (because it "blocks"
progress) and actually, I haven't found that to be the case for the
terminals where I already looked at the git-blame (ansi-term and
vterm -- for both of them, the missing XTerm compatibility seems like
an oversight). Note that the curly underline feature is mostly used
by text editors, which may be why no one had encountered this problem
yet with Emacs terminals.
> I'd rather ansi-term ignore the curly underline command until it's
> implemented.
We could have this behavior as an option, conditioned on some user
option, perhaps.
I would not be interested in an off-by-default option because if a
user says "this doesn't work in ansi-term" then I can already tell
them "use a different terminal, such as M-x vterm", so it wouldn't
really reduce effort.
I would be surprised if better XTerm compatibility would make things
worse but let's see what the consensus is.
If it's not acceptable to ignore curly underline sequences, I can
probably also implement them in term.el (i.e. actually recognize
\e[4:3m).
That would give us curly underlines in graphical emacs, but it
would barely change the "emacs -nw" behavior in practice, because,
as described above, asking terminfo for Smulx/Su has a lot of false
negatives. Patch 2/2 would change "emacs -nw" to not
emit anything if the capability is not advertised.
So my perspective is: if we want to implement \e[4:3m now, we should
also take the second patch to actually fix my use case for wrong
terminfo.
In future, to get out of this mess, we should consider not asking
for Smulx/Su and simply emit the styled sequences unconditionally.
Unless Emacs still needs to run on hardware terminals that misinterpret
those, that would be the sanest option going forward. It will
essentially force all terminal emulators to be compatible with this,
which is fine I think? From the terminals I have tested, emitting
\e[4:3m would only cause regressions on these terminals: abduco, dvtm,
JetBrains IDE terminals and urxvt. It should be easy to fix them or
add workarounds; in a few years it should no longer be an issue.
> FWIW Emacs itself already supports various underline styles, even in
> a TTY -- see e.g. 9f589eb9240 (Add support for colored and styled
> underlines on tty frames, 2023-04-20). It would be fairly easy to
> add support for curly underlines to ansi-term.
>
> In general, I wonder what's story on Emacs terminals. ansi-term
> has some other compatibility issues, for example it cannot parse
> OSC (Operating System Command) sequences, which causes friction
> with the fish shell. We can fix this fairly easily; alternatively
> maybe default to another terminal implementation such as vterm
> (https://github.com/akermu/emacs-libvterm) that offers better
> xterm-compatibility.
You don't tell in which Emacs version you see these issues. Please
tell, it might be important.
Ah it seems like "cannot parse OSC" was the wrong conclusion because
a bash command like
printf "\x1b]133;A;special_key=1\x07"
is already correctly ignored (though there is the problem that the
prompt is not redrawn).
The actual issue I'm seeing is that running fish shell version 4.0.1,
which prints the above command at startup, causes this text to show
in the terminal:
133;A;special_key=1
This is with both Emacs 30.1 and latest master (2808bef2522). fish
assumes basic VT100 compatibility. Perhaps this is a timing issue.
I'm in a good position to extract a minimal reproducer, I can follow
up with that (probably in a separate thread?).
Jared, any comments to the above and/or to the patch?
In any case, we are unable to accept such a large contribution without
you assigning the copyright for your changes to the FSF. If you are
willing to do that, I will send you the form to fill and the
instructions to go with it, to start your legal paperwork rolling.
sure.
Thanks.
> ---
> lisp/term.el | 230 ++++++++++++++++++++++++++++-----------------------