bug#50865: 28.0.50; Emoji with emoji modifier in Linux console garbles emacs display

[Aura Kelloniemi]

On 2021-09-29 at 16:00 +0300, Eli Zaretskii <eliz@gnu.org> wrote:
 > > From: Aura Kelloniemi <kaura.dev@sange.fi>
 > > Cc: 50865@debbugs.gnu.org
 > > Date: Tue, 28 Sep 2021 23:32:53 +0300
 > > 
 > > On 2021-09-28 at 21:35 +0300, Eli Zaretskii <eliz@gnu.org> wrote:
 > >  > That should not be that way.  Some characters are double-width, and
 > >  > should take up 2 columns on display.
 > > 
 > > I noticed, that Linux console does not understand most of the zero-width
 > > characters either.

 > It doesn't need to: Emacs displays those characters on a TTY as
 > spaces.

Can this be configured – i.e. can I change the space to something else to ease
debugging?

 > > It happily prints most of the code points in the list of
 > > zero-width characters. Of course they are printed just as diamonds, because
 > > Linux cannot store enough glyphs in its 512-glyph font space, but anyway it
 > > prints a diamond for such characters as <COMBINING GRAVE ACCENT>.

 > COMBINING GRAVE ACCENT (or any other combining codepoint) is not a
 > good example of zero-width characters.

On modern terminal emulators this certainly holds, but Linux is not a modern
terminal emulator and does not support combining characters. It just prints a
diamond for all codepoitns which don't have an associated glyph in the font
(or the kernel knows them to be zero-wide, and this information is out of
date).

 > Try "C-x 8 RET 200c RET"
 > instead.

 > Or FEFF
 > or 1D173 or E007f or 1BCA0.

They print just a single space within emacs. If I print them with echo,
they print a diamond.

 > > The character range \y200B-\u200F seems to be an exception here. When I 
 > > try to
 > > print one of these characters on a Linxu VT, it really prints nothing.

 > That's not exception, that's the rule, actually, for true zero-width
 > characters, not for accents.  Accents exist to combine with preceding
 > base character, and what you seem to describe means the Linux console
 > is unable to do even Latin accents?

Here is a sample Bash session for demonstration:
$ echo $'i\u300'
i◈
$ echo $'\uEC'
ì

 > > When I insert zero-width characters in Emacs, the diamonds representing the
 > > characters are printed interspersed by the padding spaces added by emacs. 
 > > The
 > > cursor is left behind the extending line of characters as a type, because
 > > Emacs thinks, that the zero-width characters really do not print anything,
 > > even though they do.

 > Is this with or without auto-composition-mode?

Ok, this was with auto-composition-mode set to t. And it only happens with
combining characters. Other zero-wide characters print the single space, as
should be.

If I set auto-composition-mode to nil, then Emacs does not print anything (not
even the space) when I insert a combining character. If I then move the point
over the invisible combining character, the point moves, but the screen cursor
does not. This is a very confusing behaviour.

Non-combining zero-wide characters print the space (as you said), and there
are no cursor movement issues.

When running in the Linux console emacs's term/linux.el sets
auto-composition-mode to a special value of "linux". I don't know what this
means.

-- 
Aura