bug#50983: 28.0.50; [REGRESSION, BUG] Display bugs with uncommon characters

[Eli Zaretskii]

> Date: Mon, 04 Oct 2021 15:40:05 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 50983@debbugs.gnu.org, alan@idiocy.org
> 
> > From: Rudi C <rudiwillalwaysloveyou@gmail.com>
> > Date: Mon, 4 Oct 2021 11:35:41 +0330
> > Cc: Alan Third <alan@idiocy.org>, 50983@debbugs.gnu.org
> > 
> > But the problem does not happen with vim (nor with emacs 27 for 
> > `weird.txt`), so it is clearly an interaction
> > of different elements. 
> > 
> > Anyhow, I have opened an [upstream 
> > issue](https://github.com/kovidgoyal/kitty/issues/4094). Please
> > subscribe to it so that you might offer your emacs expertise there, if 
> > needed.
> 
> I subscribed and posted the following comment:
> 
> Emacs uses character width tables computed from the latest Unicode
> Standard version 14.0.0, using the data in the file
> EastAsianWidth.txt.  In that text, the U+00AD SOFT HYPHEN character,
> which caused the problems in your file, has the East Asian Width
> property value of A, which stands for "Ambiguous".  The definition of
> this value in the Unicode Standard Annex 11 (UAX#11) is as follows:
> 
>   East Asian Ambiguous (A): All characters that can be sometimes wide
>   and sometimes narrow. Ambiguous characters require additional
>   information not contained in the character code to further resolve
>   their width.
> 
>     Ambiguous characters occur in East Asian legacy character sets as
>     wide characters, but as narrow (i.e., normal-width) characters in
>     non-East Asian usage.
> 
> And since the file you show didn't have any East Asian legacy
> characters, treating SOFT HYPHEN as narrow is IMO correct.

To summarize the comments there:

The problematic character in the first example is U+00AD SOFT HYPHEN.
Kitty assumes that character is never rendered, and therefore
effectively treats it as zero-width character.

I don't see how Emacs display can possibly work correctly on such a
terminal, so I think we should close this bug report as "notabug".

For the second example, I think there could be an issue with character
compositions on this terminal, so the OP is advised to try turning off
auto-composition-mode.  If that solves the problem, fine; if not, I
guess Kitty once again assumes something about how such sequences are
rendered, and those assumptions don't fit how Emacs displays them in
reality, and if so, that problem, too, has no satisfactory solution
(and isn't an Emacs bug).