bug#50983: 28.0.50; [REGRESSION, BUG] Display bugs with uncommon characters

[Eli Zaretskii]

> Date: Sun, 3 Oct 2021 13:54:33 +0100
> From: Alan Third <alan@idiocy.org>
> Cc: 50983@debbugs.gnu.org, rudiwillalwaysloveyou@gmail.com
> 
> n -> i
> o -> t
> t -> h
> e -> SPC
> - -> n
> _ -> o
> t -> t
> 
> So it's off-set by some 4 characters.
> 
> Looking at the raw file, there are 4 0xAD (SOFT HYPHEN) characters
> before "note", and after each one the offset increases by one.
> 
> I do not see them displayed in the terminal.

So this terminal seems to be unable to display those SOFT HYPHEN
characters (or maybe it's a "feature"?), and since Emacs knows nothing
about that, the relation between cursor position and buffer position
is disrupted.

> > If this is the case, the only way to fix the display is to use
> > us-ascii as terminal encoding.  Or maybe set up the terminal for a
> > "simpler" encoding, like latin-1, and then set up Emacs to that using
> > set-terminal-coding-system.
> 
> Indeed, changing the "character encoding" setting in iTerm to ASCII
> displays the soft-hyphens as a red "A" and everything seems to work
> right.
> 
> The default is UTF-8 and it reports itself as "xterm-256color". I
> suspect most terminal applications on macOS will default to UTF-8
> since that's the default everywhere else, which might help explain why
> this seems to be limited to macOS.

OK, thanks.  I think we can say this is not an Emacs problem.  I
recommend to file a bug with the developers of the terminal, and maybe
they will tell how to avoid that by some setting.