Re: Odd behavior when moving point over invisible text

[Karl Fogel]

On 04 Jun 2023, Mats Lidell wrote:
> When writing a unit test I wanted to move point over invisible 
> (hidden) text and
> got into this strange behavior:
>
> 1. Insert these two lines in an elisp file:
>    (defun func ()
>      (point)) 
> 2. Set outline-minor-mode { M-x outline-minor-mode }
> 3. Goto first line and do { M-x outline-hide-subtree }
> 4. Goto first . in the ellipsis after the parentheses () and try 
> either to use
>   the keyboard with C-f or use the command (forward-char) to 
>   move over the
>   ellipsis.
>
> With C-f point is moved over the ellipsis but with forward-char 
> point remains
> put!? I suspect it is due to point-adjustment but I was expecting 
> both of these
> to produce the same result, i.e. move over the ellipsis. Not to 
> mention that
> forward-char is also bound to C-f!?
>
> I tried this with 27.2, 28.2 and current master with the same 
> result.

This reproduces for me with current master 
(`emacs-repository-version' is 
"f947a0219bb6e43966e0e4e61ad6a15b0ed13e18").

The reason I'm looking into your report was because I thought it 
might be related to a known bug in indent.c:check_display_width() 
that affects scan_for_column() and results in a similar "point 
doesn't move forward by the right amount" misbehavior [1].  I 
don't yet know if the problem you're describing has the same 
underlying cause as [1]; I'd have to trace in and I don't know 
when I'll get time to do that.  But I thought I'd reply here in 
case they are related; maybe that would save you some time.

I looked around in the code a bit for your bug. 
cmds.c:Fforward_char() just calls move_point(), which in turn 
looks like it boils down to calling intervals.c:set_point(), which 
gets us to set_point_both() and that appears to finally be where 
some interesting action starts.  So is there a call to 
scan_for_column(), or something else that gets us to 
check_display_width(), somewhere within set_point_both() in the 
call stack?  Maybe?  I don't yet know.

I also don't know why `forward-char' is behaving differently in 
your reproduction recipe when called via `C-f' versus other ways. 
That's quite odd IMHO!  Both `M-x forward-char' and direct 
minibuffer evaluation of `(forward-char 1)' result in point 
staying on the first dot, just as you said, which is position 15 
in the buffer.  Whereas typing `C-f' moves point over the whole 
ellipsis and goes to position 28 in the buffer.  What's up with 
that?  There doesn't seem to be any advice added to anything by 
outline.el (disclaimer: I'm not very familiar with the advice 
mechanism in Emacs).  For example, calling `(ad-has-any-advice 
'forward-char)' within the reproduction buffer returns nil.

Best regards,
-Karl

[1] 
https://lists.gnu.org/archive/html/emacs-devel/2022-11/msg01170.html 
   From: Karl Fogel <kfogel@red-bean.com>
   To: Emacs Devel <emacs-devel@gnu.org>
   Subject: An interesting line-motion bug.
   Date: Wed, 16 Nov 2022 23:38:32 -0600
   Message-ID: <87cz9mywbr.fsf@red-bean.com>