bug#56682: Fix the long lines font locking related slowdowns

[Eli Zaretskii]

> Date: Thu, 28 Jul 2022 16:29:44 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: Gerd Möllmann <gerd.moellmann@gmail.com>, 
>     56682@debbugs.gnu.org, monnier@iro.umontreal.ca
> 
> Indeed, Emacs isn't really usable.  For example, with 
> multiple-long-lines.xml, after M-g g 20 C-e (which already takes several 
> seconds to complete) each motion, search or insertion command still takes 
> several seconds (even with find-file-literally).  For example (without 
> isearch-lazy-highlight), C-s </releases> RET takes about 10 seconds. 
> After that, C-s C-s takes another 10 seconds, an additional C-s also takes 
> 10 seconds, and C-g to abort isearch takes about 90 seconds.  And, even 
> when you don't do anything, Emacs uses 100% of the CPU.  All this is, in 
> the same file, instantaneous without truncate-lines.

I think such long lines happen in files that have very little except
the long line.  So duplicating it makes the use case much less
important.  And in any case, we shouldn't try dealing with such files
before we have a good solution for a single-line files.

> > (Searching for "</" in long-line.xml produces 9K overlays!)
> 
> That's (another sign of) the fundamental problem I mentioned earlier. 
> Without truncate-lines the visible portion of the buffer remains small, 
> and it is thus possible to reduce the portion of the buffer that display 
> routines will see.  With truncate-lines it can easily become huge.  For 
> example, with multiple-long-lines.xml, (- (window-end) (window-start)) is 
> 7744464 with truncate-lines and 2454 without truncate-lines.  And 
> multiple-long-lines.xml is still a relatively "small" file, it's only 15 
> MB.

This is actually a bug in isearch.el: using window-end in a buffer
under truncate-lines is simply wrong.  I'm going to file a bug about
that soon.

But that's not a fundamental problem, or at least I don't yet see why
it would be fundamental.  It's something specific to isearch.el and to
how isearch-lazy-highlight is implemented.  The fix I installed
prevented us from walking all that long line till the end of the
buffer, so in effect the display code doesn't see most of the buffer
text, they just jump over it.

> > I'll try to think how to speed up these move_it_* calls in those cases. 
> > Ideas welcome.
> 
> I cannot claim I understand the display code enough, but I see no way to 
> make the portion of the buffer that is considered by display routines 
> smaller without introducing the kind of rectangular narrowing I mentioned 
> earlier.

I still have one or two ideas to try, and they don't involve anything
as complex as some new kind of narrowing.

> Perfect is the enemy of good.  I strongly suggest we just admit that Emacs 
> can't cope with long lines and truncate-lines together, which is true 
> anyway given the DISP_INFINITY limit.

Feel free to give up on it and stop trying to make that case faster,
but I'm not ready to give up yet.