bug#56682: Fix the long lines font locking related slowdowns

[Gregory Heytings]

> Thanks, I installed an optimization, which makes Isearch faster in such 
> cases.

Thanks.  Alas, as far as I can see, we're still quite far from a usable 
Emacs with long-and-truncated lines.  A recipe:

emacs -Q
C-x C-f long-line.xml RET
M-g g 12
C-u 12 M-x duplicate-line
C-x C-s multiple-long-lines.xml RET
C-x C-f multiple-long-lines.xml RET y
C-x x t
C-s </ RET

This last step is not yet finished after 10 minutes (600 seconds), and you 
cannot abort it.

> Turning off isearch-lazy-highlight in these cases is still a good idea, 
> though.

Indeed, (setq isearch-lazy-highlight nil) "fixes" the above recipe (but 
see below).

> The next problem with long-and-truncated lines is that in many places as 
> part of redisplay we begin from the line's beginning and then go to the 
> position of point or the first visible X coordinate or somesuch. This 
> becomes slow when point is very far to the right, like near the end of 
> the line.  A typical place in the code where this happens is at 
> beginning of display_line, where we call move_it_in_display_line_to to 
> get to it->first_visible_x.  This could take seconds if there are many 
> overlays on the line, which is what happens, for example, if we are in 
> the middle of Isearch and isearch-lazy-highlight is turned ON.

Indeed, Emacs isn't really usable.  For example, with 
multiple-long-lines.xml, after M-g g 20 C-e (which already takes several 
seconds to complete) each motion, search or insertion command still takes 
several seconds (even with find-file-literally).  For example (without 
isearch-lazy-highlight), C-s </releases> RET takes about 10 seconds. 
After that, C-s C-s takes another 10 seconds, an additional C-s also takes 
10 seconds, and C-g to abort isearch takes about 90 seconds.  And, even 
when you don't do anything, Emacs uses 100% of the CPU.  All this is, in 
the same file, instantaneous without truncate-lines.

> (Searching for "</" in long-line.xml produces 9K overlays!)

That's (another sign of) the fundamental problem I mentioned earlier. 
Without truncate-lines the visible portion of the buffer remains small, 
and it is thus possible to reduce the portion of the buffer that display 
routines will see.  With truncate-lines it can easily become huge.  For 
example, with multiple-long-lines.xml, (- (window-end) (window-start)) is 
7744464 with truncate-lines and 2454 without truncate-lines.  And 
multiple-long-lines.xml is still a relatively "small" file, it's only 15 
MB.

> I'll try to think how to speed up these move_it_* calls in those cases. 
> Ideas welcome.

I cannot claim I understand the display code enough, but I see no way to 
make the portion of the buffer that is considered by display routines 
smaller without introducing the kind of rectangular narrowing I mentioned 
earlier.  As far as I understand, everything else is just a band-aid, that 
will make Emacs behave a bit faster only in some cases, and with which 
Emacs will remain unresponsive in too many cases.

Perfect is the enemy of good.  I strongly suggest we just admit that Emacs 
can't cope with long lines and truncate-lines together, which is true 
anyway given the DISP_INFINITY limit.