bug#56682: Fix the long lines font locking related slowdowns

[Gregory Heytings]

> Heh, it's a case of "better is worse".  There's nothing wrong with the 
> change in commit 350e97d78e: it does indeed make redisplay of 
> long-and-truncated lines significantly faster.  And that speedup is 
> what's causing the problem: because the initial redisplay of the buffer, 
> before the results of C-s are displayed, is much faster, the 
> lazy-highlighting starts very soon, and it inserts thousands of overlays 
> into the buffer (because it by default highlights all the matches till 
> window-end, which is EOB in this case).  And once it inserts enough 
> overlays, we are dead in the water: the shortcut in 
> forward_to_next_line_start can no longer be taken, and we instead 
> iterate the slow way, one character at a time, all the way to EOB, and 
> on top of that need on the way to examine every one of the overlays 
> inserted by lazy-highlighting.

Thanks; as far as I can see your analysis is correct.  At 304e2a3a05, if I 
press C-s </ and wait long enough (a few seconds are enough), 
lazy-highlighting has enough time to put enough overlays in the buffer to 
hang Emacs.

> At this point, I think the only way to produce reasonable performance 
> from C-s in this case is to turn off lazy-highlighting when lines are 
> truncated and the buffer has long lines.  Which means we need to expose 
> the value of long_line_optimizations_p to Lisp, via an accessor 
> function.  I already have one other use for this: "C-x =", which 
> attempts to report the column of the character, something that is very 
> slow in a long-and-truncated line.  And I think we will see more cases 
> where Lisp code needs to know about this in order to adapt itself to 
> long lines.

My conclusion is different: we will see more such cases, but only with 
truncate-lines enabled, and that means that truncate-lines should be 
disabled in such buffers.

The fundamental problem is that with truncate-lines we cannot really 
narrow the buffer to a smaller (contiguous) portion without creating 
problems.  What would be necessary would be some kind of non-contiguous 
narrowing.  If we are on line 1, on column 10000, of a buffer with 80 
lines each of which is 20000 characters wide, what we see on screen are 
characters 9960-10040, 29960-30040, 49960-50040, 69960-70040, and so 
forth.  Without such a non-continguous narrowing, all kinds of problems 
like the C-s and C-x = one you identified will appear.  These problems can 
indeed, in principle, be solved by adding local fixes depending on 
(long-line-optimizations-p) whenever we encounter such a bug, but doing 
that would be most regrettable, as the simplicity of these optimizations 
would be largely lost.  Doing that while knowing that, as I said, we'll 
hit another ceil very soon is clearly (to me at least) not TRT.