bug#56682: Fix the long lines font locking related slowdowns

[Eli Zaretskii]

> Date: Fri, 22 Jul 2022 10:16:52 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: Eli Zaretskii <eliz@gnu.org>
> 
> on my laptop, with Emacs from master from a week ago, take respectively 
> 150 seconds and 40 seconds.  With the improvements on master, 1 is 
> instantaneous but 2 still takes about 5 seconds.  Now, with the changes in 
> the feature/long-lines-and-font-locking branch, both are instantaneous. 
> The price of that speedup is that some portions of the buffer will be 
> mis-highlighted, which is unavoidable.

The branch is much faster, indeed, but still font-lock imposes a
significant penalty on commands that needs to redisplay.  For example,
running the following simple benchmark:

  (defun scroll-up-benchmark ()
    (interactive)
    (let ((oldgc gcs-done)
          (oldtime (float-time)))
      (condition-case nil (while t (scroll-up) (redisplay))
        (error (message "GCs: %d Elapsed time: %f seconds"
                        (- gcs-done oldgc) (- (float-time) oldtime))))))

on long-line.xml produces a 15-fold slowdown with font-lock turned on
as compared to its being turned off (203 sec vs 13 sec).

This is an unoptimized build, so you will probably see times that are
4 times faster, but I'd be interested in the relative times on your
system.  Any explanations of the slowdown are also welcome.