bug#56393: Actually fix the long lines display bug

[Gregory Heytings]

> I can help with the latter.  When Emacs is about to display some chunk 
> of text, it checks whether the text has a non-nil 'fontified' property. 
> If it does, that chunk of text was already fontified, but if not, the 
> display engine calls font-lock (via jit-lock.el) to fontify the next 
> 1500 characters, and puts a non-nil 'fontified' property on those 1500 
> characters.  The result is face properties, which are then actually 
> displayed.
>
> So if font-lock is slow for some reason, it will slow down redisplay.

Thanks.  I meant the former, but this is very useful nonetheless.  It 
means that the Lisp code which calculates these properties is (or at least 
can be) a bottleneck.  To determine which properties must be placed on 
these 1500 characters, it must presumably look further away, possibly much 
further away.

> I think we should understand why it becomes slow before we decide 
> whether to turn it off or speed it up somehow.  Other editors fontify 
> text using very different methods, so their limitations are not 
> necessarily similar to ours.

We can try, yes.  But my feeling so far is that this won't get us 
anywhere.

> Btw, what happens if you turn on jit-lock-debug-mode?

I'll try that.

> > I mean, an actual example of "Lisp code that expects to have access to 
> > the entire buffer when it has no reason to expect narrowing", that is, 
> > Lisp code that expects to have access to the entire buffer without 
> > using (save-restriction (widen) ...).
>
> Are you saying that you don't believe in the existence of such code? 
> Every Lisp program expects to see the same narrowing (or lack thereof) 
> as the user sees.  So anything different is a time bomb waiting to go 
> off.

I don't say that I don't believe in the existence of such code.  I get 
your point, and agree that narrow-mode (ab)uses a feature in a way that it 
was never intended to be (ab)used.  But I also say that files with 
extremely long lines are rare, and that it's in my opinion much better to 
see some rare functions fail when opening such files than to see Emacs 
becoming unusable.  Or in other words: if all essential editing commands 
work on these files, that's already a big step forward.

> > Yes, I've understood that you want to do something else.
>
> No, it's the same idea, just implemented differently, and thus without 
> some of the problems we are discussing.  (But slowness of font-lock will 
> still need a separate solution -- or the conclusion that we have no 
> alternative but turn it off.)

I tried to do that, and so far I don't see how your alternative approach 
could work.  The problem with long lines is not only that redisplay takes 
forever in some cases, it's also (and perhaps mainly) that commands take 
forever.  With dictionary.json, with point on the last character, C-p 
takes (on my computer) about 26 seconds outside of redisplay.  I don't see 
how we could tell all that code that it should work on a narrowed buffer 
without actually narrowing the buffer.