bug#56393: Actually fix the long lines display bug

[Eli Zaretskii]

> Date: Wed, 06 Jul 2022 16:57:22 +0000
> From: Gregory Heytings <gregory@heytings.org>
> cc: larsi@gnus.org, 56393@debbugs.gnu.org
> 
> > How is it different from the code in the display engine that calls 
> > next-single-char-property-change or remove-text-properties, or loops 
> > over all the overlays at certain position calling overlay-get?  In Emacs 
> > nowadays font-lock is almost always called as part of redisplay, so I 
> > don't see how we can separate them and say that this is a different 
> > problem.
> 
> Again, I don't know (my understanding of the font-lock machinery is very 
> limited).

Your understanding of font-lock, or your understanding how it is
invoked from the display code?

I can help with the latter.  When Emacs is about to display some chunk
of text, it checks whether the text has a non-nil 'fontified'
property.  If it does, that chunk of text was already fontified, but
if not, the display engine calls font-lock (via jit-lock.el) to
fontify the next 1500 characters, and puts a non-nil 'fontified'
property on those 1500 characters.  The result is face properties,
which are then actually displayed.

So if font-lock is slow for some reason, it will slow down redisplay.

> I only observe that turning font lock mode off makes things
> significantly better.  And that turning highlighting off in such
> files is what other editors do, too.  So at this point I don't see
> why it wouldn't be a reasonable thing to do.

I think we should understand why it becomes slow before we decide
whether to turn it off or speed it up somehow.  Other editors fontify
text using very different methods, so their limitations are not
necessarily similar to ours.

Btw, what happens if you turn on jit-lock-debug-mode?

> That's something I already did, and yes, I do see Emacs choking.  Try to 
> navigate in the attached 30K file.  Then try again after turning font-lock 
> mode off.
> 
> (But I do not understand what you mean by "leave 'recenter' as it behaves 
> on master".)

I meant disable its disabling.

> > For example, why not use 2 window-fulls before and after the window 
> > (assuming that gives us a smaller chunk)?
> 
> That would mean to compute the dimensions of the window in a command-hook. 
> I tried to make that hook as efficient as possible, so it is on purpose 
> that I used a constant there, with something that seemed a reasonable 
> default, instead of computing it again and again.

The primitives that access the window height and width are simple
accessors to C variables, so they are very cheap.  So I think this
might be a case of premature optimization.

A fixed constant is problematic because it needs to be large enough to
satisfy the largest possible window.  So for many windows we process
too many characters, and slow down redisplay unnecessarily for the
sake of those extreme cases.

> > The point is that we should try to squeeze the most out of this 
> > narrowing idea, before we start disabling up features.  Because 
> > disabling features is a kind of retreat, an indication that we turned 
> > every stone and couldn't make Emacs fast enough, so we are kinda giving 
> > up.  And it's too early to give up, IMO.
> 
> I agree with that idea/direction, but again I think that at least now 
> Emacs doesn't do worse than all other editors out there, and in fact it 
> does way better.  If all editors turn off highlighting in such cases, that 
> must surely be for a good reason.  Highlighting means parsing, and parsing 
> is infinitely more expensive than not parsing.

Again, why give up without trying to fix that?

> > And to me this means that you see an example of a problem I mentioned 
> > earlier: code from the display engine is used in commands that basically 
> > have nothing to do with redisplay per se, and your current 
> > implementation doesn't take care of those calls into the display code by 
> > commands like recenter, C-v, C-n, etc.  I think sooner or later we will 
> > need to present a narrowed buffer to them as well.
> 
> You mean, a widened buffer?

No, a narrowed buffer.  These commands use code from the display
engine, so letting them process a widened buffer means they run
slowly.

> Yes, that's a question for which I still don't have a clear answer:
> is it better to "whitelist" commands that will always work correctly
> with a narrowed buffer or to "whitelist" commands that may require a
> widened buffer?  I tend to think that the latter is better.

I think we need to present a narrowed buffer to display code
regardless of whether it runs by redisplay or by commands like C-n or
C-v or 'recenter'.

> > What kind of example do you want to see?  What do you mean by 
> > "problematic" in this context?
> 
> I mean, an actual example of "Lisp code that expects to have access to the 
> entire buffer when it has no reason to expect narrowing", that is, Lisp 
> code that expects to have access to the entire buffer without using 
> (save-restriction (widen) ...).

Are you saying that you don't believe in the existence of such code?
Every Lisp program expects to see the same narrowing (or lack thereof)
as the user sees.  So anything different is a time bomb waiting to go
off.

> > Which is why I'm saying that it is better to restrict the display code 
> > via some other means, and leave the narrowing alone.  After all, it is 
> > our code that deliberately references BEGV and ZV in the display engine, 
> > so using some other values shouldn't be that difficult, I think.
> 
> Yes, I've understood that you want to do something else.

No, it's the same idea, just implemented differently, and thus without
some of the problems we are discussing.  (But slowness of font-lock
will still need a separate solution -- or the conclusion that we have
no alternative but turn it off.)