bug#56393: Actually fix the long lines display bug

[Gregory Heytings]

Thanks for your review; some comments below.

> This removal is premature, IMO.  We should first establish that this 
> feature can indeed cope well enough with the important use cases, and 
> I'm not sure we are there yet.  It is possible that this feature will be 
> one of several available features that attempt to solve the same problem 
> in different ways, each way with its own advantages and disadvantages.

AFAICS, this patch properly solves the problem indicated in that section. 
So I don't see why it should be kept.

> > -** 'longlines-mode' is no longer obsolete.
>
> Likewise here.

This mode was unobsoleted a few days ago, because the conclusion was that 
there was no better mechanism available to cope with large files.  Now 
there is.

> > +** Emacs is now capable of editing files with arbitarily long lines.
>
> I'd suggest saying something like
>
> Emacs now copes better with editing files with extremely long lines.

As I just said to Lars, Emacs can now handle files with 1GB long lines 
just fine, as if it were a small file.  I used

for I in $(seq 1 60); do cat dictionary.json; done > hugedictionary.json

So I think this title is appropriate.

> > +(defcustom auto-narrow-widen-automatically
> > +  '( move-beginning-of-line move-end-of-line backward-sentence forward-sentence
> > +     backward-sexp forward-sexp beginning-of-defun end-of-defun
> > +     beginning-of-buffer end-of-buffer goto-char goto-line
> > +     mark-sexp mark-defun mark-paragraph mark-whole-buffer mark-page
> > +     exchange-point-and-mark pop-global-mark set-mark-command jump-to-register
> > +     bookmark-jump)
> > +  "Commands for which display is automatically widened in `auto-narrow-mode'."
>
> What are the criteria for this default?  (I'm surprised the list is so 
> short, for the reasons I explain below.)

The review I asked was in particular meant to add commands in these 
defaults.  I reviewed all ordinary editing commands, but I may have missed 
some of them.  Please tell me which command you'd like to add in that 
list.  (Also note that isearch commands are handled with their own 
mechanism.)

> > +(setq auto-narrow-long-line-threshold 30000
> > +      auto-narrow-pre-command-function #'auto-narrow-pre-command-function
> > +      auto-narrow-post-command-function #'auto-narrow-post-command-function)
>
> We might eventually decide to have this turned off by default.

Sorry, but no.

> This will invoke auto-narrow-mode for any file that is for some reason 
> inserted by Emacs into some buffer.  Many Emacs commands insert files 
> into temporary buffers or buffers that are never meant for display, and 
> AFAIU the above will narrow such a buffer if a file inserted happens to 
> have long lines, is that correct?

It might, everything is possible with software.  Let's see if it happens 
in practice and take the appropriate measure if it happens.  And the cure 
for these potential cases is easy: (let ((auto-narrow-long-line-threshold 
nil)) ...)

> I don't quite see clearly what these general issues could mean, but they 
> are at least worrisome, I think, because they could potentially mean 
> significant breakage in many places.

Again, let's see if it happens in practice and take the appropriate 
measure if it actually happens.

> Also for commands written by users: for example, if someone writes a 
> command that is a trivial wrapper around goto-line, that command will no 
> longer automatically widen as goto-line does, right?

Indeed, that's the purpose of the auto-narrow-widen-automatically 
defcustom: users can easily add their own commands to that list.

> Alternatively, maybe you will explain that I'm bothered by a 
> non-existent problem?

What I think is that you are bothered with a potential problem which is 
far less important than seeing Emacs becoming unresponsive.

> Since the narrowing is basically in effect only during redisplay, it 
> doesn't help with commands that use display code outside of redisplay 
> proper.  For example, C-n, C-v, C-l, and other commands are still 
> extremely sluggish in files with long lines.  E.g., try the file 
> long-line.xml mentioned here:
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=45898#80

Did you actually try the patch?  This is (one of) the files I used while 
designing this code, and navigation commands work perfectly well (on a 
seven year old laptop).  In that particular XML file, you can turn off 
font-lock-mode to make navigation even faster (instantaneous).

> Also, the automatic restriction is very visible and causes surprising 
> effects: (point-min) and (point-max) evaluate to the narrowed region (so 
> it's basically very hard to know the real size of the buffer), and 
> scrolling through the buffer causes the scroll-bar thumb move in a 
> non-monotonic manner: I scroll towards the end of the buffer, but the 
> thumb sometimes jumps back up, as if I moved towards the beginning.

Indeed.  Users will see the "Auto-Narrow" indication in the modeline with 
which they can easily find an explanation for that behavior.  If this 
behavior is problematic, it could be fixed by using the dimensions of the 
whole buffer when updating the scroll bar (and perhaps also the modeline), 
when auto-narrow-mode is active.

> I think at this point it is best to have this feature on a feature 
> branch, so that people could try it, and so that improving it won't need 
> to post the entire patch each time anew.

Okay, will do.