bug#56682: Fix the long lines font locking related slowdowns

[Gregory Heytings]

> > > Let's focus on making it easy to make it work well, rather than making 
> > > it impossible to make it work poorly.
> >
> > You lost me here.  I've read that sentence twenty times, and cannot 
> > understand what you mean.
>
> Your current code makes it impossible for a major mode to make Emacs 
> slow by widening in a too-long-line.

Which is a good thing, isn't it?  Or do you think that it's okay for Emacs 
to become unresponsive just because it is busy highlighting characters in 
the buffer?

> I'd prefer if we made it easy (i.e. the default) for Emacs to work well 
> in that case, without making it impossible for the major mode to mess 
> things up.

Sure, I would also welcome a better solution.  Until it materializes, the 
only reasonable way is to use a less optimal solution.

> For specific languages, you can use various heuristics to guess which 
> quotes start and which quotes end a string (for some languages you can 
> even do it reliably), but `syntax-ppss` handles all kinds of languages 
> (and doesn't have access to such heuristics currently), such as ELisp 
> where it's hard to do it well.

Don't worry, I've not yet seen an Elisp file with long lines.  If using 
various heuristics is sometimes or often feasible, that's already a good 
thing.

> I'd prefer to first see concrete examples where speeding up the 
> "syntax-ppss in a 1GB buffer" would make a significant difference to the 
> end-user's experience.

I just sent one such example to Dmitry.  And I pointed to another possible 
solution, namely to scan the whole buffer while opening it (instead of 
scanning it lazily, which is IIUC what currently happens).  From a user 
viewpoint, it's understandable that opening a big file takes some time.

> Then we can think about what's the better way to solve the problem 
> (which may be to just give up on font-lock altogether,

That would be regrettable, given the amount of effort that has been put 
into making font-lock work "as much as possible".

> or maybe to refine the `syntax.el` code (maybe move some of it to C), or 
> to speed up `parse-partial-sexp`, or maybe let major modes provide those 
> heuristics to find a "safe point" again (these used to exist, see 
> `syntax-begin-function`, for example, but they tended to suck)).

All this is possible.