bug#56682: Fix the long lines font locking related slowdowns

[Stefan Monnier]

>> I'm not opposed to reducing the size of the text that's considered, but
>> doing it via narrowing is a blunt tool.
> It isn't.

There's no point arguing about it.  I find it to be and you don't and
that's that.  The reason why I find it to be is because it removes all
possibilities of making different choices for different elements
depending on the cost of those elements and the amount of
available information.

> The only way to make sure that the size of the text is small
> enough is a forced narrowing from which fontification-functions
> cannot escape.

It's clearly not the only way, nor is it an absolute guarantee.
I agree that it's a convenient way, tho.

> But let's try to be constructive.  You tell me that you're not opposed to
>  reducing the size of the text, and that font-lock.el could enforce
>  a smaller scope.  So could you please design a (Elisp) function (in
>  font-lock.el) which, given a (beg end) with beg <= end in a buffer, returns
>  a (beg' end') with beg <= beg' <= end' <= end that are better starting end
>  end points for the forced narrowing?  That function would be run in
>  handle_fontified_prop, before fontification-functions, and would have
> access to the whole buffer.

That can't be done.  What I think would be a better option is to
(somehow) pass the `beg..end` "limit" to jit-lock which can then pass it
on to its own clients (e.g. font-lock) so they each can make their
own choices.

E.g. the syntax-ppss part of the job performed by font-lock is heavily
cached, does not depends on lines, is theoretically always computed from
BOB but with a cache which makes it fast even when working near EOB (tho
it can still be somewhat slow when jumping from BOB to EOB, but that
depends on the size of the buffer, not the size of lines).  This part
*should* ignore your limits, which will make sure comments and strings are
recognized correctly at least in simple cases (i.e. cases which don't
depend on `syntax-propertize-function`).

>>> Think of it as POSIX's ulimits.
>> That's also a blunt tool.
> It isn't either.

Maybe we don't use the same meaning for "blunt".  What I mean is that
it's a tool whose effect cannot be fine-tuned for specific cases.
E.g. limiting the amount of memory used to store images, or the amount
of time spent in a particular operation, rather than applying those
limits to the whole process (even all its children as well).

> It's a practical way to limit what a single process can do

Yup, I use it too, but it's a one-size fits all.

Eli wrote:
> Feel free to suggest better ways of handling these issues, or even
> ways to solve this entirely inside font-lock.  If and when such
> suggestions materialize, I'm sure we will be glad to use them instead
> of less elegant/more direct solutions.

I'd suggest to keep things mostly as they are but move the decision to
ELisp: i.e. pass the beg..end limits to jit-lock and let jit-lock do
the narrowing.  This way it's easy to later refine the mechanism.


        Stefan