bug#56637: 28.1.90; [FR] Allow a way around font-lock-mode being unconditionally disabled in " *hidden*" buffers

[Ihor Radchenko]

Eli Zaretskii <eliz@gnu.org> writes:

>> Is it always guaranteed by Emacs that font-lock-fontify-buffer correctly
>> fontifies that buffer even when font-lock-mode is disabled?
>
> IMO, if it doesn't, it's a bug that should be fixed.  If you have
> recipes where it happens, please submit a bug report.

Create a file called bug.el with the following text:

(with-current-buffer (get-buffer-create "Test")
  (emacs-lisp-mode)
  (insert "(message \"Foo\")")
  (font-lock-ensure)
  (message "%S" (buffer-string)))

Then, run
emacs-29-vcs -Q --script /tmp/bug.el

Observed output: #("(message \"Foo\")" 9 14 (face font-lock-string-face))
Expected: #("(message \"Foo\")" 0 9 (fontified t) 9 14 (face 
font-lock-string-face fontified t) 14 15 (fontified t))

The expected output appears in *Messages* when you run 
emacs-29-vcs -Q -l /tmp/bug.el

>> The need to compute fontification is not limited to the described Reddit
>> report. Org also needs fontification for export purposes - we copy the
>> Emacs code colours into exported documents. With current behaviour of
>> font-lock-mode, programmatic fontification may be problematic not only
>> when we use "hidden" buffers starting with space, but also in
>> non-interactive mode as in recent related bug report
>> https://orgmode.org/list/wxaUFiqi8BmIPv8pcYRVHAFa0hTzM35roQxpVVRkgddjRkesPGX1kVBL3G0fr42FonlRq5FNjapV8RiovXV-RGEDehXn-cmIebf4HWBhzIQ=@protonmail.com
>
> If you create buffer, process text in it, then write it out and kill
> the buffer, all in one go, there should be no reason to use a buffer
> whose name begins with a space, because interactive users will not
> have an opportunity to see such a buffer in the Emacs session.
>
> This problem is only real when a temporary buffer is left around
> because, for example, it is being collected/processed asynchronously,
> or in several stages with the user being able to interact with Emacs
> in-between.

Killing the buffer where we do fontification is too slow for Org.
If Org were to use temporary buffers, it would need to load major mode
for every text fragment to be fontified. Loading major modes (at least
some major modes) is taking significant amount of time and should
better be avoided if we can simply reuse a single buffer with major mode
that is already loaded.

>> As for not using buffers with names starting with a space, we do need
>> such buffers as means to fontify foreign major mode blocks inside Org.
>> How else do you suggest computing fontification of an arbitrary text in
>> arbitrary major-mode (not org-mode) without polluting the buffer list?
>
> Once again, if the buffer is killed before the user can see it,
> there's no problem to use a name which doesn't start with a space.
>
> But let me turn the table and ask you: how have you been doing this
> till now?

See the above.

Best,
Ihor