bug#50946: insert-file-contents can corrupt buffers. [Was: bug#50946: Emacs-28: Inadequate coding in hack-elisp-shorthands]

[Eli Zaretskii]

> Date: Sun, 3 Oct 2021 12:10:19 +0000
> Cc: joaotavora@gmail.com, 50946@debbugs.gnu.org
> From: Alan Mackenzie <acm@muc.de>
> 
> Create a file ~/utf8-chars.txt as follows.  All the non-ascii characters
> are 2-byte German UTF8 characters.  Only the Q is an ascii character.
> There is a LF at the end:
> 
> ÄäÖöQÜüß
> 
> Now, in an empty buffer,
> 
>    M-: (insert-file-contents "~/utf8-chars.txt" nil 3 15)
> 
> ..  The first character of this buffer is now the Emacs encoding of the
> raw byte 0xa4.
> 
> Now do
> 
>    M-: (insert-file-contents "~/utf8-chars.txt" nil 0 3)
> 
> The entire buffer, apart from the Q and the LF, now consists of raw
> bytes, and the buffer is now 16 characters long.  (Is this a bug?).
> Note that the Q is now further back from the end of the buffer than it
> should be.

OK, thanks.

> My opinion, for what it's worth, is that using insert-file-contents in
> hack-elisp-shorthands is a Bad Thing.  Even if it is possible to get it
> working rigorously, it is surely not worth the trouble.  Why not simply
> visit the file in a buffer, and check for buffer local variables in the
> normal fashion?

We already visit the file when we load it.

João, why didn't you simply insert

  (alist-get 'elisp-shorthands (hack-local-variables--find-variables))

in load-with-code-conversion, immediately after it calls
insert-file-contents?  Are there any problems with that, and if so,
what are they?

> There are bugs in the documentation of insert-file-contents in the elisp
> manual.  It confuses bytes with characters, and it fails to mention the
> need to keep BEG and END at character boundaries.  I propose installing
> the following patch to the release branch:

Thanks, I will review this later.  However:

> @@ -580,7 +583,8 @@ Reading from Files
>  This function works like @code{insert-file-contents} except that it
>  does not run @code{after-insert-file-functions}, and does not do
>  format decoding, character code conversion, automatic uncompression,
> -and so on.
> +and so on.  @var{beg} and @var{end}, if non-@code{nil}, should be at
> +character boundaries, as in @code{insert-file-contents}.
>  @end defun

I don't think I understand why you made this second correction:
insert-file-contents-literally deals with bytes to begin with.

> The doc strings of insert-file-contents\(-literally\)? will also need to
> be updated.

In some sense, yes.