bug#63716: closed (30.0.50; Unable to install external packages with Nix after recent native-compilation changes)

[GNU bug Tracking System]

Your message dated Mon, 5 Jun 2023 12:06:55 +0200
with message-id 
<CAO48Bk-pK3Vtc5VP7QWF8nqdRJ7uq4abegRpA74eZ3QbuiLZDQ@mail.gmail.com>
and subject line Re: bug#63710: use-package not working
has caused the debbugs.gnu.org bug report #63710,
regarding 30.0.50; Unable to install external packages with Nix after recent 
native-compilation changes
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs@gnu.org.)


-- 
63710: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=63710
GNU Bug Tracking System
Contact help-debbugs@gnu.org with problems
> --- Begin Message ---
>
>
>
> Subject: 
>
> 30.0.50; Unable to install external packages with Nix after recent native-compilation changes
>
>
>
>
> Date: 
>
> Thu, 25 May 2023 11:12:58 +0000
>
>
>
>
>
> Starting with the recent batch of native-comp commits ending with 
> and including Emacs commit 
> b2b2be98da5825e6bc130999ffe38f5ed615586a, using the Nix package 
> manager I cannot install an Emacs containing natively-compiled 
> external packages. (I have no problems on Emacs commit 
> f4de81af8fc54ef278cdb76fbc5885ed7d05b2d7, which is the commit 
> immediately preceding the first of the recent native-comp commits)
>
> Building core Emacs (with native compilation) is successful, but I 
> encounter problems upon running 
> (https://github.com/nixos/nixpkgs/blob/8fbdc4e6114795323ad2b74d13e1c842c5618b78/pkgs/build-support/emacs/melpa.nix#L99-L102):
>
> >    emacs --batch -Q \
> >        -l "$elpa2nix" \
> >        -f elpa2nix-install-package \
> >        "$archive" "$out/share/emacs/site-lisp/elpa"
>
> This fails when trying to run package-tar-file-info 
> (https://github.com/nixos/nixpkgs/blob/8fbdc4e6114795323ad2b74d13e1c842c5618b78/pkgs/build-support/emacs/elpa2nix.el#L14-L15). The 
> full backtrace is here:
>
> >    Error: error ("Can't find define-package in 
> >    vterm-20230417.424/vterm-pkg.el")
> >    mapbacktrace(#f(compiled-function (evald func args flags) 
> >    #<bytecode -0xe76c55f6510e41>))
> >    debug-early-backtrace()
> >    debug-early(error (error "Can't find define-package in 
> >    vterm-20230417.424/vterm-pkg.el"))
> >    error("Can't find define-package in %s" 
> >    "vterm-20230417.424/vterm-pkg.el")
> >    package-tar-file-info()
> >    (if (derived-mode-p 'tar-mode) (package-tar-file-info) 
> >    (package-buffer-info))
> >    (let ((pkg-desc (if (derived-mode-p 'tar-mode) 
> >    (package-tar-file-info) (package-buffer-info)))) 
> >    (package-unpack pkg-desc) pkg-desc)
> >    elpa2nix-install-from-buffer()
> >    (progn (if is-tar (insert-file-contents-literally file) 
> >    (insert-file-contents file)) (if is-tar (progn (tar-mode))) 
> >    (elpa2nix-install-from-buffer))
> >    (unwind-protect (progn (if is-tar 
> >    (insert-file-contents-literally file) (insert-file-contents 
> >    file)) (if is-tar (progn (tar-mode))) 
> >    (elpa2nix-install-from-buffer)) (and (buffer-name 
> >    temp-buffer) (kill-buffer temp-buffer)))
> >    (save-current-buffer (set-buffer temp-buffer) (unwind-protect 
> >    (progn (if is-tar (insert-file-contents-literally file) 
> >    (insert-file-contents file)) (if is-tar (progn (tar-mode))) 
> >    (elpa2nix-install-from-buffer)) (and (buffer-name 
> >    temp-buffer) (kill-buffer temp-buffer))))
> >    (let ((temp-buffer (generate-new-buffer " *temp*" t))) 
> >    (save-current-buffer (set-buffer temp-buffer) (unwind-protect 
> >    (progn (if is-tar (insert-file-contents-literally file) 
> >    (insert-file-contents file)) (if is-tar (progn (tar-mode))) 
> >    (elpa2nix-install-from-buffer)) (and (buffer-name 
> >    temp-buffer) (kill-buffer temp-buffer)))))
> >    (let ((is-tar (string-match "\\.tar\\'" file))) (let 
> >    ((temp-buffer (generate-new-buffer " *temp*" t))) 
> >    (save-current-buffer (set-buffer temp-buffer) (unwind-protect 
> >    (progn (if is-tar (insert-file-contents-literally file) 
> >    (insert-file-contents file)) (if is-tar (progn (tar-mode))) 
> >    (elpa2nix-install-from-buffer)) (and (buffer-name 
> >    temp-buffer) (kill-buffer temp-buffer))))))
> >    elpa2nix-install-file("/build/packages/vterm-20230417.424.tar")
> >    (progn (setq package-user-dir elpa) (elpa2nix-install-file 
> >    archive))
> >    (let ((archive x0) (elpa x2)) (progn (setq package-user-dir 
> >    elpa) (elpa2nix-install-file archive)))
> >    (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq 
> >    package-user-dir elpa) (elpa2nix-install-file archive))))
> >    (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) (if (null x3) 
> >    (let ((archive x0) (elpa x2)) (progn (setq package-user-dir 
> >    elpa) (elpa2nix-install-file archive)))))
> >    (if (consp x1) (let* ((x2 (car-safe x1)) (x3 (cdr-safe x1))) 
> >    (if (null x3) (let ((archive x0) (elpa x2)) (progn (setq 
> >    package-user-dir elpa) (elpa2nix-install-file archive))))))
> >    (let* ((x0 (car-safe command-line-args-left)) (x1 (cdr-safe 
> >    command-line-args-left))) (if (consp x1) (let* ((x2 (car-safe 
> >    x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) 
> >    (elpa x2)) (progn (setq package-user-dir elpa) 
> >    (elpa2nix-install-file archive)))))))
> >    (if (consp command-line-args-left) (let* ((x0 (car-safe 
> >    command-line-args-left)) (x1 (cdr-safe 
> >    command-line-args-left))) (if (consp x1) (let* ((x2 (car-safe 
> >    x1)) (x3 (cdr-safe x1))) (if (null x3) (let ((archive x0) 
> >    (elpa x2)) (progn (setq package-user-dir elpa) 
> >    (elpa2nix-install-file archive))))))))
> >    elpa2nix-install-package()
> >    command-line-1(("-l" 
> >    "/nix/store/4jj63z4v1xp13rh2md053dccq920hd45-elpa2nix.el" 
> >    "-f" "elpa2nix-install-package" 
> >    "/build/packages/vterm-20230417.424.tar" 
> >    "/nix/store/rf3rsbn6sa1l5v8pbsk02g0nrm5a7pnr-emacs-vterm-20230417.424/share/emacs/site-lisp/elpa"))
> >    command-line()
> >    normal-top-level()
>
> I do not encounter any problems if I do not native-compile, or if 
> I use an earlier commit as I mentioned.
>
> Unlike the possibly-related and unresolved bug#63288, the error I 
> am now getting appears to be consistently reproducible when I 
> revert ea9831bb3cb4878273f6f848051c9b8c3c76d5f1, which in a Nix 
> sandbox would non-deterministically cause external packages to 
> fail during native compilation. I have also noticed the same error 
> occurring when I leave ea9831bb3cb4878273f6f848051c9b8c3c76d5f1 
> unreverted, though I haven't tested this latter scenario often 
> enough to be sure that ea9831bb3cb4878273f6f848051c9b8c3c76d5f1 
> has no impact since I was worried that the non-determinism would 
> complicate things.
>
> CCing Andrea Corallo.
>
>
>
> --- End Message ---

--- Begin Message --- Subject: Re: bug#63710: use-package not working Date: Mon, 5 Jun 2023 12:06:55 +0200

HI,

I was going to suggest it. I have rebuilt emacs w/native today and it seems to be stable. 

Thanks! /PA

On Mon, 5 Jun 2023 at 10:36, Andrea Corallo <acorallo@gnu.org> wrote:

Andrea Corallo <acorallo@gnu.org> writes:

> Andrea Corallo <akrl@sdf.org> writes:
>
>> Andrea Corallo <akrl@sdf.org> writes:
>>
>>> Pedro Andres Aranda Gutierrez <paaguti@gmail.com> writes:
>>>
>>>> First: the .emacs.d/init.el test file
>>>>
>>>> ------
>>>> ;;
>>>> ;; My .emacs file
>>>> ;;
>>>> (require 'package)
>>>> (setq package-archives '(("melpa" . "https://melpa.org/packages/")
>>>> ("gnu" . "https://elpa.gnu.org/packages/")
>>>>                          ("melpa-stable" . "https://stable.melpa.org/packages/")
>>>> ("nongnu" . "https://elpa.nongnu.org/nongnu/")))
>>>> (eval-when-compile
>>>>   (require 'use-package))
>>>> (unless (file-directory-p package-user-dir)
>>>>     (package-refresh-contents))
>>>> (require 'use-package-ensure)
>>>> (setq use-package-always-ensure t)
>>>>
>>>> (use-package "compat")
>>>> ------
>>>>
>>>> Second: 1d5b164109b59559d34c545c2a163fa067ca22b2 does work
>>>>
>>>> BTW,  it has to do with native compilation. I have accidentally not deleted the eln-cache in on of the tests and the
>>>> 'wrong' Emacs has started correctly:
>>>> 1. started 1d5b164109b59559d34c545c2a163fa067ca22b2 with eln-clean cache ---> OK
>>>> 2. started b2b2be98da5825e6bc130999ffe38f5ed615586a with eln-cache from the last start --> OK
>>>> 3. started b2b2be98da5825e6bc130999ffe38f5ed615586a with clean eln-cache ---> KO
>>>
>>>>From these SHA1s looks like there's good chance it's my fault...
>>>
>>> I'll have a look.
>>
>> Okay I did some investigation and understood the issue.
>>
>> `package-load-descriptor' was misscompiled and returning always nil as
>> the compiler was not fully picking up the user defined types at compile
>> time, it worked only after the first faulty compilation (thing that made
>> the investigaiton a bit slower).
>>
>> I'm working on a more complete fix but this requires careful testing, I
>> pushed now c8e527cbeee in order to have master stable again while I keep
>> working on the most comprehensive fix.
>
> Okay RE-sending from this account and ditching my sdf.org probably
> forever as I'm getting a bunch of bounces classifying it again as spam:
>
> Hi all,
>
> I pushed what I think is the final fix and reverted the temporary one.
>
> As of fbc96dcc4ea I'm not aware of any issue with package.el or other
> code.
>
> Please let us know if works for you and we can close the bug or, if any
> other issue arise (please remember to clean the eln-cache before testing
> as it may contain misscompiled code ;).

Okay one week passed since the fix was pushed and no more issues were
reported on this, so I'm closing it.  Happy to reopen in case.

BR

  Andrea

--

Fragen sind nicht da, um beantwortet zu werden,

Fragen sind da um gestellt zu werden

Georg Kreisler

Headaches with a Juju log:

unit-basic-16: 09:17:36 WARNING juju.worker.uniter.operation we should run a leader-deposed hook here, but we can't yet

--- End Message ---