Re: Quicklisp importer bug in tarball->extract->parse

[swedebugia]

Hi

Thanks a lot for taking the time to look at this.

On 2019-04-11 16:31, Pierre Neidhardt wrote:
> Hi!
> 
> I've had time to look at it a little.
> 
> If you want to debug, you can, for instance, put a breakpoint in
> guess-requirements-from-source, 

How exactly do you do this?
I looked in the guile manual and found the module (system vm trap-state)
and tried calling add-trap-at-procedure-call! like this:

(add-trap-at-procedure-call!

  (system* "tar" "xf" tarball "-C" dir "--wildcards" (string-append "*"
file-name)))))

But it failed with:
;;; (file-name "1am.asd")

Backtrace:

          13 (apply-smob/1 #<catch-closure 1480820>)

In ice-9/boot-9.scm:

    705:2 12 (call-with-prompt _ _ #<procedure default-prompt-handler (k
proc)>)
In ice-9/eval.scm:

    619:8 11 (_ #(#(#<directory (guile-user) 14de140>)))

In ice-9/boot-9.scm:

   2312:4 10 (save-module-excursion _)

  3831:12  9 (_)

In guix/import/quicklisp.scm:

    404:3  8 (_)

In guix/utils.scm:

    618:8  7 (call-with-temporary-output-file _)

    632:8  6 (call-with-temporary-directory _)

In guix/import/quicklisp.scm:

   374:32  5 (_ "/tmp/guix-directory.Z4chBm")

In srfi/srfi-1.scm:

   592:17  4 (map1 ("1am.asd"))

In guix/import/quicklisp.scm:

   380:41  3 (_ _)

In system/vm/trap-state.scm:

   213:15  2 (add-trap-at-procedure-call! 0 _)

In system/vm/traps.scm:

   137:45  1 (trap-at-procedure-call 0 #<procedure 3777e00 at
system/vm/trap-state.scm:128:2 (frame)> #:our-frame? _)

In unknown file:

           0 (scm-error misc-error #f "~A ~S" ("Not a VM program" 0) #f)



ERROR: In procedure scm-error:

Not a VM program 0


> There is nothing wrong with find-files: the "dir" directory is empty.
> This is because the extraction failed: 
> check the exit-code of system* before proceeding, you'll see what
> happens ;)
> 
> --8<---------------cut here---------------start------------->8---
> (system* "tar" "xf" tarball "-C" dir file-name)
> --8<---------------cut here---------------end--------------->8---
> 
> The above will error out if "file-name" is not found.  For 1am:
> 
> Queried file-name:
> 
>   "1am.asd"
> 
> Tarball actual file:
> 
>   "1am-20141106-git/1am.asd"
> 
> I don't think we can predict the root folder, and more generally
> projects have different trees.
> So I suggest we extract the whole tarball and proceed from there.  So
> your code should work by just removing "file-name":
> 
> --8<---------------cut here---------------start------------->8---
> (system* "tar" "xf" tarball "-C" dir)
> --8<---------------cut here---------------end--------------->8---
> 
> Makes sense?

Yes. I implemented Ricardos wildcard-suggestion:

(system* "tar" "xf" tarball "-C" dir "--wildcards" (string-append "*"
file-name))

> 
> Unrelated comment: If I'm not mistaken, you are fetching the meta-file
> and the index-file on every query.  I suggest you keep those in memories
> as soon as possible (memoize them), this will allow you to iterate much
> faster when developing.

I already tried by adding:


 (memoize

  (let* ( ;;(name "circular-streams")

         (name "1am")

         (source-url (ql-extract 'url name))

         (asd (ql-extract 'system-files name)))

    (peek 'asd asd)

    (call-with-temporary-output-file

     (lambda (temp port)

       (begin (url-fetch source-url temp)

              (guess-requirements-from-source source-url asd temp)

              )))))

and

(let* ( ;;(name "circular-streams")

       (name "1am")

       (source-url (memoize (ql-extract 'url name)))

       (asd (memoize (ql-extract 'system-files name))))

  (peek 'asd asd)

  (call-with-temporary-output-file

   (lambda (temp port)

     (begin (url-fetch source-url temp)

            (guess-requirements-from-source source-url asd temp)

            ))))

But it still fetches the index-file 2 times regardless where I put it.
Any ideas?

> It seems that you are on the right track, great job!

Thanks for the encouragement!

-- 
Cheers Swedebugia

signature.asc
Description: OpenPGP digital signature