emacs-bug-tracker
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[debbugs-tracker] bug#35813: closed ([PATCH] import: crate: add recursiv


From: GNU bug Tracking System
Subject: [debbugs-tracker] bug#35813: closed ([PATCH] import: crate: add recursive option)
Date: Sun, 13 Oct 2019 07:44:02 +0000

Your message dated Sun, 13 Oct 2019 00:42:22 -0700
with message-id <CAAc=MEx=2ja4qba-dms=address@hidden>
and subject line Re: [bug#35813] [PATCH] Add crate-recursive-import.
has caused the debbugs.gnu.org bug report #35813,
regarding [PATCH] import: crate: add recursive option
to be marked as done.

(If you believe you have received this mail in error, please contact
address@hidden.)


-- 
35813: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=35813
GNU Bug Tracking System
Contact address@hidden with problems
--- Begin Message --- Subject: [PATCH] import: crate: add recursive option Date: Mon, 20 May 2019 19:23:06 +0100
* guix/script/import/crate.scm: Add recursive option.
* guix/import/crate.scm (crate-recursive-import): New variable.
---
 doc/guix.texi                 |  8 ++++++++
 guix/import/crate.scm         | 24 ++++++++++++++++--------
 guix/scripts/import/crate.scm | 27 ++++++++++++++++++++++-----
 3 files changed, 46 insertions(+), 13 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index ae9ad0739e..636bb7521d 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -8787,6 +8787,14 @@ in Guix.
 Import metadata from the crates.io Rust package repository
 @uref{https://crates.io, crates.io}.
 
+@table @code
+@item --recursive
+@itemx -r
+Traverse the dependency graph of the given upstream package recursively
+and generate package expressions for all those packages that are not yet
+in Guix.
+@end table
+
 @item opam
 @cindex OPAM
 @cindex OCaml
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index e0b400d054..d8554b0e7a 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -37,6 +37,7 @@
   #:use-module (srfi srfi-26)
   #:export (crate->guix-package
             guix-package->crate-name
+            crate-recursive-import
             %crate-updater))
 
 (define (crate-fetch crate-name callback)
@@ -86,8 +87,8 @@
 VERSION, INPUTS, NATIVE-INPUTS, HOME-PAGE, SYNOPSIS, DESCRIPTION, and LICENSE."
   (let* ((port (http-fetch (crate-uri name version)))
          (guix-name (crate-name->package-name name))
-         (inputs (map crate-name->package-name inputs))
-         (native-inputs (map crate-name->package-name native-inputs))
+         (input-packages (map crate-name->package-name inputs))
+         (native-input-packages (map crate-name->package-name native-inputs))
          (pkg `(package
                    (name ,guix-name)
                    (version ,version)
@@ -99,8 +100,8 @@ VERSION, INPUTS, NATIVE-INPUTS, HOME-PAGE, SYNOPSIS, 
DESCRIPTION, and LICENSE."
                               (base32
                                ,(bytevector->nix-base32-string (port-sha256 
port))))))
                    (build-system cargo-build-system)
-                   ,@(maybe-native-inputs native-inputs "src")
-                   ,@(maybe-inputs inputs "src")
+                   ,@(maybe-native-inputs native-input-packages "src")
+                   ,@(maybe-inputs input-packages "src")
                    (home-page ,(match home-page
                                  (() "")
                                  (_ home-page)))
@@ -111,12 +112,14 @@ VERSION, INPUTS, NATIVE-INPUTS, HOME-PAGE, SYNOPSIS, 
DESCRIPTION, and LICENSE."
                                ((license) license)
                                (_ `(list ,@license)))))))
          (close-port port)
-         pkg))
+         (values pkg inputs)))
 
-(define (crate->guix-package crate-name)
-  "Fetch the metadata for CRATE-NAME from crates.io, and return the
+(define crate->guix-package
+  (memoize
+   (lambda* (crate-name _)
+     "Fetch the metadata for CRATE-NAME from crates.io, and return the
 `package' s-expression corresponding to that package, or #f on failure."
-  (crate-fetch crate-name make-crate-sexp))
+     (crate-fetch crate-name make-crate-sexp))))
 
 (define (guix-package->crate-name package)
   "Return the crate name of PACKAGE."
@@ -158,6 +161,11 @@ VERSION, INPUTS, NATIVE-INPUTS, HOME-PAGE, SYNOPSIS, 
DESCRIPTION, and LICENSE."
      (version version)
      (urls (list url)))))
 
+(define* (crate-recursive-import package-name)
+  (recursive-import package-name #f
+                    #:repo->guix-package crate->guix-package
+                    #:guix-name crate-name->package-name))
+
 (define %crate-updater
   (upstream-updater
    (name 'crates)
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index cab9a4397b..8fadcdd57c 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -27,6 +27,7 @@
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-37)
+  #:use-module (srfi srfi-41)
   #:use-module (ice-9 match)
   #:use-module (ice-9 format)
   #:export (guix-import-crate))
@@ -45,6 +46,8 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
   (display (G_ "
   -h, --help             display this help and exit"))
   (display (G_ "
+  -r, --recursive        import packages recursively"))
+  (display (G_ "
   -V, --version          display version information and exit"))
   (newline)
   (show-bug-report-information))
@@ -58,6 +61,9 @@ Import and convert the crate.io package for PACKAGE-NAME.\n"))
          (option '(#\V "version") #f #f
                  (lambda args
                    (show-version-and-exit "guix import crate")))
+         (option '(#\r "recursive") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'recursive #t result)))
          %standard-import-options))
 
 
@@ -83,11 +89,22 @@ Import and convert the crate.io package for 
PACKAGE-NAME.\n"))
                            (reverse opts))))
     (match args
       ((package-name)
-       (let ((sexp (crate->guix-package package-name)))
-         (unless sexp
-           (leave (G_ "failed to download meta-data for package '~a'~%")
-                  package-name))
-         sexp))
+       (if (assoc-ref opts 'recursive)
+           ;; Recursive import
+           (map (match-lambda
+                  ((and ('package ('name name) . rest) pkg)
+                   `(define-public ,(string->symbol name)
+                      ,pkg))
+                  (_ #f))
+                (reverse
+                 (stream->list
+                  (crate-recursive-import package-name))))
+           ;; Single import
+           (let ((sexp (crate->guix-package package-name #f)))
+             (unless sexp
+               (leave (G_ "failed to download meta-data for package '~a'~%")
+                      package-name))
+             sexp)))
       (()
        (leave (G_ "too few arguments~%")))
       ((many ...)
-- 
2.21.0




--- End Message ---
--- Begin Message --- Subject: Re: [bug#35813] [PATCH] Add crate-recursive-import. Date: Sun, 13 Oct 2019 00:42:22 -0700
Closing this since someone else already merged their own crate recursive importer.

On Sun, Sep 8, 2019 at 12:57 AM Efraim Flashner <address@hidden> wrote:
As a simple test I ran 'guix import crate encoding -r' and it gave me
the 6 packages I expected. 'guix import crate winapi -r' only gave me
rust-winapi, as the dependent crates are already packaged. When I tried
'guix import crate rand -r' it found the updated version and started
importing all the new dependencies also.

It looks good. I'm tempted to leave it running with 'guix import crate
serde -r' just to see if we're ever going to make it there.

I see that it imports A then B then C then D, and prints out D then C
then B then A. For the one I tested with is still rust-encoding. I'll
try my hand at ascii art:

                            encoding
                                |
  -------------------------------------------------------
  |         |               |           |               |
japanese  korean      simpchinese    singlebyte     tradchinese
  |         |               |           |               |
  -------------------------------------------------------
                        |
                 encoding-tests

import went encoding, japanese, tests, korean, simpchinese, singlebyte,
tradchinese

I think the only thing I would wish for would be to do tests, then the
languages and then encoding (best for upstreaming one at a time), or to
do them alphabetically (plop them in alphabetically all at once). This
I'm happy to live without I think.

The other thing was I ran 'guix import crate security-framework -r' and
after ~40 crates it crashed on me with:
    web/http.scm:1186:15: In procedure read-response-line:
    Bad Read-Header-Line header: #<eof>
and I would prefer to have the ~40 crates it did grab first to be
printed out and not lost. Between these two I would like most to not
lose the imported crates than worrying over the printed order.

Great job! From what I've tested I think it's ready as-is and any
changes would just be gravy.


--
Efraim Flashner   <address@hidden>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

--- End Message ---

reply via email to

[Prev in Thread] Current Thread [Next in Thread]