[bug#39258] Faster guix search using an sqlite cache

[Ludovic Courtès]

Hello Arun!

Arun Isaac <address@hidden> skribis:

> From 4c883fcff1f44339b28df6ccdb2b10c906439e3d Mon Sep 17 00:00:00 2001
> From: Arun Isaac <address@hidden>
> Date: Tue, 21 Jan 2020 20:45:43 +0530
> Subject: [PATCH] fast search

[...]

> --- a/gnu/packages.scm
> +++ b/gnu/packages.scm
> @@ -43,6 +43,7 @@
>    #:use-module (srfi srfi-34)
>    #:use-module (srfi srfi-35)
>    #:use-module (srfi srfi-39)
> +  #:use-module (sqlite3)
>    #:export (search-patch
>              search-patches
>              search-auxiliary-file
> @@ -204,10 +205,8 @@ PROC is called along these lines:
>  PROC can use #:allow-other-keys to ignore the bits it's not interested in.
>  When a package cache is available, this procedure does not actually load any
>  package module."
> -  (define cache
> -    (load-package-cache (current-profile)))
> -
> -  (if (and cache (cache-is-authoritative?))
> +  (if (and (cache-is-authoritative?)
> +           (current-profile))
>        (vhash-fold (lambda (name vector result)
>                      (match vector
>                        (#(name version module symbol outputs
> @@ -220,7 +219,7 @@ package module."
>                               #:supported? supported?
>                               #:deprecated? deprecated?))))
>                    init
> -                  cache)
> +                  (cache-lookup (current-profile)))
>        (fold-packages (lambda (package result)
>                         (proc (package-name package)
>                               (package-version package)
> @@ -252,31 +251,7 @@ is guaranteed to never traverse the same package twice."
>  
>  (define %package-cache-file
>    ;; Location of the package cache.
> -  "/lib/guix/package.cache")
> -
> -(define load-package-cache

[...]

> +(define* (cache-lookup profile #:optional name)
>    "Lookup package NAME in CACHE.  Return a list sorted in increasing version
>  order."
>    (define (package-version<? v1 v2)
>      (version>? (vector-ref v2 1) (vector-ref v1 1)))
>  
> -  (sort (vhash-fold* cons '() name cache)
> -        package-version<?))
> +  (define (int->boolean n)
> +    (case n
> +      ((0) #f)
> +      ((1) #t)))
> +
> +  (define (string->list str)
> +    (call-with-input-string str read))
> +
> +  (define select-statement
> +    (string-append
> +     "SELECT name, version, module, symbol, outputs, supported, superseded, 
> locationFile, locationLine, locationColumn from packages"
> +     (if name " WHERE name = :name" "")))

I would rather keep the current package cache as-is instead of inserting
sqlite in here.  I don’t expect it to bring much compared
performance-wise to the current simple cache (especially if we look at
load time), and it does increase complexity quite a bit.

However, using sqlite for keyword search as you initially proposed on
guix-devel does sound like a great idea to me.

WDYT?

Thanks,
Ludo’.