[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile
From: |
zimoun |
Subject: |
[bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile |
Date: |
Wed, 27 Jan 2021 22:02:53 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux) |
Hi Ludo,
On Wed, 27 Jan 2021 at 14:14, Ludovic Courtès <ludo@gnu.org> wrote:
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>> Here’s a simple but oft-requested feature (I remember discussing
>> with Pierre and Simon a year ago at the Guix Days about ways to
>> implement it—time flies!): these patches provide ‘guix package
>> --export-manifest’ and ‘--export-channels’. These options spit
>> a manifest and a channel spec, respectively, with the goal of
>> helping users who wish to migrate to the declarative model.
>
> I’m rather happy with this patch set but since this is something we’ve
> discussed several times in the past, I think it’d be great if those
> interested could chime in and comment:
>
> https://issues.guix.gnu.org/45919
>
> I’ll leave a few more days and then… push!
You are looking at me? :-)
Well, the feature is nice! At first, I thought that it could be a bit
smarter than using only one commit. But as you said, the aim is for
transitioning. Maybe a future improvement should to list somewhere in
the comments which commit provides which set of packages. It could
help… or not. :-)
Using Docker, it works. Let describe how in case people are interested.
On machine A, I have:
--8<---------------cut here---------------start------------->8---
$ guix describe -f channels
(list (channel
(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git";)
(commit
"cb68ae668af2ade4b0777d82f227e5462768e9e5")
(introduction
(make-channel-introduction
"9edb3f66fd807b096b48283debdcddccfea34bad"
(openpgp-fingerprint
"BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA")))))
$ cat ~/.config/guix/manifests/python.scm
(specifications->manifest
(append
'("python"
)
(map
(lambda (pkg)
(string-append "python-" pkg))
'("ipython"
"numpy"
"matplotlib"
"scipy"
"biopython"
))))
--8<---------------cut here---------------end--------------->8---
And I generate a Docker pack with:
--8<---------------cut here---------------start------------->8---
$ guix pack -f docker --save-provenance -m ~/.config/guix/manifests/python.scm
/gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz
--8<---------------cut here---------------end--------------->8---
Then on machine B, after fetching this tarball, I run:
--8<---------------cut here---------------start------------->8---
$ docker load < /tmp/img/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz
$ docker images
REPOSITORY TAG IMAGE ID
CREATED SIZE
python-python-ipython-python-numpy latest 49ddfedf1e27 51
years ago 1.45GB
--8<---------------cut here---------------end--------------->8---
And it works as expected:
--8<---------------cut here---------------start------------->8---
$ docker run -ti python-python-ipython-python-numpy:latest python3
Python 3.8.2 (default, Jan 1 1970, 00:00:01)
[GCC 7.5.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters
Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
>>> import numpy as np
>>> A = np.array([[1,0,1],[0,1,0],[0,0,1]])
>>> _, s, _ = np.linalg.svd(A); s; abs(s[0] - 1./s[2])
array([1.61803399, 1. , 0.61803399])
0.0
>>>
--8<---------------cut here---------------end--------------->8---
Neat!
So far, so good. Well, let extract the ’manifest’ from this Docker
blob.
--8<---------------cut here---------------start------------->8---
$ docker export -o /tmp/img/re-pack.tar $(docker ps -a --format "{{.ID}}" |
head -n1)
$ tar -xf /tmp/img/re-pack.tar $(tar -tf /tmp/img/re-pack.tar | grep
'profile/manifest')
$ cat gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile/manifest | grep -E
"(\(\"python|cb68ae)" | head -n5
(("python"
"cb68ae668af2ade4b0777d82f227e5462768e9e5")
("python-ipython"
(("python-backcall"
("python-pyzmq"
--8<---------------cut here---------------end--------------->8---
Now, a trick to get the channels and specifications:
--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix package -p
/tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-channels
;; This channel file can be passed to 'guix pull -C' or to
;; 'guix time-machine -C' to obtain the Guix revision that was
;; used to populate this profile.
(list
(channel
(name 'guix)
(url "https://git.savannah.gnu.org/git/guix.git";)
(commit
"cb68ae668af2ade4b0777d82f227e5462768e9e5")
(introduction
(make-channel-introduction
"9edb3f66fd807b096b48283debdcddccfea34bad"
(openpgp-fingerprint
"BBB0 2DDF 2CEA F6A8 0D1D E643 A2A0 6DF2 A33A 54FA"))))
)
$ ./pre-inst-env guix package -p
/tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-manifest
$ ./pre-inst-env guix package -p
/tmp/img/gnu/store/7frdchgf5sqw8b83azsml3lw0h52gfbk-profile --export-manifest
;; This "manifest" file can be passed to 'guix package -m' to reproduce
;; the content of your profile. This is "symbolic": it only specifies
;; package names. To reproduce the exact same profile, you also need to
;; capture the channels being used, as returned by "guix describe".
;; See the "Replicating Guix" section in the manual.
(specifications->manifest
(list "python"
"python-ipython"
"python-numpy"
"python-matplotlib"
"python-scipy"
"python-biopython"))
--8<---------------cut here---------------end--------------->8---
Awesome!
The unexpected is this channels and manifests files do not reproduce the
same Docker pack tarball:
--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 99 Jan 05 2021 16:56:39 (current)
guix-past 829923f
repository URL: https://gitlab.inria.fr/guix-hpc/guix-past
branch: master
commit: 829923f01f894f1e687735627025ada26230832f
guix-bimsb a8b539d
repository URL: https://github.com/BIMSBbioinfo/guix-bimsb
branch: master
commit: a8b539d61a359060c35f3cb34c7edd1d9d14241d
bimsb-nonfree 4084e63
repository URL: https://github.com/BIMSBbioinfo/guix-bimsb-nonfree.git
branch: master
commit: 4084e63c9c0d662780870aded9f5a6ca1b063780
guix-science cf87b05
repository URL: https://github.com/guix-science/guix-science.git
branch: master
commit: cf87b0501c4a38b96edf41025a27bf1cb91f521a
guix 957f0c4
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: 957f0c40327ce00f53db22737e3775ce616ac258
$ guix time-machine -C /tmp/img/channels.scm -- pack -f docker
--save-provenance -m /tmp/img/manifest.scm
Updating channel 'guix' from Git repository at
'https://git.savannah.gnu.org/git/guix.git'...
/gnu/store/xzk604g8gysv4azn7sf9nylr6iah97gl-docker-pack.tar.gz
--8<---------------cut here---------------end--------------->8---
To compare with
/gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz.
On a third machine, I get:
/gnu/store/wxymmnxdvdvf08ifsfy39xjaxilhrigk-docker-pack.tar.gz
Well, that’s another story and I have not inspected yet the
derivations and what could be wrong on the machine B.
Cheers,
simon
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 1/8] profiles: Add 'manifest->code'., Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 2/8] utils: Add 'version-unique-prefix'., Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 3/8] guix package: Add '--export-manifest'., Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 5/8] channels: Add the channel name to channel sexps., Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 8/8] guix package: Add '--export-channels'., Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 4/8] channels: Factorize 'manifest-entry-channel' and channel serialization., Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 6/8] guix describe: Use 'manifest-entry-channel'., Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 7/8] channels: Add 'channel->code'., Ludovic Courtès, 2021/01/16
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, Ludovic Courtès, 2021/01/27
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile,
zimoun <=
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, Pierre Neidhardt, 2021/01/28
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, Pierre Neidhardt, 2021/01/29
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, zimoun, 2021/01/29
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, Pierre Neidhardt, 2021/01/29
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, zimoun, 2021/01/30
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, Ludovic Courtès, 2021/01/30
- [bug#45919] [PATCH 0/8] Exporting a manifest and channels from a profile, Ryan Prior, 2021/01/31