01/01: doc: Add "Guix Profiles in Practice" to the cookbook.

[guix-commits]

ambrevar pushed a commit to branch master
in repository guix.

commit 4c463569b7af3ee064d323691bc0284155f9a85d
Author: Pierre Neidhardt <address@hidden>
Date:   Thu Oct 24 11:14:23 2019 +0200

    doc: Add "Guix Profiles in Practice" to the cookbook.
    
    * doc/guix-cookbook.texi (Advanced package management): New chapter.
    * doc/guix-cookbook.texi (Guix Profiles in Practice): New section.
---
 doc/guix-cookbook.texi | 389 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 389 insertions(+)

diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index 66f94a0..3fee6b2 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -58,6 +58,7 @@ Translation Project}.
 * Scheme tutorials::            Meet your new favorite language!
 * Packaging::                   Packaging tutorials
 * System Configuration::        Customizing the GNU System
+* Advanced package management:: Power to the users!
 
 * Acknowledgments::             Thanks!
 * GNU Free Documentation License::  The license of this document.
@@ -779,6 +780,394 @@ kernel, since certain modules which are expected to be 
built may not be
 available for inclusion into the initrd.
 
 @c *********************************************************************
+@node Advanced package management
+@chapter Advanced package management
+
+Guix is a functional package manager that offers many features beyond
+what more traditional package managers can do.  To the uninitiated,
+those features might not have obvious use cases at first.  The purpose
+of this chapter is to demonstrate some advanced package management
+concepts.
+
+@pxref{Package Management,,, guix, GNU Guix Reference Manual} for a complete
+reference.
+
+@menu
+* Guix Profiles in Practice::     Strategies for multiple profiles and 
manifests.
+@end menu
+
+@node Guix Profiles in Practice
+@section Guix Profiles in Practice
+
+Guix provides a very useful feature that may be quite foreign to newcomers:
+@emph{profiles}.  They are a way to group package installations together and 
all users
+on a same system are free to use as many profiles as they want.
+
+Whether you're a developer or not, you may find that multiple profiles bring 
you
+great power and flexibility.  While they shift the paradigm somewhat compared 
to
+@emph{traditional package managers}, they are very convenient to use once 
you've
+understood how to set them up.
+
+If you are familiar with Python's @samp{virtualenv}, you can think of a 
profile as a
+kind of universal @samp{virtualenv} that can hold any kind of software 
whatsoever, not
+just Python software.  Furthermore, profiles are self-sufficient: they capture
+all the runtime dependencies which guarantees that all programs within a 
profile
+will always work at any point in time.
+
+Multiple profiles have many benefits:
+
+@itemize
+@item
+Clean semantic separation of the various packages a user needs for different 
contexts.
+
+@item
+Multiple profiles can be made available into the environment either on login
+or within a dedicated shell.
+
+@item
+Profiles can be loaded on demand.  For instance, the user can use multiple
+shells, each of them running different profiles.
+
+@item
+Isolation: Programs from one profile will not use programs from the other, and
+they user can even install different versions of the same programs to the two
+profiles without conflict.
+
+@item
+Deduplication: Profiles share dependencies that happens to be the exact same.
+This makes multiple profiles storage-efficient.
+
+@item
+Reproducible: when used with declarative manifests, a profile can be fully
+specified by the Guix commit that was active when it was set up.  This means
+that the exact same profile can be 
@uref{https://guix.gnu.org/blog/2018/multi-dimensional-transactions-and-rollbacks-oh-my/,
 set up anywhere, anytime}, with just the
+commit information.  See the section on @ref{Reproducible profiles}.
+
+@item
+Easier upgrades and maintenance: Multiple profiles make it easy to keep
+package listings at hand and make upgrades completely friction-less.
+@end itemize
+
+Concretely, here follows some typical profiles:
+
+@itemize
+@item
+The dependencies of a project you are working on.
+
+@item
+Your favourite programming language libraries.
+
+@item
+Laptop-specific programs (like @samp{powertop}) that you don't need on a 
desktop.
+
+@item
+@TeX{}live (this one can be really useful when you need to install just one
+package for this one document you've just received over email).
+
+@item
+Games.
+@end itemize
+
+Let's dive in the set up!
+
+@node Basic setup with manifests
+@subsection Basic setup with manifests
+
+A Guix profile can be set up @emph{via} a so-called @emph{manifest 
specification} that looks like
+this:
+
+@example
+(specifications->manifest
+  '("package-1"
+    ;; Version 1.3 of package-2.
+    "package-2@@1.3"
+    ;; The "lib" output of package-3.
+    "package-3:lib"
+    ; ...
+    "package-N"))
+@end example
+
+See @pxref{Invoking guix package,,, guix, GNU Guix Reference Manual} for
+the syntax details.
+
+We can create a manifest specification per profile and install them this way:
+
+@example
+GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
+mkdir -p "$GUIX_EXTRA_PROFILES"/my-project # if it does not exist yet
+guix package --manifest=/path/to/guix-my-project-manifest.scm 
--profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
+@end example
+
+Here we set an arbitrary variable @samp{GUIX_EXTRA_PROFILES} to point to the 
directory
+where we will store our profiles in the rest of this article.
+
+Placing all your profiles in a single directory, with each profile getting its
+own sub-directory, is somewhat cleaner.  This way, each sub-directory will
+contain all the symlinks for precisely one profile.  Besides, "looping over
+profiles" becomes obvious from any programming language (e.g. a shell script) 
by
+simply looping over the sub-directories of @samp{$GUIX_EXTRA_PROFILES}.
+
+Note that it's also possible to loop over the output of
+
+@example
+guix package --list-profiles
+@end example
+
+although you'll probably have to filter out @samp{~/.config/guix/current}.
+
+To enable all profiles on login, add this to your @samp{~/.bash_profile} (or 
similar):
+
+@example
+for i in $GUIX_EXTRA_PROFILES/*; do
+  profile=$i/$(basename "$i")
+  if [ -f "$profile"/etc/profile ]; then
+    GUIX_PROFILE="$profile"
+    . "$GUIX_PROFILE"/etc/profile
+  fi
+  unset profile
+done
+@end example
+
+Note to Guix System users: the above reflects how your default profile
+@samp{~/.guix-profile} is activated from @samp{/etc/profile}, that latter 
being loaded by
+@samp{~/.bashrc} by default.
+
+You can obviously choose to only enable a subset of them:
+
+@example
+for i in "$GUIX_EXTRA_PROFILES"/my-project-1 
"$GUIX_EXTRA_PROFILES"/my-project-2; do
+  profile=$i/$(basename "$i")
+  if [ -f "$profile"/etc/profile ]; then
+    GUIX_PROFILE="$profile"
+    . "$GUIX_PROFILE"/etc/profile
+  fi
+  unset profile
+done
+@end example
+
+When a profile is off, it's straightforward to enable it for an individual 
shell
+without "polluting" the rest of the user session:
+
+@example
+GUIX_PROFILE="path/to/my-project" ; . "$GUIX_PROFILE"/etc/profile
+@end example
+
+The key to enabling a profile is to @emph{source} its @samp{etc/profile} file. 
 This file
+contains shell code that exports the right environment variables necessary to
+activate the software contained in the profile.  It is built automatically by
+Guix and meant to be sourced.
+It contains the same variables you would get if you ran:
+
+@example
+guix package --search-paths=prefix --profile=$my_profile"
+@end example
+
+Once again, see (@pxref{Invoking guix package,,, guix, GNU Guix Reference 
Manual})
+for the command line options.
+
+To upgrade a profile, simply install the manifest again:
+
+@example
+guix package -m /path/to/guix-my-project-manifest.scm -p 
"$GUIX_EXTRA_PROFILES"/my-project/my-project
+@end example
+
+To upgrade all profiles, it's easy enough to loop over them.  For instance,
+assuming your manifest specifications are stored in
+@samp{~/.guix-manifests/guix-$profile-manifest.scm}, with @samp{$profile} 
being the name
+of the profile (e.g. "project1"), you could do the following in Bourne shell:
+
+@example
+for profile in "$GUIX_EXTRA_PROFILES"/*; do
+  guix package --profile="$profile" 
--manifest="$HOME/.guix-manifests/guix-$profile-manifest.scm"
+done
+@end example
+
+Each profile has its own generations:
+
+@example
+guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project --list-generations
+@end example
+
+You can roll-back to any generation of a given profile:
+
+@example
+guix package -p "$GUIX_EXTRA_PROFILES"/my-project/my-project 
--switch-generations=17
+@end example
+
+@node Required packages
+@subsection Required packages
+
+Activating a profile essentially boils down to exporting a bunch of
+environmental variables.  This is the role of the @samp{etc/profile} within the
+profile.
+
+@emph{Note: Only the environmental variables of the packages that consume them 
will
+be set.}
+
+For instance, @samp{MANPATH} won't be set if there is no consumer application 
for man
+pages within the profile.  So if you need to transparently access man pages 
once
+the profile is loaded, you've got two options:
+
+@itemize
+@item
+Either export the variable manually, e.g.
+@example
+export MANPATH=/path/to/profile$@{MANPATH:+:@}$MANPATH"
+@end example
+
+@item
+Or include @samp{man-db} to the profile manifest.
+@end itemize
+
+The same is true for @samp{INFOPATH} (you can install @samp{info-reader}),
+@samp{PKG_CONFIG_PATH} (install @samp{pkg-config}), etc.
+
+@node Default profile
+@subsection Default profile
+
+What about the default profile that Guix keeps in @samp{~/.guix-profile}?
+
+You can assign it the role you want.  Typically you would install the manifest
+of the packages you want to use all the time.
+
+Alternatively, you could keep it "manifest-less" for throw-away packages
+that you would just use for a couple of days.
+This way makes it convenient to run
+
+@example
+guix install package-foo
+guix upgrade package-bar
+@end example
+
+without having to specify the path to a profile.
+
+@node The benefits of manifests
+@subsection The benefits of manifests
+
+Manifests are a convenient way to keep your package lists around and, say,
+to synchronize them across multiple machines using a version control system.
+
+A common complaint about manifests is that they can be slow to install when 
they
+contain large number of packages.  This is especially cumbersome when you just
+want get an upgrade for one package within a big manifest.
+
+This is one more reason to use multiple profiles, which happen to be just
+perfect to break down manifests into multiple sets of semantically connected
+packages.  Using multiple, small profiles provides more flexibility and
+usability.
+
+Manifests come with multiple benefits.  In particular, they ease maintenance:
+
+@itemize
+@item
+When a profile is set up from a manifest, the manifest itself is
+self-sufficient to keep a "package listing" around and reinstall the profile
+later or on a different system.  For ad-hoc profiles, we would need to
+generate a manifest specification manually and maintain the package versions
+for the packages that don't use the default version.
+
+@item
+@code{guix package --upgrade} always tries to update the packages that have
+propagated inputs, even if there is nothing to do.  Guix manifests remove this
+problem.
+
+@item
+When partially upgrading a profile, conflicts may arise (due to diverging
+dependencies between the updated and the non-updated packages) and they can be
+annoying to resolve manually.  Manifests remove this problem altogether since
+all packages are always upgraded at once.
+
+@item
+As mentioned above, manifests allow for reproducible profiles, while the
+imperative @code{guix install}, @code{guix upgrade}, etc. do not, since they 
produce
+different profiles every time even when they hold the same packages.  See
+@uref{https://issues.guix.gnu.org/issue/33285, the related discussion on the 
matter}.
+
+@item
+Manifest specifications are usable by other @samp{guix} commands.  For 
example, you
+can run @code{guix weather -m manifest.scm} to see how many substitutes are
+available, which can help you decide whether you want to try upgrading today
+or wait a while.  Another example: you can run @code{guix pack -m 
manifest.scm} to
+create a pack containing all the packages in the manifest (and their
+transitive references).
+
+@item
+Finally, manifests have a Scheme representation, the @samp{<manifest>} record 
type.
+They can be manipulated in Scheme and passed to the various Guix 
@uref{https://en.wikipedia.org/wiki/Api, APIs}.
+@end itemize
+
+It's important to understand that while manifests can be used to declare
+profiles, they are not strictly equivalent: profiles have the side effect that
+they "pin" packages in the store, which prevents them from being
+garbage-collected (@pxref{Invoking guix gc,,, guix, GNU Guix Reference Manual})
+and ensures that they will still be available at any point in
+the future.
+
+Let's take an example:
+
+@enumerate
+@item
+We have an environment for hacking on a project for which there isn't a Guix
+package yet.  We build the environment using a manifest, and then run 
@code{guix
+   environment -m manifest.scm}.  So far so good.
+
+@item
+Many weeks pass and we have run a couple of @code{guix pull} in the mean time.
+Maybe a dependency from our manifest has been updated; or we may have run
+@code{guix gc} and some packages needed by our manifest have been
+garbage-collected.
+
+@item
+Eventually, we set to work on that project again, so we run @code{guix 
environment
+   -m manifest.scm}.  But now we have to wait for Guix to build and install
+stuff!
+@end enumerate
+
+Ideally, we could spare the rebuild time.  And indeed we can, all we need is to
+install the manifest to a profile and use @code{GUIX_PROFILE=/the/profile;
+. "$GUIX_PROFILE"/etc/profile} as explained above: this guarantees that our
+hacking environment will be available at all times.
+
+@emph{Security warning:} While keeping old profiles around can be convenient, 
keep in
+mind that outdated packages may not have received the latest security fixes.
+
+@node Reproducible profiles
+@subsection Reproducible profiles
+
+To reproduce a profile bit-for-bit, we need two pieces of information:
+
+@itemize
+@item
+a manifest,
+@item
+a Guix channel specification.
+@end itemize
+
+Indeed, manifests alone might not be enough: different Guix versions (or
+different channels) can produce different outputs for a given manifest.
+
+You can output the Guix channel specification with @samp{guix describe
+--format=channels}.
+Save this to a file, say @samp{channel-specs.scm}.
+
+On another computer, you can use the channel specification file and the 
manifest
+to reproduce the exact same profile:
+
+@example
+GUIX_EXTRA_PROFILES=$HOME/.guix-extra-profiles
+GUIX_EXTRA=$HOME/.guix-extra
+
+mkdir "$GUIX_EXTRA"/my-project
+guix pull --channels=channel-specs.scm --profile "$GUIX_EXTRA/my-project/guix"
+
+mkdir -p "$GUIX_EXTRA_PROFILES/my-project"
+"$GUIX_EXTRA"/my-project/guix/bin/guix package 
--manifest=/path/to/guix-my-project-manifest.scm 
--profile="$GUIX_EXTRA_PROFILES"/my-project/my-project
+@end example
+
+It's safe to delete the Guix channel profile you've just installed with the
+channel specification, the project profile does not depend on it.
+
+@c *********************************************************************
 @node Acknowledgments
 @chapter Acknowledgments