Re: [Chicken-hackers] new CHICKEN 5 egg installation

[felix . winkelmann]

> > * When a test in a depency fails, the installation process still continues,
> >   should the whole operation abort?
>
> Is this related to the question ""-test" implies that all tests are
> executed ..."?  I think the current behavior in C4 is nice: tests are
> only executed for the eggs given on the command line.  I think in C4 you
> actually have to install eggs before testing them, no?

Yes, only if installed. What the sentence above is intended to mean is
that if a test fails, installtion is not aborted. I guess it would be better
to abort, and only run tests for the explicitly given eggs.

>
> > * What should be the default optimizations, or should there be a
> >   default optimization level at all? (this can be given in the .egg file);
> >   the same applies to compiled import libs, where the user can not
> >   override the options.
>
> I think the default should be csc's default, and users should be able to
> specify the optimization level they want.

That would be no optimization at all. Perhaps use "-O2" by default
(unless the .egg file says differently)? Also, import libraries should
be compiled with high settings (-O3 ?), perhaps even without debug
info (-d0), but I'm not sure about this.

>
> > * Is an option for not fetching/installing depedencies desirable?
> >   This has been added recently to C4, what is the inention of this,
> >   and how does it work if the dependency is required for building?
>
> That option was added to C4 to provide a way for packagers to determine
> what files belong to each egg.  That's not trivial to determine in C4.
> With -no-install-deps, we can install dependencies in the canonical egg
> repo and specific eggs in another directory (their dependencies are
> loaded from the canonical repo).  With this, packaging an egg means
> packaging the directory where the egg was installed.

Hm... So this effectively means multiple egg repos, right?

> I personally like the current behavior in C4 (run tests only for eggs
> explicitly given on the command line).

Ok.

>
> > * "extension-information" actually retrieves information for a whole egg.
> >   Should this be renamed, or should it extract information for a particular
> >   component?
>
> If I understand correctly, "extensions" are library components of eggs.
> Is my understanding correct?  In this case, I suppose we should use
> egg-information to obtain information for a whole egg, and
> extension-information for the library component (if it makes sense).

In this regard, extension actually means "component" (extension library,
program, even files, i.e. the parts that make up an egg.)

Getting component-specific information is a bit tricky, since this has to
parse the .egg-info file (which exists for the whole egg). A simpler
way would be to provide "extension-information" (possibly renamed)
and let the user extract whatever he or she needs. Usually 
"extension-information"
is a way to determine whether an egg is installed, I haven't found many
other uses yet. It can also be used to check what files are installed.

> > * I would REALLY REALLY like to get rid of "-prefix", "chicken-prefix",
> >   CHICKEN_PREFIX, etc. as they cause pain and confusion, but
> >   Mario says that it is important for Salmonella.
>
> Currently, salmonella uses -init to initialize a private local egg repo
> for tests, and uses -prefix to install eggs in it.  In the non-cached
> mode, it sets that private egg repo empty for each egg it tests.  That's
> how salmonella tests eggs in a fresh environment, and that's how missing
> dependencies are caught.
>
> I don't know another way to implement this feature.  Things like chroots
> and a special CHICKEN installation for salmonella have been suggested,
> but they are not really good options in my opinion.  chroot is not
> portable and would require administrator privileges.  A dedicated
> CHICKEN installation for salmonella would totally kill the convenience
> of using salmonella as an egg checker for casual users.
>
> Another suggestion was about making chicken-install generate a list of
> modules imported by egg components.  With this, it would be possible to
> cross-check the generated list of module dependencies against the
> dependencies specified by `dependencies' in .egg.  The list of imported
> modules is actually generated by csc.  So, to test `test-dependencies',
> chicken-install would have to make tests/run.scm go through csc.  And if
> we have custom build scripts in CHICKEN specified via `custom-build'
> (e.g., scripts that use the make egg, setup-helper etc.), how can we
> detect dependency problems?  I don't know how many times I saw eggs
> breaking because their .setup used some egg that was not specified in
> `depends'.

Some time ago there was a strong interest in supporting mulitple local egg
repos (i.e. CHICKEN_REPOSITORY would specify a list of directories, each
of which may contain eggs). Would that allow dropping -prefix, and still
provide a way to split installations across multiple directories, and thus solve
the requirements for packaging (-no-install-deps) and Salmonella?

> > * It has been suggested to sign eggs (or egg hashes). I don't know enough
> >   about this to decide how to proceeed. If such a feature is desired, it 
> > should
> >   be added early (standalone crypto code seems to be available, e.g.
> >   "tweetnacl")
>
> That'd be nice.  Are you also considering tarballs?

For eggs? I don't know. I wouldn't want to add an untar tool to the core,
and bandwidth is usually not the biggest problem.

> * Do we need `doc-from-wiki'?

I forgot. What was the original intention? I think it was to mark where 
documentation
was to be found, for some tools or so, but I can't remember.

> * chicken-uninstall should uninstall _eggs_, no?
>
>   I ask because the documentation for chicken-uninstall and chicken-status
>   mention "extension":
>
>     -exact
>         match extension-name exactly (do not match as pattern)
>

Yes, full eggs, as components of extensions may have interdependencies.
That's the whole point of an egg: to package togther related parts have
mutual dependencies. I will fix the documentation.

>   By the way, I think the default behavior should be exact matching
>   (http://bugs.call-cc.org/ticket/1093).

Quite true. I'll do this.

>
> PS.: I'm having trouble to compile the chicken-5-new-egg-install branch
> using 4.12.0 to bootstrap chicken (I can compile the chicken-5 branch,
> though).  Is this a known issue?  It's possible that I'm doing something
> wrong, naturally.  I tried to rebase against chicken-5, but git scared
> me a little.

I think the reorganization of the libraries and modules by that notorious
newzealander has made bootstrapping somewhat difficult... I did the
following, IIRC:

1. build a bootstrap chicken from 4.12.0
2. used that chicken-boot to build C5
3. build a boot-chicken from the C5 chicken
4. used that bootstrap chicken to build the branch


felix