On Tue, Nov 27, 2018 at 4:32 PM Taylor R Campbell <
address@hidden> wrote:
Stepping back a moment from file systems and on-disk vs in-memory
caches, can you summarize the usage model of this?
Here's some of the relevant operations that I want to be able to do,
illustrated with how a C system would do it (which obviously we don't
have to mimic but which serves as a conceptual reference that has seen
plenty of practical use) and how we do it currently in MIT Scheme
(which obviously we don't have to keep). How would I do each of
these? If this question doesn't make sense, what are the operations
that you have in mind instead of these?
1. Install a library somewhere, either in some system-wide sense or in
for use in a specific project.
(For a C system, this typically means putting its .h files in
/usr/local/include and its .so files /usr/local/lib, or
/home/riastradh/project/obj/destdir.amd64/usr/include and
/home/risatradh/project/obj/destdir.amd64/usr/lib.
For MIT Scheme we dump stuff ad hoc in
/usr/local/lib/mit-scheme-$ARCH.)
In the new model you put the libraries anywhere you like, then register them using find-scheme-libraries. Obviously putting them in a single directory makes the registration easy.
For "system" libraries, of which there are none at the moment, I expect they'll be installed somewhere in the /usr/local/lib/mit-scheme-XXX tree and registered during install.
It would be easy to extend this by having some predefined locations that are automatically registered at some appropriate time (e.g. startup time or time of first library operation).
The critical part that I'm trying to preserve is that the location of a library need not be tied to its name. In addition, due to the way that compiled libraries are stored, it's easy to write a program that collects a set of libraries and stores them in a single file. This could be helpful when distributing a group of related libraries, and could be extended to do static linking.
2. Build/link/expand/analyze a program that draws from a set of
libraries I installed somewhere.
(For a C system, the compiler or static analyzer looks in
/usr/include for .h files, and the linker in /usr/lib for .so
files, or in /home/riastradh/project/obj/destdir.amd64/... if I
specify -I/-L or --sysroot.
For MIT Scheme we load libraries in /usr/local/lib/mit-scheme-$ARCH
for their macro definitions, but only if (a) we load them by
pathname or (b) they are named in optiondb.scm which we don't have
any formal mechanism to update.)
The libraries, once registered, will be loaded on demand. For some uses loading isn't necessary, since registration stores the imports/exports in the library database.
Of course loading requires a complete tree of library dependencies; if all the dependencies are available and well-formed, then the loader will load all of them in order. Otherwise an exception will be thrown before any loading happens.
3. Run a program I built that uses the libraries it was built with.
(For a C system that uses libfoo, the loader looks in /usr/lib for
libfoo.so, and maybe /usr/local/lib or /opt/local/lib if I
specified -R at build-time or if I use the debugging hack
LD_LIBRARY_PATH at runtime.
For MIT Scheme we load libraries in /usr/local/lib/mit-scheme-$ARCH
but only if (a) we load them by pathname or (b) they are named in
optiondb.scm which we don't have any formal mechanism to update.)
Running a compiled program requires loading the libraries it depends on. If they aren't registered, this will fail. So the libraries need to be registered independently at both times if program is run in a different instance than the one that was used to compile. This does allow the library locations to differ between the two times, however, as would happen when building a distribution.
There's no guarantee that the library loaded at run time will be identical to the one loaded at compile time. Probably there should be a theory of versioning to fix that, but at the moment I'm trying to make something basic that works.
Alternatively, it's possible to statically link a program with its libraries (except the standard ones, which are built in), simplifying the problem.