[Chicken-hackers] library unit restructuring

[Felix Winkelmann]

Hello, again...


I am currently trying to put together a plan about the library
restructuring that seems to be badly needed to make the core libraries
smaller, more fine-grained and hopefully easier to use.

* As already stated, I think it would be good to move some units 
  into eggs. Currently these are

  - srfi-69: hash-tables are relatively fundamental, but this egg is
    very comprehensive (and thus large) and only used in
    chicken-profile and lolevel (the object-evict stuff).

  - srfi-18: it may be surprising, but this isn't really needed in
    core, only the scheduler is. I never was a great admirer of the
    SRFI-18 API anyway, as it is more or less a thin layer over the
    pthreads API (a message-passing API, similar to termite, is much
    simpler, more convenient, easier to use and less error-prone. And
    probably even faster...)

* Some groups of procedures could be moved to eggs as well:

  - binary-search, made as a functor, this would be more flexible
    and much faster.

  - queues

  - memory-mapped-files

  - object-evict and related procedures. These need srfi-69, and are
    obscure enough to be moved out of the core (actually the whole
    functionality is somewhat fishy and "object-release" is in fact
    not safe to use.)

  - I have started extracting the code to be eggified, along with
    tests and .types files, but this work is not complete yet.

  - there is probably more, but these were the ones I could easily
    spot. Suggestions are appreciated. 

* Here is a list of the procedure groups in the "utils",
  "data-structures", "extras", "files", "ports" and "lolevel"
  units. Note that sensible and intuitive names are needed, which is
  something that promises endless bikeshedding, so I implore everybody
  who is interested in assisting to invent names or structure for this
  list not to find the best possible solution, but one that is easy to
  remember and makes at least a little bit of sense.

  - data-structures
    - sort (+ friends)
    - functionals / combinators ("identity", "flip", ...)
    - lists ("atom?", "intersperse", ...)
    - alists (including "rassoc" - or should that in the "lists" unit?)
    - strings ("reverse-string-append", ... - this is quite a lot. eggify?)
    - topological-sort (needed in chicken-install)
  - extras
    - extended I/O ("read-file", "read-line", ...)
    - random
    - pretty-print
    - printf (+ friends, including SRFI-28, unfortunately used everywhere)
  - files
    - file-ops ("file-copy", ...)
    - pathnames
    - temporary-files
    - directories (or put this to "pathnames"?)
  - utils (note that this is a unit that uses many others, but contains quite 
useful
    stuff)
    - "system*" (move somewhere else?)
    - "read-all" (move somewhere else? to "extended-I/O" perhaps?)
    - "qs"
    - "compile-file"
    - "scan-input-lines"
    - "yes-or-no?"
  - lolevel (I'd love to get rid of this, but that may be hard)
    - "move-memory!"
    - "object-copy" (put this into the same place as "object-evict"?)
    - "allocate"/"free" (silly, but sometimes necessary)
    - pointers (move into egg?)
    - locatives
    - extended-procedures
    - blocks
    - records (lowlevel operations)
    - become! + mutate-procedure!
    - pointer-vectors
  - ports
    - "port-map", etc.
    - "funky"-ports
    - redirect-ports
    - string-port-extensions
    - custom-ports

* Backwards-compatibility could be kept by providing "wrapper" eggs,
  that are named like the current library units ("extras",
  ...). chicken-install should normally ignore core units, when they
  are listed in "depends" meta-clauses or given on the command-line (I
  think we did this before). Still this is a massive change, suitable
  for a major release only.

* This doesn't touch srfi-1/13/14 yet, there still has to be figured
  out what's needed and what not, something that calls out for a
  cross-referencing tool.

* This is just a proposal, nothing is fixed.


felix