Re: [Chicken-hackers] [PATCH] Fix #1133

[Peter Bex]

On Mon, Jun 23, 2014 at 04:02:59AM +0400, Oleg Kolosov wrote:
> On 06/19/14 23:27, Peter Bex wrote:
> > Well, kind of.  Let me tip-toe around the definitions to clarify:
> > Currently units are used in core to define dependencies, which is one
> > of the things that modules are used for as well. And as we saw in
> > this ticket and the other one there are some problems which can be
> > detected in dependencies expressed as modules but not when using
> > "just units".
> 
> Units are also used for separate compilation of large code bases. It is
> not always make sense to separate everything into modules. This is
> useful, but should be build only matter. AFAIK, current system does not
> allow to transparently switch between module and unit mode for the given
> source file. We can try to add an option for chicken to mostly ignore
> module declarations and just issue some useful warnings but compile as a
> unit. This approach will enable incremental migration.

That's an interesting idea.  It might work, but I'm also unsure of the
added benefit; if the code is always compiled separately in the "ignore
modules" mode, it won't add the intended benefits of additional checking
of dependencies and exports.

> This area definitely needs attention. Especially handling of external
> extensions (eggs). Last time I've tried to build completely static
> binary it required adjustments to an eggs source code. I don't remember
> specific details, maybe it is already improved. An ability to split up
> the Chicken runtime might will be helpful, currently it is too big and
> contains many unnecessary things (can be moved into eggs). Windows
> support is nice to have, but too alien for current Chicken toolchain.
> Also, seamless compiling for Android and iOS is a big deal.

I'm unsure how much further we can strip down the core.  Perhaps we can
drop SRFI-1, SRFI-13 and SRFI-14, which would be just fine as eggs.  But
currently core is using some procedures from these.  I think that this is
a separate issue, though.  Refactoring stuff as modules is a shitload of
work which is not yet certain to be feasible, and we should not drag more
work into this project.

> What do you think about Chibi Scheme approach
> (http://synthcode.com/scheme/chibi/#h2_ModuleSystem)?

That's just R7RS.

> Providing module
> definitions separately, avoids aforementioned issues nicely, and,
> coupled with an option to ignore everything as proposed above will allow
> incremental migration.

It's not 100% separate, the (begin ...) form allows embedded Scheme forms.
Also, our module system already supports separation into files, either
by using INCLUDE or the shorthand form (module NAME (EXPORT ...) FILENAME)
from http://wiki.call-cc.org/man/4/Modules

> Also, having compatible (and standard compliant)
> module system is a huge selling point IMO.

We already support this syntax through the R7RS egg.  We can't just
switch to the R7RS module system in core because it would break 100% of
the user code out there, for no technical gain whatsoever.  CHICKEN core
is an R5RS system, and adding an R7RS module system to it adds no
benefit if everything else from R7RS is only available as an extension.

We have already too many people asking about the difference between use,
require, require-extension, uses etc.  Adding a second module system to
core would only add to the confusion (and *replacing* it would break the
world).

> > Indeed, bootstrapping is an ever-present pitfall.  And there's also
> > library.scm which does not really correspond to a particular module.
> > Perhaps to "scheme", but not exactly.
> 
> I have not dived into this too deeply yet, but can you elaborate what
> real problems you anticipate?

All the ##sys# stuff and other "internal" procedures don't really belong
anywhere, including a lot of runtime support code that other core units
and the compiler need.  Finally, there's an overlap between the r4rs and
r5rs primitive modules.  That r5rs module is currently defined as r4rs
plus VALUES, DYNAMIC-WIND and CALL-WITH-VALUES.

The simple solution would be to stuff everything into a "core" module
which nobody but the core system is supposed to really use.  We could
then define r4rs and r5rs/scheme as modules which import the whole of
core and reexport only the relevant procedures.

But there's still the problem of code which replaces primitive
procedures.  However much this is frowned upon by "modern" Scheme
standards, this is something that is explicitly allowed by RnRS | n < 6.
AFAIK toplevel code can do that even with identifiers exported by egg
modules, so perhaps this is not an issue.  We have to test it though, and
luckily r4rstest.scm already checks this.

> > I just wanted to put the idea out there, as a sort of "call to arms".
> > This is another large-scale project that will take a lot of energy which
> > I personally don't have right now.  Maybe I'll put it on the wish-list
> > for a long-term goal.
> 
> We recently migrated our project to modules precisely for a reason
> started this thread and still cleaning up the mess. Suddenly, we
> realized that many files are placed in wrong folders, some parts have
> circular dependencies, some know way too much about others. But seeing
> this on the surface is a good thing in the end.

Yeah, people often complain when they find out that circular dependencies
are impossible in the module system, but that's a good thing if you want
to have a fighting chance at maintaining your code later.

> IMO, Chicken have some of these problems too, it is often hard to
> predict which module has which function due to responsibilities mixed
> between library, posix, files, extras, utils, et al.

Yeah, we have had problems this.

> First of all, I suggest to break things up into very small modules based
> on intended usage and common sense (see R7RS or Chibi for examples) and
> provide reexporting declarations for compatibility. This will allow to
> freely move things around. Then refactor, cleanup, repeat.

Yeah.  Especially the stuff from posix should be cleaned up.  There are
plenty of things unrelated things in there at varying abstraction levels,
which is only stuffed in there because it's somehow related to an
underlying syscall that's defined by POSIX.  This is a little too
system-specific for me; it would make more sense to group those things
together into things like files, process-control, user-management etc,
and define modules in that way.

> I am very interested in this undertaking and want to help you out.

That would be much appreciated, once we get started.

Cheers,
Peter
-- 
http://www.more-magic.net