feature/pkg 7da73fea98: Improve notes

[Gerd Moellmann]

branch: feature/pkg
commit 7da73fea981ab09334a9eb2a51592728583ba3d6
Author: Gerd Möllmann <gerd@gnu.org>
Commit: Gerd Möllmann <gerd@gnu.org>

    Improve notes
---
 admin/cl-packages.org | 204 ++++++++++++++++++++++++++++++--------------------
 1 file changed, 122 insertions(+), 82 deletions(-)

diff --git a/admin/cl-packages.org b/admin/cl-packages.org
index abba9c4c73..fb2bed42f8 100644
--- a/admin/cl-packages.org
+++ b/admin/cl-packages.org
@@ -1,78 +1,97 @@
 # -*- mode: org; eval: (auto-fill-mode 1); org-indent-mode: 1; -*-
 #+STARTUP: show3levels
 
-* Common Lisp Packages for Emacs
+* Common Lisp packages for Emacs
 
 This is an experimental implementation of CL packages for Emacs.
-
 The question of the experiment is if it is possible to add CL packages
 to Emacs with reasonable effort and reasonable compatibility.
 
-Note that this branch is only known to build and run under macOS 12.6.
-I don't have other systems.
+Note that this branch is only known to build and run under macOS.  I
+don't have access to other systems, so it might not compile or work on
+other systems.  Patches welcome.
 
-** Overview
-There are two packages defined at present.  The keyword package, named
-"keyword" or "" contains keywords, the Emacs package, with name
-"emacs" contains all other symbols.
+Please see a book like Common Lisp the Language (CLtL2) for a
+description of the CL package systen.  The book is freely available
+from CMU.
 
-Please see a description of the CL package system on the web for what
-you can do with it.  Not everything might yet be implemented.
+** Status
+This builds and runs with unchanged Magit, Lsp-mode, and other
+packages for me, so it seems to be pretty backwards-compatible.  I
+can't gurantee anything, of course.  If you find a problem, please let
+me know.
 
-And bugs, for sure, and so on...
+** User-visible functionality
+There are three pre-defined packages.
 
-** Problems Found and Approaches to Solving Them
+The keyword package, named "keyword" or "" contains keywords.
 
-Here are the main problems found, and how I approached them.
+The Emacs package, with name "emacs" contains all other symbols.  All
+code is currently loaded in this package, for compatibility.  All
+symbols in the package are currently exported.
 
-*** Keywords
-In CL, keywords are symbols in the keyword package.  The leading colon
-of a keyword is not part of its symbol name, but a package prefix.
-The keyword package has a nickname that is an empty string.
+The "emacs-user" package is intended for user-code, for example in
+*scratch*, and uses the Emacs package, so that everything in Emacs can
+be used.
 
-In Emacs, keywords are just symbols whose names start with a colon,
-and that is expected in a ton of places.
+These variables are defined:
 
-Solution:
+"*package*" holds the current package like in CL.  It's buffer-local,
+and you can't set it to a non-package value, to prevent havoc.
 
-- Internally, keyword names don't contain the colon, which is TRT.
-- symbol-name returns a name with colon for keywords.
-- cl-symbol-name returns the symbol name as-is.
-- intern and intern-soft when called with a name starting with a colon
-  interpret that as wanting a keyword.
+"*emacs-package*", "*keyword-package*", "*emacs-user-package*" hold
+the package objects.  This is mainly for easier debugging and testing.
+The variables may go at some point.  Or not.
 
-*** Package Prefixes
-Existing code contains symbols like GUI:xyz which look like GUI is a
-pracke prefix.
+"*package-registry* is a hash-table of registered packages.  The
+variable may go at some point.  Or not.
+
+Various functions related to packages are defined.  Depending on the
+time when you read this, this may be in some state of incompleteness,
+and it probably has bugs.  Reports or fixes welcome.
 
 ** Implementation notes
+*** Where is it?
+The C part is in src/pkg.c.  I chose that name because package.c
+resulted in conflicts in the tests (with package.el).
+
+The Lisp part is in lisp/emacs-lisp/pkg.el.  I've done as much of this
+in Lisp because that's much easier and faster.  If packages are used
+in files loaded in loadup, changes might be necessary to make this
+possible.  I consider this out of scope, ATM.
+
 *** No pure space support
 The branch contains a patch by Stefan Monnier that makes it no longer
-use pure space.
+use pure space.  I didn't want to deal with pure space.  Note that a
+small fix in init_vectors is needed for making Stefan's patch work.
+There is nothing preventing the use of pure space though, in
+principle.
 
-I didn't want to deal with pure space.  Note that a small fix in
-init_vectors is needed for making Stefan's patch work.
+*** Shorthands
+Are currently not supported.  I understand what they do
+but I don't understand their end-purpose.
 
-*** New type Lisp_Package
+*** Lisp_Package
 There is a new Lisp data type Lisp_Package defined in lisp.h.
 
 ***  Lisp_Symbol
 Struct Lisp_Symbol has lost its interned flag and its next pointer.
-The interned flag was an implementation detail necessary because there
-were no packages.  the next pointer was only necessary for the obarray
-implementation of symbol tables.
+Both were an implementation detail of obarrays, which are gone.
 
 All symbols now have a package.  Uninterned symbols have a nil
 package.
 
-Keywords have the keyword package.  Other symbols currently are in the
-Emacs package.  Keyword symbol names do not contain the colon.
+Keywords have the keyword package. Note that keyword symbol names do
+not contain the colon.  The function symbol-name still returns a
+string with a leading colon.  I found this was necessary to achieve
+backwards-compatibility.  At least at this point.  The function
+cl-symbol-name returns the real name of a keyword, without the colon.
 
-*** Obarray
-Obarrays have been removed, to be able to remove Lisp_Symbol::next
-whose sole purpose was to support obarray's hash collision lists.
+Other symbols have the Emacs package.
 
-Legacy code is supported by the following
+*** Obarrays
+Obarrays have been removed.  Backwards-compatibility is achieved by
+the following
 
 - The variable 'obarray' still exists.  Its value is now the Emacs
   package.
@@ -80,57 +99,78 @@ Legacy code is supported by the following
   obarrays).  When called with a vector, they secretly create and use
   packages.  This is done because legacy code uses make-vector instead
   of obarray-make to create obarrays.
-- obarray.el has been changed accordingly.
-
-*** Predefined packages
-
-The packages with names "emacs" and "keyword" are defined in
-init_pkg_once as Vemacs_package and Vkeyword_package.  This is called
-directly after init_alloc, which means that the package system is
-ready to use in C code from the start.
-
-The initialization in init_pkg_once includes defining built-in symbols
-(defined with DEFSYM etc, so these are also ready to use.
-
-The variable *package* is found in Vearmuffs_package and default to
-the Emacs package.
 
 *** Reader
-
 The variable 'package-prefixes' determines if the reader will
-interpret a colon in a symbol name part of a package prefix or not.
+interpret colons in a symbol name as part of a package name or not.
 Default is nil.
 
-With package-prefix nil
-
-
+*** Printer
+The printer prints package prefixes if necessary, as in CL.
 
-Use a file-local package-prefix to enable it.
+*** Completions
+The completion functions accept packages as collections.
 
-*** Printer
-*** Shorthands
-Are currently not supported.
+** Problems and how they are approached (currently)
+*** Keywords
+In CL, keywords are symbols in the keyword package.  The leading colon
+of a keyword is not part of its symbol name, but a package prefix.
+The keyword package has a nickname that is an empty string.
 
-The printer prints package prefixes if necessary, which is the case if
-*package* is different from a symbol's package.
+In Emacs, keywords are just symbols whose names start with a colon,
+and that is expected in a ton of places both implicity and explicitly
+in various ways.
 
-With package-prefixes nil:
-#+begin_src
-  'GUI:hansi
-  -> 'GUI:hansi
-#+end_src
+Solution:
 
-Without:
-#+begin_src
-  'GUI:hansi
-  -> unknown package GUI
-#+end_src
+- Internally, keyword names don't contain the colon, which is TRT.
+- symbol-name returns a name with colon for keywords.
+- cl-symbol-name returns the symbol name as-is.
+- intern and intern-soft when called with a name starting with a colon
+  interpret that as wanting a keyword.
 
+*** Fake package qualification
+Existing code contains symbols like GUI:xyz which look like GUI is a
+package qualification.  That's the reason for package-prefixes which
+means to interpret the : as part of the symbol name.
 
 ** Ideas / Todo
-
-- Buffer-local *package*, package-prefixes
-- make_package: allow specifying a start size for symbol hash-table
-- shorthands
-- Add (declare (ignore ...)) goddam :-).
-- Offer cl-symbol-name for sanity.
+*** Completions
+It might be useful to complete over all symbols in all packages.
+I haven't added that.
+
+*** Existing package extensions
+There are some language extensions available in CL implementations
+that might be nice to have:
+
+*** Changing symbol names
+A trap that I always fall into, constantly, in Emacs, is to use CL
+functions without the cl- prefix.  It would be nice to have something
+that makes these symbols available without the cl-.
+
+Just ideas:
+
+- (shadow-alias multiple-value-bind cl-multiple-value-bind) or maybe
+  with regexs. Or something.
+- (import sym as another-sym)
+
+*** Package-prefixes
+Would it make sense to record the value of package-prefixes when read
+or compiled in functions?  This would allow some notorious cases of
+intern, for example, which use colons in the traditional way to behave
+differently.
+
+This could also be used for symbol-name, which would only return a
+name with leading colon if package-prefixes is nil.
+
+*** Modeline
+A mode-line indicator showing the current package and package-prefixes
+would be nice.  Can be done with (:eval ...) in global-mode-string
+now.
+
+*** Tests
+Should be much improved.
+*** Documentation
+Doesn't exist :-).
+*** Other
+- Add (declare (ignore ...)) and (declare (ignorable ...) goddam :-).