Re: Writing 'help' functions as m-files

[John W. Eaton]

On 10-Feb-2009, Søren Hauberg wrote:

| man, 09 02 2009 kl. 22:58 -0500, skrev John W. Eaton:
|
| >   * Since all the data are character strings, using a binary format
| >     doesn't save much space.  Compression would help, so maybe using
| >     -text -z as options for save would be better than just using
| >     binary.
| 
| I've changed this. I didn't do this at first, since I wasn't sure what
| would happen if Octave was built without support for compression. Will
| the call to 'save' fail, or will it simply skip the compression?

I expect it fails.

| >   * Running the funtion takes some time, so I think it would be best
| >     to run it at build time.  It looks like most of the files will not
| >     need to change when Octave is built, so it probably also makes
| >     sense to distribute these files with the tar.gz files.
| 
| I agree. The only problem I see, is what if we distribute compressed
| caches, and the user doesn't link to to 'zlib' (or whatever we use).

Yes, that is a problem, but it doesn't seem critical to me.  If you
want the full functionality, install the prerequisites.

| >   * What is the slow part?  Running makeinfo?  If so, then I don't see
| >     much we can do about that.  Or is it extracting the first sentence
| >     of the doc string?
| 
| I haven't done any profiling, but I'm guessing it's the many calls to
| 'makeinfo'. The algorithm for generating caches are in many ways similar
| to the 'lookfor' code. When I moved 'lookfor' from C++ to an m-file, I
| didn't really see any significant speed changes, so from that I
| concluded (a bit hasty) that the slow part was 'makeinfo'.

I suspect there is also some time spent parsing the files to extract
the docstrings.  Instead of just looking for the docstrings, your
function is parsing them.

Also, we are currently finding the functions in the path, and doing

  if (!dir_in_path)
    addpath (directory);
  endif

  ...

  if (!dir_in_path)
    rmpath (directory);
  endif

which has some problems.  First, there's no unwind_protect here, so
you can end up modifying the global path if this function is
interrupted.  Second, the current directory is still searched before
the added path, so you could end up witn the wrong docstring in the
DOC.gz file.  I think it would be better to temporarily cd to the
directory, do the work, then cd back (using unwind_protect to ensure
the change is temporary).

It occurred to me that in the process of building Octave, we already
generate DOCSTRING files that contain the raw Texinfo docstrings.  I
think we should be taking advantage of that.  The new function could
still be useful for generating a DOC.gz file later, but I'm not sure
it is the best thing to use during the build process.

How about making the following changes to the build process:

  * generate a DOCSTRINGS file for each directory in the source tree
    that contains .m files or C++ files with DEFUNs.  We'll read these
    files to generate the .texi files for the manual, same as we do
    now for the two DOCSTRINGS files that we currently generate.  I
    can do this part.

  * use the DOCSTRINGS files to generate the DOC.gz files.  It should
    be much faster to read the DOCSTRINGS files than it is to parse
    all the .m files in Octave.  We need to pass each docstring in a
    DOCSTRINGS file through makeinfo and then extract the first
    sentence.

| > I propose making gen_doc_cache take two arguments.  The first argument
| > names the output file.  The second names the directory to work on.  If
| > only one argument is given, generate the DOC.gz file for for keywords,
| > operators, etc.
| 
| The attached changeset does this. The change also fixes the bug you
| mentioned when doing 'gen_doc_cache (".")'.

OK.

| One thing to consider is where to put the cache for builtin stuff
| (operators, keywords, ...). With the attached patch this is stored in
| the file
| 
|   fullfile (octave_config_info.datadir, "DOC-builtin.gz");
| 
| (see 'lookfor.m' at line 58). Is that the right position/name?

My Debian-packaged copy of Octave sets datadir to /usr/share, so
that's probably not the best place.  Maybe /usr/share/octave/VERSION?
But there is no variable for just this directory yet, so we would need
to add one.

jwe