[Monotone-devel] [long] subdirectory restrictions

[Derek Scherger]

I'm happy to report that I finally have restricted/subdirectory
operations essentially working in the code on the restrictions branch.

I've added a new command (called "setup" for now) that creates the
book keeping dir to be used when starting to track files in a new
directory. Previously monotone add would create this directory if it
didn't exist but this now seems dubious, considering that add may be
running from a subdirectory of an existing working copy and then not
need to create an MT dir.

There is now a fairly clear distinction between three different types
of commands, with regards to the MT dir and MT/options in particular.

existing working copy commands 

   (add, drop, rename, commit, update, revert, diff, log, status,
    ls unknown, ls ignored, ls missing)

    These require the MT directory to exist somewhere up the tree and
    will fail if it can't be found.  They read MT/options for defaults
    and write MT/options to save updated settings in the working copy.

    Note that log and diff have both working copy and non-working copy
    variants depending on which args are supplied and these rules
    apply only to the working copy variants.

new working copy commands

   (checkout, setup)

    These require that the MT directory does NOT exist in the target
    directory and will fail if it does.  They create the MT dir and
    write the initial MT/options for the new working copy.

    The fact that checkout and setup fail if an MT dir is found in the
    target directory is probably debatable, a few tests did need
    convincing. My rationale is that it's easy to manually delete this
    dir if you really want to and overwriting it might cause problems
    that would be considerably harder to fix.

non-working copy commands.

   (others...)

    These will read MT/options for defaults if it's found by the
    search up the tree for the MT dir but will work fine if no MT dir
    is found.  They do NOT write out an updated MT/options file.

    Note that log and diff have both working copy and non-working copy
    variants depending on which args are supplied and these rules
    apply only to the non-working copy variants.

Previously, MT/options was effectively read for all commands as it was
read before the command had been determined in the app_state
constructor. It was written out by add, drop, rename, commit, update,
revert, checkout and also by merge which in particular seemed a bit
odd.  It seems that either all commands or only working copy commands
should write MT/options, rather than most working copy commands and a
few others. Having non-working copy commands write it out seems
questionable.

The --include/--exclude options are no longer supported and instead
commands may include a list of files or directories to include in the
current operation. Lacking any such list all files are included. This
means that the only way to exclude things is with standard command
line tools like xargs, grep, etc. Overly long lists of files from
xargs can certainly cause problems.

Lists of file arguments are considered as complete file or directory
paths and must match either complete manifest paths or directories
thereof. Previously a file argument like "foo" would have matched foo.c
foobar.c and the directory foo. Now it will match a file or directory
named foo (all paths like foo/*) but not foo.c or foobar.c.

Searching for the MT dir is initiated by each command so that the
distinction between working copy and non-working copy commands can be
made first. I'm sure there are more ingenious implementations but the
fact that commands like diff and log currently have working copy and
non-working copy variants does complicate things.

Loading the various lua hook files has moved into app_state so that it
can be delayed until after the search for the MT dir has completed.
This is required so that MT/monotonerc can be loaded at the right
time.

I've added a --message option for commit and --manifest options for
diff to allow the list of file arguments for these commands to be
exclusive of changelog messages and manifest ids. These need cleaning
up and may put some more fuel on the command specific options fire.

The following relatively minor issues remain to be cleaned up but I
think I want to wait for the changeset stuff to land before dealing
with them.

".." in path arguments

   Arguments to commands within subdirectories may expand a
   restriction by including .. path elements in arguments. Too many
   .. path elements in an argument will attempt to escape the working
   copy and fail with:

        monotone: misuse: prohibited path component ".." 

   One less .. argument than this will effectively attempt to remove
   the restriction and fail with:

       monotone: misuse: prohibited path component "." 

   This restriction (in vocab.cc) seems like it's safe to remove but I
   haven't touched it for now. Otherwise, including .. path elements
   in arguments to restricted commands should work as expected to
   expand the scope of a subdir restriction.

missing files

   Commands from within subdirs will fail with a disappeared file
   error for missing files *anywhere* in the working copy. It would
   probably be better if missing file failures were limited to the
   current subdirectory.

bad path arguments

   Bad (i.e. non-existent) paths in argument lists are only detected
   by commands that explicitly read files from the filesystem named by
   these paths like add, drop, and revert. Commands such as status,
   diff and commit only use these paths to limit visible changes
   between a pair of manifests. A non-existent path simply doesn't
   include anything and is simply ignored.

restricted updates

   For now the update command does honor a restriction but I'm still
   wondering whether restricted updates make sense.  As I understand
   it, the idea behind update is to move a working copy up to the
   current version on some fork or line of development. Doing a
   *restricted* update will move the working copy to some point along
   this line, but not all the way to the end. This will leave the
   working copy based off of a version that, in all probability,
   doesn't exist in the database.

   A subsequent update should presumably move the working copy further
   along on this line, but I think the restricted update will have
   created what amounts to a new fork, from which no subsequent
   development will likely have occured.

I'm sure there's plenty of room for improvements beyond these things
too but hopefully this will serve as a good starting point.
--
Cheers,
Derek