Re: [Gluster-devel] file version on glusterfs using libgit

[Kaleb KEITHLEY]

On 03/08/2013 04:50 PM, Niels de Vos wrote:
> On Fri, Mar 08, 2013 at 06:00:24AM -0500, Shishir Gowda wrote:
> > Hi Niels,
> >
> > Thinking out aloud, I think the snaps(in file version context) can be
> > displayed as branches (list).
>
> Well, I am not sure if branches are really needed. Isn't linear history
> sufficient? Every change should be committed to the master branch
> anyway. Branches may be useful for switching between versions, but
> nothing prevents you from checking out (or "git ls-files") with a commit
> or a date.

I think snapshots can be implemented with tags, and had also envisioned 
a virtual directory scheme like Niels has described. There could/should 
be a cron job that makes yearly, monthly, daily, and/or even hourly 
snapshots.

I could imagine using branches for a kind of auto-dedupe copy mechanism. 
If I copy a file to another location, a branch is made, and the new copy 
is a magic symlink that points to the branch.

> I'm thinking of a virtual .snaps directory:
>
> $ cd $VOLUME/.snaps
> $ ls
> 2013-03-07/
> 2013-03-06/
> .....
> current/
> changelog
> yesterday -> 2013-03-07/
>
> This makes it possible to do something like:
> $ cat changelog
>     - virtual file, showing the contents of 'git log'
>     - find a commit you're interested in
> $ mkdir $GIT_COMMIT_ID
> $ ls $GIT_COMMIT_ID/
>     - get the state just like 'git checkout $GIT_COMMIT_ID'
>
> Maybe it would be helpful to be able to create tags inside this .snaps
> directory. But I would refrain from branches for now (unless there is
> a clear use-case).
>
> Cheers,
> Niels
>
> > Once the user cd's into any one of them, we could do a git checkout of the 
> > branch.
> >
> > That should mimic the behaviour.
> >
> > With regards,
> > Shishir
> >
> > ----- Original Message -----
> > From: "Shishir Gowda" <address@hidden>
> > To: "Niels de Vos" <address@hidden>
> > Cc: address@hidden
> > Sent: Friday, March 8, 2013 4:04:42 PM
> > Subject: Re: [Gluster-devel] file version on glusterfs using libgit
> >
> > Hi Niels,
> >
> > My inclination too is to load git ontop of posix xlator.
> >
> > I was thinking of making previous versions (based on some policy) to be treated 
> > a new branch.
> >
> > We could see how to export these branches as user visible dirs.
> >
> > With regards,
> > Shishir
> >
> > ----- Original Message -----
> > From: "Niels de Vos" <address@hidden>
> > To: "Shishir Gowda" <address@hidden>
> > Cc: address@hidden
> > Sent: Friday, March 8, 2013 3:25:57 PM
> > Subject: Re: [Gluster-devel] file version on glusterfs using libgit
> >
> > On Thu, Mar 07, 2013 at 12:54:41AM -0500, Shishir Gowda wrote:
> > > Hi All,
> > >
> > > Was playing around with git on glusterfs volume, to provide was of file version 
> > > support.
> > >
> > > And initial run is encouraging.
> > >
> > > A brief overview what was tried:
> > >
> > > Approach 1: Glusterfs volume as a git repo
> > >
> > > 1. created a 2 brick distribute volume
> > > 2. inited a git repo on fuse volume
> > > 3. created files, committed them in git.
> > > 4. Modified files, and committed them again
> > > 5. Did branch check-outs, to simulate versions @ point in time
> > > 6. reset branch heads, and was able access older version of files (after a 
> > > stash).
> > > 7. Was able to create files/dirs/symlinks/hardlinks
> > > 8. Both NFS/FUSE clients were used.
> > >
> > > Approach 2: Glusterfs bricks as git repo's
> > >
> > > 1. created a 2 brick distribute volume
> > > 2. inited git repo on brick1
> > > 3. inited git repo on brick2
> > > 4. created files, committed the relevant brick's git.
> > > 5. Modified files, and committed them again on brick's git
> > > 6. Did branch check-outs, to simulate versions @ point in time on individual 
> > > bricks
> > > 7. reset branch heads, and was able access older version of files (after a 
> > > stash).
> > > 8. Was able to create files/dirs/symlinks/hardlinks
> > > 9. Both NFS/FUSE clients were used.
> > >
> > > Buoyed by this, will start prototyping integration of libgit2 as xlator for 
> > > file version support.
> > >
> > > There are 3 approaches to consider:
> > >
> > > 1. Load git xlator on clients volfiles
> > > 2. Load git xlator on server volfiles
> > > 3. Replace posix interface with git interface.
> > >
> > > Please provide feedback, on what would be more desirable.
> >
> > Very interesting! Option 2 makes most sense to me, the posix xlator
> > contains some access checks and such, which you probably should not need
> > to duplicate.
> >
> > Have you thought about making the previous version accessible through
> > the glusterfs/nfs mount? Other vendors seem to have a .snapshot
> > directory with previous versions, would something like that be possible?
> > Users would be able to recover deleted files themselves that way.
> >
> > Also, I do not know if git stores xattrs and their changes...
> >
> > Cheers,
> > Niels
> >
> > _______________________________________________
> > Gluster-devel mailing list
> > address@hidden
> > https://lists.nongnu.org/mailman/listinfo/gluster-devel