Re: [Gluster-devel] [RFC] on the fly GFID change for a existing file

[Raghavendra G]

On Fri, Apr 26, 2013 at 3:37 PM, Amar Tumballi <address@hidden> wrote:

Hi,

This is an extended discussion on patch http://review.gluster.org/4702

With this patch going in, a mount point can be made to access the files directly using the gfid of the files, and not just path.

We are planning to use this along with Changelog [1] for enhanced geo-replication feature we are planning to develop. Now, the good thing is, with these combination we have many benefits, which primarily includes not crawling the system to find the changes done in last N minutes.

But the challenge we have now is with the upgrading of the existing geo-replication setup to newer one, where we would need to keep the 'slave' volume's files to have the exact same GFID as that of 'master' volume. To achieve this, when we upgrade, we would need a *method* to change the GFID of the existing files in 'slave' volume on the fly.

We have couple of options:

1. delete the '.glusterfs/' directory from the slave volume and use the aux-gfid-path based mount to do the lookup (with proper GLUSTERFS_GFID env variable set), so it creates the gfid with new one.
 * needs a change in posix xlator to overwrite existing 'trusted.gfid' attribute too.

This approach is inline with gfid healing we already have and requires very less code change within glusterfs (we might need a script to remove existing gfid xattrs and send a stat on each file with correct gfid from master). However the parallel use of mount point  by other applications during the window of time (after gfid xattr deletion, before lookup with correct gfid copied from master) might result in a file having a different gfid from its counterpart in master. Hence, this approach does not bode well for rolling upgrades (upgrades without downtime).
 

2. bring a setxattr() interface to change the gfid on the fly based on a virtual xattr.
 * needs extra check as the existing inode number (aka, gfid) suddenly changes, and we need to handle it gracefully.

This approach is also simple, but requires relatively more code change within glusterfs (compared to 1). If we happen to change gfid of an already looked up inode, protocol/client fails revalidate (re-lookup) on that file with ESTALE and fuse-bridge sends a fresh lookup on that path. The effect of this would be we send a new nodeid to kernel as reply of revalidate. And kernel too will create a new inode replacing old inode for that file. Hence AFAIK this wouldn't cause any problem. This approach is also good since we change gfid atomically. Hence rolling upgrades are possible with this approach.

Given the advantages of approach 2, I would prefer it. Thanks to Csaba for his inputs.

Let me know if someone has better options, or can take a call on which is the better approach.

Regards,
Amar

[1] - http://review.gluster.org/4766

_______________________________________________
Gluster-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/gluster-devel

regards,

--
Raghavendra G