I wasn't using the synctask_new worker, but now I implemented my code using it to trigger the create/write call. Btw, is the usage of syncop calls documented somewhere? I found it through the history of the dev-list.
But going ahead on my issue, I was able to create the file avoiding the NFS stale handle error. The file is being created at the backend, but the syncop_create call it's still returning an error from the posix translator at the server side. I copied the error from the glusterd log below... I am checking the code trying to figure it out.
To write the file I am generating a new gfid for the local file and setting the pargfid as 1 (root node) with the loc.parent pointing to the inode->table->root inode. Below I copied what is being copied the loc, fd, flag and mode_t structures being used at the create call.
*** From the glusterd log:
[2012-09-10 18:12:34.035940] W [posix-handle.c:590:posix_handle_hard] 0-examplevol1: mismatching ino/dev between file /bricks/exampleDir_local/1ccfca9f (146628/2049) and handle /bricks/exampleDir_local/.glusterfs/00/00/00000000-0000-0000-0000-000000000000 (146617/2049)
[2012-09-10 18:12:34.035977] E [posix.c:1724:posix_create] 0-examplevol1: setting gfid on /bricks/exampleDir_local/1ccfca9f failed
*** Relevant code before calling the create function
107 inode_t *newinode = inode_new (old_loc.inode->table);
108
109 loc.path = strdup(hashed_path_str);
110
loc.name = strdup(hashed_filename_str);
111 loc.inode = newinode;
113 loc.parent = loc.inode->table->root;
115 uuid_copy(loc.gfid, gfid);
116
118 memset (loc.pargfid, 0, 16);
119 loc.pargfid[15] = 1;
120
121 fd = fd_create (loc.inode, getpid());
...
127
128 uint32_t flags = O_CREAT | O_RDWR | O_EXCL;
129 mode_t mode = (mode_t)0700;
130 fd->flags = flags;
131 fd = fd_ref(fd);
Gustavo Bervian Brand
---------------------------------------------------------------------------------
On Sun, Sep 9, 2012 at 8:10 PM, Anand Avati
<address@hidden> wrote:
On Sun, Sep 9, 2012 at 11:09 AM, Anand Avati
<address@hidden> wrote:
On Sun, Sep 9, 2012 at 10:53 AM, Gustavo Bervian Brand
<address@hidden> wrote:
Hello,
I'm implementing a translator and at the moment I'm trying to "cache" at the local node's disk the files read from other node.
When reading a file from a certain node, at the file open function it's added an extended metadata at the original file with info about where this file (or part of it) is copied, calling syncop_setxattr to the proper child to do it.
So, once open is done the readv comes and if this file was not already read/cached, at the readv_cbk (before stack unwind) I am trying to create a local file with the content got from the original node.
For that purpose, I am using syncop_create / setattr / write. But the syncop_create is always failing with Stale NFS file handle error.
I tried to copy the same gfid and pargfid of the loc_t used at open function (saved), it didn't work, tried to zero them, nothing... and manipulated all the variables used at the syncop_create call... but nothing worked to get a success return from syncop_create.
Is there some requirement to call syncop_create, or a good way to track the exact reason for this failure (I could see it's not even reaching the client_create() call through gdb)? I am using the code from the master branch of 1 week ago. Any comment is welcome.
And BTW, I hope you are performing the syncop_XXXX() calls inside a synctask_new() worker?
Avati
Part of the log:
[2012-09-09 17:04:16.357875] I [gbfs_t.c:361:gbfs_open_cbk] 0-gbfs-test: entering gbfs_open_cbk function
[2012-09-09 17:04:16.358026] I [gbfs_t.c:169:gbfs_readv] 0-gbfs-test: entering gbfs_readv function
[2012-09-09 17:04:16.358156] I [gbfs_t.c:46:gbfs_readv_cbk] 0-gbfs-test: entering gbfs_readv_cbk function
[2012-09-09 17:04:16.358192] I [gbfs_t.c:132:gbfs_readv_cbk] 0-gbfs-test: failed to create /35a710dd on examplevol-client-0 (Stale NFS file handle)
It will be good to put a break-point at gbfs_readv_cbk to inspect the backtrace to find which layer did the STACK_UNWIND(..,ESTALE...); and continue debugging from there.
For a create, you are expected to:
- fill loc.pargfid
- fill loc.basename
- fill loc.inode (obtained via inode_new())
- fill "gfid-req" key in xdata dictionary with a GFID to be assigned for the newly created file.
You can inspect posix_create() and client_create() to double check what the "dependencies" are. Note that if what you are creating is just a cached version of the original file, then it might be a good idea to re-use its gfid too.
Avati