They are both regular file on the node and the replicas and they have the same
GFID. I ran also the gluster volume heal gv0 split-brain command and the file
is not in the list. We have an entire directory though (1023 entry on a node)
However, the file was already on the brick before uploading it and I noticed
that that the write did not work since the last modification date does not
match the upload time.
Through a web service, we offer to upload files on the gluster mount. This
web service upload the file on a temporary folder and then MOVE the file on
the gluster mount.
Could the move operation give strange behavior like this?
Alexandre Fournier
Tools Programmer
Ubisoft Production Services
-----Original Message-----
From: Pranith Kumar Karampuri [mailto:address@hidden
Sent: 21 novembre 2013 00:47
To: Lalatendu Mohanty
Cc: Alexandre Fournier; address@hidden; address@hidden
Subject: Re: [Gluster-devel] [Gluster-users] Self Heal and dangling symlinks
Alexandre,
Seems like there is an entry split-brain (same file/dir name but on one
brick it is a file and on the other it is a directory) according to the
following log:
[2013-11-18 18:18:43.052446] W
[afr-common.c:1411:afr_conflicting_iattrs]
0-gv0-replicate-0: /aa/aa/aa/aa: filetype differs on subvolumes (0, 1)
Could you get us the output of "stat <brick-dir-path>/aa/aa/aa/aa/aa" and "getfattr -d -m.
-e hex <brick-dir-path>/aa/aa/aa/aa/aa" on both the bricks.
Pranith
----- Original Message -----
From: "Lalatendu Mohanty" <address@hidden>
To: "Alexandre Fournier" <address@hidden>,
address@hidden, address@hidden
Sent: Thursday, November 21, 2013 1:28:01 AM
Subject: Re: [Gluster-devel] [Gluster-users] Self Heal and dangling
symlinks
On 11/19/2013 10:49 PM, Alexandre Fournier wrote:
Hello,
We are experiencing strange behavior when writing file on the Gluster
mount point. On some occasion, when writing to the Gluster Mount we
have an Open Stream error. We’ve looked the gluster logs and found the
following faulty entries :
[From /var/log/glusterfs/mnt-gv0.log]
[2013-11-18 18:18:43.052446] W
[afr-common.c:1411:afr_conflicting_iattrs]
0-gv0-replicate-0: /aa/aa/aa/aa: filetype differs on subvolumes (0, 1)
[2013-11-18 18:18:43.052468] E
[afr-self-heal-common.c:1409:afr_sh_common_lookup_cbk] 0-gv0-replicate-0:
Conflicting entries for /aa/aa/aa/aa
[2013-11-18 18:18:43.052757] E
[afr-self-heal-common.c:2160:afr_self_heal_completion_cbk]
0-gv0-replicate-0: background meta-data data entry missing-entry gfid
self-heal
failed on /aa/aa/aa/aa/aa
[2013-11-18 18:18:43.052780] W [fuse-bridge.c:292:fuse_entry_cbk]
0-glusterfs-fuse: 439382194: LOOKUP() /aa/aa/aa/aa/aa => -1
(Input/output
error)
We’ve looked at the log file etc-glusterfs-glusterd.vol.log but we
found nothing related to this problem. Then, we’ve looked at the log
From /var/log/glusterfs/bricks/mnt-data.log and we found 70 gig of
logs of the same type :
[2013-11-19 17:13:32.269757] W [posix-handle.c:538:posix_handle_soft]
0-gv0-posix: symlink
../../ab/fe/abfeb61c-501d-4417-b8fb-0accdd57146f/cf ->
/mnt/data/.glusterfs/ab/fe/abfeb61c-501d-4417-b8fb-0accdd57146f/cf
failed (No such file or directory)
[2013-11-19 17:13:32.269978] W [posix-handle.c:538:posix_handle_soft]
0-gv0-posix: symlink
../../c7/8b/c78be78f-cc95-47b2-a27f-4217f1759b67/d2 ->
/mnt/data/.glusterfs/c7/8b/c78be78f-cc95-47b2-a27f-4217f1759b67/d2
failed (No such file or directory)
[2013-11-19 17:13:32.270190] W [posix-handle.c:538:posix_handle_soft]
0-gv0-posix: symlink
../../5a/8f/5a8fa43c-4ccc-4d88-9122-a96bc8ffaebc/f2 ->
/mnt/data/.glusterfs/5a/8f/5a8fa43c-4ccc-4d88-9122-a96bc8ffaebc/f2
failed (No such file or directory)
This looks like a bug, unless there is something wrong with the
set-up. I have copied gluster-devel in this thread as I think they might help.
Just curious, is all your gluster nodes have equal time (i.e. ntp synced).
And it does not stop logging. It seems that the self heal is not
working properly when there are broking symlinks in the gluster. It is
worth saying also that this log is only produce on a single node but
the write fail on several node though. Also, we try to clean the
symlinks manually but it always come back.
Is it possible to recover from broken symlinks?
Configuration :
Gluster Version : 3.3.2
Cluster setup : 4 X 2
OS : Ubuntu
On Fuse
Thanks,
Alexandre
_______________________________________________
Gluster-users mailing list address@hidden
http://supercolony.gluster.org/mailman/listinfo/gluster-users
_______________________________________________
Gluster-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/gluster-devel