[Gluster-devel] LOOKUP conflict => OPEN fails_

[Fredrik Widlund]

Hi,

 

I’m running a simple AFR setup, thouch currently with only one backend, and 2 tcp clients. Version is 3.0.0 from jan 20.

 

Basically one client is writing a large number of files, continuously, and the other client is reading.

 

I have a growing problem with lookup “conflicts”, resulting in files being listed in directories but where reads are returning “-1 (No such file…”.

 

Restarting the client does not solve the conflict, but restarting the server does and the files becomes available again.

 

The filesystem is a 5TB XFS hw raid-5 with around 150k files.

 

Debug trace of client:

[2010-02-08 13:39:29] N [trace.c:148:trace_open_cbk] replicated: 3073: (op_ret=0, op_errno=117, *fd=0x129a430)

[2010-02-08 13:39:37] N [trace.c:1837:trace_open] replicated: 3094: (loc {path=/download/90910/live/webb1/webb1/Layer3/prog_index.m3u8, ino=5042185}, flags=32768, fd=0x1296fc0, wbflags=0)

[2010-02-08 13:39:37] N [trace.c:148:trace_open_cbk] replicated: 3094: (op_ret=-1, op_errno=2, *fd=0x1296fc0)

[2010-02-08 13:39:37] W [fuse-bridge.c:858:fuse_fd_cbk] glusterfs-fuse: 3094: OPEN() /download/90910/live/webb1/webb1/Layer3/prog_index.m3u8 => -1 (No such file or directory)

[2010-02-08 13:39:38] N [trace.c:1837:trace_open] replicated: 3100: (loc {path=/download/90910/live/webb1/webb1/Layer4/prog_index.m3u8, ino=5013773}, flags=32768, fd=0x1296fc0, wbflags=0)

[2010-02-08 13:39:38] N [trace.c:148:trace_open_cbk] replicated: 3100: (op_ret=0, op_errno=117, *fd=0x1296fc0)

[2010-02-08 13:39:38] N [trace.c:1837:trace_open] replicated: 3106: (loc {path=/download/90910/live/webb1/webb1/Layer4/Period1/segment277.ts, ino=5050371}, flags=32768, fd=0x129a430, wbflags=0)

[…]

 

And server:

[2010-02-08 13:39:09] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7fedee4e43f3

[2010-02-08 13:39:09] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7fedee4e440b

[2010-02-08 13:39:17] D [server-protocol.c:2037:server_open_cbk] server: 1719: OPEN (null) (0) ==> -1 (No such file or directory)

[2010-02-08 13:39:18] D [server-protocol.c:2037:server_open_cbk] server: 1724: OPEN (null) (0) ==> -1 (No such file or directory)

[2010-02-08 13:39:28] D [server-resolve.c:238:resolve_path_deep] store0: RESOLVE OPEN() seeking deep resolution of /download/90910/live/webb1/webb1/Layer3/prog_index.m3u8

[2010-02-08 13:39:28] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7fedee4e43db

[2010-02-08 13:39:28] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7fedee4e43f3

[2010-02-08 13:39:28] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7fedee4e440b

[2010-02-08 13:39:28] D [dict.c:303:dict_get] dict: @this=(nil) @key=0x7fedee4e43db

[…]

 

Kind regards,

Fredrik Widlund