[Gluster-devel] deadlock in locks xlator

[Pranith Kumar Karampuri]

I ran 'iozone -a' on the mount on master and saw that there is a deadlock:
It should most probably exist even in 3.5 It seems like inodelk blocked list is 
getting corrupted. I need to debug a bit more to comeup with the fix. Will post 
again with more information.

(gdb) info threads
  Id   Target Id         Frame 
  23   Thread 0x7f2f82657700 (LWP 17395) "glusterfsd" 0x00000034eea0ed41 in 
sigwait () from /lib64/libpthread.so.0
  22   Thread 0x7f2f81e56700 (LWP 17396) "glusterfsd" 0x00000034eea0b922 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  21   Thread 0x7f2f81655700 (LWP 17397) "glusterfsd" 0x00000034eea0b922 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  20   Thread 0x7f2f80a17700 (LWP 17398) "glusterfsd" 0x00000034eea0e80d in 
nanosleep () from /lib64/libpthread.so.0
  19   Thread 0x7f2f7a8d5700 (LWP 17399) "glusterfsd" 0x00000034eea0b575 in 
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  18   Thread 0x7f2f7a7d4700 (LWP 17400) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  17   Thread 0x7f2f7a6d3700 (LWP 17401) "glusterfsd" 0x00000034ee6bc71d in 
nanosleep () from /lib64/libc.so.6
  16   Thread 0x7f2f79ed2700 (LWP 17402) "glusterfsd" 0x00000034eea0b922 in 
pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  15   Thread 0x7f2f796d1700 (LWP 17403) "glusterfsd" 0x00000034eea0b575 in 
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  14   Thread 0x7f2f78d98700 (LWP 17523) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  13   Thread 0x7f2f78c97700 (LWP 17524) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  12   Thread 0x7f2f78b96700 (LWP 17533) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  11   Thread 0x7f2f78a95700 (LWP 17534) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  10   Thread 0x7f2f78994700 (LWP 17535) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  9    Thread 0x7f2f78893700 (LWP 17536) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
* 8    Thread 0x7f2f78792700 (LWP 17537) "glusterfsd" 0x00000034ee67ad27 in 
_IO_str_init_static_internal () from /lib64/libc.so.6 <<------------- This is 
going in infinite loop in 'grant blocked locks'.
  7    Thread 0x7f2f78691700 (LWP 17538) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  6    Thread 0x7f2f78590700 (LWP 17540) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  5    Thread 0x7f2f7848f700 (LWP 17541) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  4    Thread 0x7f2f7838e700 (LWP 17701) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  3    Thread 0x7f2f7828d700 (LWP 17702) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  2    Thread 0x7f2f7818c700 (LWP 17708) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0
  1    Thread 0x7f2f8983e740 (LWP 17394) "glusterfsd" 0x00000034eea0dded in 
__lll_lock_wait () from /lib64/libpthread.so.0


Pranith