Re: [Gluster-devel] Error: Transport endpoint is not connected

[JV]

Hello.

There wasn't anything interesting:

================================================================================
Version      : glusterfs 2.0.0rc8 built on Apr 20 2009 23:04:47
TLA Revision : 82394d484803e02e28441bc0b988efaaff60dd94
Starting Time: 2009-04-23 17:17:31
Command line : /usr/local/sbin/glusterfs --log-level=NORMAL 
--volfile=/etc/glusterfs/glusterfs-client.vol /
mnt/glusterfs
PID          : 3759
System name  : Linux
Nodename     : f1
Kernel Release : 2.6.29.1
Hardware Identifier: i686

Given volfile:
+------------------------------------------------------------------------------+
  1: volume g1b1
  2:    type protocol/client
  3:    option transport-type tcp/client
  4:    option remote-host 10.10.70.2
  5:    option remote-subvolume brick1
  6: end-volume
  7: volume g1b2
  8:    type protocol/client
  9:    option transport-type tcp
 10:    option remote-host 10.10.70.2
 11:    option remote-subvolume brick2
 12: end-volume
 13: volume g1b3
 14:    type protocol/client
 15:    option transport-type tcp
 16:    option remote-host 10.10.70.2
 17:    option remote-subvolume brick3
 18: end-volume
 19:
 20: volume g2b1
 21:    type protocol/client
 22:    option transport-type tcp/client
 23:    option remote-host 10.10.70.3
 24:    option remote-subvolume brick1
 25: end-volume
 26: volume g2b2
 27:    type protocol/client
 28:    option transport-type tcp
 29:    option remote-host 10.10.70.3
 30:    option remote-subvolume brick2
 31: end-volume
 32: volume g2b3
 33:    type protocol/client
 34:    option transport-type tcp
 35:    option remote-host 10.10.70.3
 36:    option remote-subvolume brick3
 37: end-volume
 38:
 39: volume replicate1
 40:    type cluster/replicate
 41:    subvolumes g1b1 g2b1
 42: end-volume
 43:
 44: volume replicate2
 45:    type cluster/replicate
 46:    subvolumes g1b2 g2b2
 47: end-volume
 48:
 49: volume replicate3
 50:    type cluster/replicate
 51:    subvolumes g1b3 g2b3
 52: end-volume
 53:
 54: volume distribute
 55:   type cluster/distribute
 56:   subvolumes replicate1 replicate2 replicate3
 57: end-volume
 58:

+------------------------------------------------------------------------------+
2009-04-23 17:17:31 N [glusterfsd.c:1152:main] glusterfs: Successfully 
started
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b1: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate1: subvolume g1b1 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b1: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate1: subvolume g1b1 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b1: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate1: subvolume g2b1 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b2: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate2: subvolume g1b2 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b1: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate1: subvolume g2b1 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b2: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate2: subvolume g1b2 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b2: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate2: subvolume g2b2 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b2: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate2: subvolume g2b2 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b3: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate3: subvolume g1b3 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b3: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate3: subvolume g2b3 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g1b3: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate3: subvolume g1b3 came up
2009-04-23 17:17:31 N [client-protocol.c:6327:client_setvolume_cbk] 
g2b3: connection and handshake succeeded
2009-04-23 17:17:31 N [afr.c:2126:notify] replicate3: subvolume g2b3 came up
pending frames:

patchset: 82394d484803e02e28441bc0b988efaaff60dd94
signal received: 6
configuration details:argp 1
backtrace 1
dlfcn 1
fdatasync 1
libpthread 1
llistxattr 1
setfsid 1
spinlock 1
epoll.h 1
xattr.h 1
st_atim.tv_nsec 1
package-string: glusterfs 2.0.0rc8
[0xb803c400]
/lib/i686/cmov/libc.so.6(abort+0x188)[0xb7eb3018]
/lib/i686/cmov/libc.so.6(__assert_fail+0xee)[0xb7eaa5be]
/lib/i686/cmov/libpthread.so.0(pthread_mutex_lock+0x5d4)[0xb7fe8f54]
/usr/lib/libfuse.so.2[0xb75c0425]
/usr/local/lib/glusterfs/2.0.0rc8/xlator/mount/fuse.so[0xb75d519a]
/usr/lib/libfuse.so.2[0xb75c1cf8]
/usr/lib/libfuse.so.2[0xb75c2211]
/usr/lib/libfuse.so.2(fuse_session_process+0x26)[0xb75c3bf6]
/usr/local/lib/glusterfs/2.0.0rc8/xlator/mount/fuse.so[0xb75d5f31]
/lib/i686/cmov/libpthread.so.0[0xb7fe74c0]
/lib/i686/cmov/libc.so.6(clone+0x5e)[0xb7f666de]
---------


Anyhow - it seems to be working without problems using
git version 689347f278e5acfda95a24f7804a1450043311f3

Also it seems to be working faster without any performance translators, 
however gluster is using only part of dedicated GigE link - no more than 
25% for writing and 30% for reading on client. Half that on each storage 
node. What can I do to improve this?

P.S. sorry for the late response -  your message got into junk folder.

JV

Anand Avati wrote:
> There has to be some more before this. Can you paste the full log?
>
> Avati
>
> > in client log is only this:
> >
> > pending frames:
> >
> > patchset: 82394d484803e02e28441bc0b988efaaff60dd94
> > signal received: 6
> > configuration details:argp 1
> > backtrace 1
> > dlfcn 1
> > fdatasync 1
> > libpthread 1
> > llistxattr 1
> > setfsid 1
> > spinlock 1
> > epoll.h 1
> > xattr.h 1
> > st_atim.tv_nsec 1
> > package-string: glusterfs 2.0.0rc8
> > [0xb8067400]
> > /lib/i686/cmov/libc.so.6(abort+0x188)[0xb7ede018]
> > /lib/i686/cmov/libc.so.6(__assert_fail+0xee)[0xb7ed55be]
> > /lib/i686/cmov/libpthread.so.0(pthread_mutex_lock+0x5d4)[0xb8013f54]
> > /usr/lib/libfuse.so.2[0xb75d3147]
> > /usr/lib/libfuse.so.2(fuse_session_process+0x26)[0xb75d4bf6]
> > /usr/local/lib/glusterfs/2.0.0rc8/xlator/mount/fuse.so[0xb75eef31]
> > /lib/i686/cmov/libpthread.so.0[0xb80124c0]
> > /lib/i686/cmov/libc.so.6(clone+0x5e)[0xb7f916de]
> > ---------