gluster-devel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Gluster-devel] QEMU (and other libgfapi client?) crashes on add-bri


From: Anand Avati
Subject: Re: [Gluster-devel] QEMU (and other libgfapi client?) crashes on add-brick / replace-brick
Date: Sat, 27 Jul 2013 21:04:20 -0700

Guido,
You need a couple of fixes:

http://review.gluster.org/5378 - to fix the root cause of the failure (portmap failure)
http://review.gluster.org/5407 - to prevent a crash in case of such failures.

Can you please apply the patches and confirm if they fix your issue?

Thanks,
Avati


On Fri, Jul 26, 2013 at 4:02 PM, Guido De Rosa <address@hidden> wrote:
2013/7/26 Anand Avati <address@hidden>:
> Can you please post the backtrace and logs from the crash?

# gdb
...
(gdb) file /usr/local/bin/qemu-system-x86_64
Reading symbols from /usr/local/bin/qemu-system-x86_64...done.
(gdb) run -uuid "e06dc280-d74a-0130-49e3-003018a4d17c" -name
"deb-on-gluster" -m "2048" -vnc ":1" -k "it" -pidfile
"/var/run/onboard/qemu-e06dc280.pid" -monitor
unix:"/var/run/onboard/qemu-e06dc280.sock",server,nowait -smp 2
-device piix3-usb-uhci,id=piix3-uhci -device usb-ehci,id=ehci -device
usb-tablet,bus=piix3-uhci.0 -drive
serial="QME06DC28000",if="virtio",media="disk",cache="unsafe",file="gluster://localhost/gvtest/QEMU/deb-on-gluster/disk0.qcow2",index=0
-drive serial="QME06DC28001",if="ide",media="cdrom",bus=1,unit=0 -net
nic,vlan=0,model=virtio,macaddr=DE:AD:BE:6C:CF:30 -net
tap,vlan=0,ifname=vDebOnGluster_0,script=no,downscript=no -boot
menu=on,order=dc -cpu host -enable-kvm -runas onboard

...

On another terminal:

# gluster volume add-brick gvtest replica 2
192.168.232.101:/var/export/gluster/gvtest
volume add-brick: success

And here the crash:

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffe5dad700 (LWP 8611)]
[New Thread 0x7fffe55ac700 (LWP 8618)]
[New Thread 0x7fffe4548700 (LWP 8621)]
[New Thread 0x7fffe3b33700 (LWP 8622)]
[New Thread 0x7fffdbdfd700 (LWP 8623)]
[New Thread 0x7fffdb5fc700 (LWP 8624)]
[New Thread 0x7fffd9dff700 (LWP 8626)]
[New Thread 0x7fffc7fff700 (LWP 8627)]
[New Thread 0x7fffc63fe700 (LWP 8628)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffe5dad700 (LWP 8611)]
glfs_subvol_done (fs=0x5555566cfc20, address@hidden) at
glfs-resolve.c:802
802 glfs-resolve.c: No such file or directory.
(gdb)
(gdb) bt
#0  glfs_subvol_done (fs=0x5555566cfc20, address@hidden) at
glfs-resolve.c:802
#1  0x00007ffff70aff50 in glfs_pwritev (glfd=0x5555566d79b0,
iovec=<optimized out>,
    iovcnt=<optimized out>, offset=8368975872, flags=0) at glfs-fops.c:761
#2  0x00007ffff70b04e7 in glfs_io_async_task (data="" out>) at
glfs-fops.c:584
#3  0x00007ffff0080b22 in synctask_wrap (old_task=<optimized out>) at
syncop.c:131
#4  0x00007ffff0541710 in ?? () from /lib/x86_64-linux-gnu/libc.so.6
#5  0x0000000000000000 in ?? ()


Log: /var/log/glusterfs/etc-glusterfs-glusterd.vol.log

[2013-07-26 22:27:45.877325] I
[glusterd-brick-ops.c:370:__glusterd_handle_add_brick] 0-management:
Received add brick req
[2013-07-26 22:27:45.877398] I
[glusterd-brick-ops.c:417:__glusterd_handle_add_brick] 0-management:
repl
ica-count is 2
[2013-07-26 22:27:45.877415] I
[glusterd-brick-ops.c:193:gd_addbr_validate_replica_count]
0-management:
Changing the type of volume gvtest from 'distribute' to 'replica'
[2013-07-26 22:27:45.879877] I
[glusterd-brick-ops.c:894:glusterd_op_perform_add_bricks]
0-management: r
eplica-count is set 2
[2013-07-26 22:27:45.879907] I
[glusterd-brick-ops.c:898:glusterd_op_perform_add_bricks]
0-management: t
ype is set 2, need to change it
[2013-07-26 22:27:45.882314] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:45.882349] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:45.883952] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:45.883978] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:45.884704] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:46.617898] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:46.617945] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:47.618736] E
[glusterd-utils.c:3627:glusterd_nodesvc_unlink_socket_file]
0-management:
[2013-07-26 22:27:45.877325] I
[glusterd-brick-ops.c:370:__glusterd_handle_add_brick] 0-management:
Rece
ived add brick req
[2013-07-26 22:27:45.877398] I
[glusterd-brick-ops.c:417:__glusterd_handle_add_brick] 0-management:
repl
ica-count is 2
[2013-07-26 22:27:45.877415] I
[glusterd-brick-ops.c:193:gd_addbr_validate_replica_count]
0-management:
Changing the type of volume gvtest from 'distribute' to 'replica'
[2013-07-26 22:27:45.879877] I
[glusterd-brick-ops.c:894:glusterd_op_perform_add_bricks]
0-management: r
eplica-count is set 2
[2013-07-26 22:27:45.879907] I
[glusterd-brick-ops.c:898:glusterd_op_perform_add_bricks]
0-management: t
ype is set 2, need to change it
[2013-07-26 22:27:45.882314] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:45.882349] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:45.883952] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:45.883978] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:45.884704] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:46.617898] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:46.617945] I
[glusterd-utils.c:954:glusterd_volume_brickinfo_get] 0-management:
Found
brick
[2013-07-26 22:27:47.618736] E
[glusterd-utils.c:3627:glusterd_nodesvc_unlink_socket_file]
0-management:
[2013-07-26 22:27:47.619092] I
[glusterd-utils.c:3661:glusterd_nfs_pmap_deregister] 0-: De-registered
MOUNTV3 successfully
[2013-07-26 22:27:47.619386] I
[glusterd-utils.c:3666:glusterd_nfs_pmap_deregister] 0-: De-registered
MOUNTV1 successfully
[2013-07-26 22:27:47.619682] I
[glusterd-utils.c:3671:glusterd_nfs_pmap_deregister] 0-: De-registered
NFSV3 successfully
[2013-07-26 22:27:47.620004] I
[glusterd-utils.c:3676:glusterd_nfs_pmap_deregister] 0-: De-registered
NLM v4 successfully
[2013-07-26 22:27:47.620343] I
[glusterd-utils.c:3681:glusterd_nfs_pmap_deregister] 0-: De-registered
NLM v1 successfully
[2013-07-26 22:27:47.622905] I
[rpc-clnt.c:962:rpc_clnt_connection_init] 0-management: setting
frame-timeout to 600
[2013-07-26 22:27:47.622988] I [socket.c:3480:socket_init]
0-management: SSL support is NOT enabled
[2013-07-26 22:27:47.623003] I [socket.c:3495:socket_init]
0-management: using system polling thread
[2013-07-26 22:27:48.650339] E
[glusterd-utils.c:3627:glusterd_nodesvc_unlink_socket_file]
0-management: Failed to remove
/var/run/52b6ed075a07af2e9235e49dd9d214af.socke
t error: No such file or directory
[2013-07-26 22:27:48.652919] I
[rpc-clnt.c:962:rpc_clnt_connection_init] 0-management: setting
frame-timeout to 600
[2013-07-26 22:27:48.653002] I [socket.c:3480:socket_init]
0-management: SSL support is NOT enabled
[2013-07-26 22:27:48.653017] I [socket.c:3495:socket_init]
0-management: using system polling thread
[2013-07-26 22:27:48.653215] W [socket.c:514:__socket_rwv]
0-management: readv failed (No data available)
[2013-07-26 22:27:48.653333] I [mem-pool.c:541:mem_pool_destroy]
0-management: size=2236 max=0 total=0
[2013-07-26 22:27:48.653354] I [mem-pool.c:541:mem_pool_destroy]
0-management: size=124 max=0 total=0
[2013-07-26 22:27:48.653398] I [socket.c:2236:socket_event_handler]
0-transport: disconnecting now
[2013-07-26 22:27:48.653453] W [socket.c:514:__socket_rwv]
0-management: readv failed (No data available)
[2013-07-26 22:27:48.653478] I [mem-pool.c:541:mem_pool_destroy]
0-management: size=2236 max=0 total=0
[2013-07-26 22:27:48.653493] I [mem-pool.c:541:mem_pool_destroy]
0-management: size=124 max=0 total=0
[2013-07-26 22:27:48.653537] I [socket.c:2236:socket_event_handler]
0-transport: disconnecting now
[2013-07-26 22:27:48.680240] E [rpcsvc.c:519:rpcsvc_handle_rpc_call]
0-glusterd: Request received from non-privileged port. Failing request
[2013-07-26 22:27:52.093567] E [rpcsvc.c:519:rpcsvc_handle_rpc_call]
0-glusterd: Request received from non-privileged port. Failing request
[2013-07-26 22:27:55.094469] E [rpcsvc.c:519:rpcsvc_handle_rpc_call]
0-glusterd: Request received from non-privileged port. Failing request

--

G.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]