[Gluster-devel] OS X client wrong disk size in combination with DHT
From:
Ate Poorthuis
Subject:
[Gluster-devel] OS X client wrong disk size in combination with DHT
Date:
Fri, 3 Jul 2009 14:45:57 +0200
Hi,
Using gluster 2.0.2 client os OS X 10.5. Disk size is not outputted correctly, only the size of the first brick in dht is counted. Used disk space is calculated correctly.
OS X client: /Users/ate/glusterfs_ip.vol 5.4Ti 1.1Ti 3.2Ti 26% /mnt/gluster
Debian client: /etc/glusterfs/client_st.vol 22T 1.1T 20T 6% /mnt/gluster
There are also some errors from dht-diskusage in the log about full disks. The % differs each time I mount (ranging from 96-100%).
================================================================================ Version : glusterfs 2.0.2 built on Jun 30 2009 14:14:33 TLA Revision : 07019da2e16534d527215a91904298ede09bb798 Starting Time: 2009-07-03 14:40:34
Command line : /usr/local/gluster/sbin/glusterfs --debug -f /Users/ate/glusterfs_ip.vol /mnt/gluster/ PID : 97785 System name : Darwin Nodename : ate-poorthuiss-macbook-2.local Kernel Release : 9.7.0
Hardware Identifier: i386
Given volfile: +------------------------------------------------------------------------------+ 1: ### file: client-volume.vol 2: 3: #####################################
4: ### GlusterFS Client Volume File ## 5: ##################################### 6: 7: #### CONFIG FILE RULES: 8: ### "#" is comment character. 9: ### - Config file is case sensitive
10: ### - Options within a volume block can be in any order. 11: ### - Spaces or tabs are used as delimitter within a line. 12: ### - Each option should end within a line. 13: ### - Missing or commented fields will assume default values.
14: ### - Blank/commented lines are allowed. 15: ### - Sub-volumes should already be defined above before referring. 16: 17: ### Add client feature and attach to remote subvolume 18: volume gfs-001-afr1
19: type protocol/client 20: option transport-type tcp 21: option remote-host 10.0.0.30 # IP address of the remote brick 22: option remote-subvolume afr1 # name of the remote volume
23: # option ping-timeout 5 24: end-volume 25: volume gfs-001-afr2 26: type protocol/client 27: option transport-type tcp 28: option remote-host 10.0.0.30 # IP address of the remote brick
29: option remote-subvolume afr2 # name of the remote volume 30: # option ping-timeout 5 31: end-volume 32: volume gfs-002-afr1 33: type protocol/client 34: option transport-type tcp
35: option remote-host 10.0.0.32 # IP address of the remote brick 36: option remote-subvolume afr1 # name of the remote volume 37: # option ping-timeout 5 38: end-volume 39: volume gfs-002-afr2
40: type protocol/client 41: option transport-type tcp 42: option remote-host 10.0.0.32 # IP address of the remote brick 43: option remote-subvolume afr2 # name of the remote volume
44: # option ping-timeout 5 45: end-volume 46: 47: volume bricks 48: type cluster/distribute 49: # option lookup-unhashed yes 50: option min-free-disk 5% 51: subvolumes gfs-001-afr1 gfs-001-afr2 gfs-002-afr1 gfs-002-afr2
52: end-volume 53: 54: ### Add readahead feature 55: volume readahead 56: type performance/read-ahead 57: option page-count 8 # cache per file = (page-count x page-size) 58: subvolumes bricks
59: end-volume 60: 61: ### Add IO-Cache feature 62: volume iocache 63: type performance/io-cache 64: option cache-size 256MB 65: subvolumes readahead 66: end-volume 67: volume iothreads
68: type performance/io-threads 69: option thread-count 32 # default is 1 70: subvolumes iocache 71: end-volume 72: ### Add writeback feature 73: volume writeback 74: type performance/write-behind
75: option cache-size 2MB 76: option flush-behind off 77: subvolumes iothreads 78: end-volume 79:
+------------------------------------------------------------------------------+ [2009-07-03 14:40:34] D [xlator.c:634:xlator_set_type] xlator: dlsym(notify) on dlsym(0x101730, notify): symbol not found -- neglecting
[2009-07-03 14:40:34] D [xlator.c:634:xlator_set_type] xlator: dlsym(notify) on dlsym(0x101a40, notify): symbol not found -- neglecting [2009-07-03 14:40:34] D [xlator.c:634:xlator_set_type] xlator: dlsym(notify) on dlsym(0x101d50, notify): symbol not found -- neglecting
[2009-07-03 14:40:34] D [xlator.c:634:xlator_set_type] xlator: dlsym(notify) on dlsym(0x102060, notify): symbol not found -- neglecting [2009-07-03 14:40:34] D [glusterfsd.c:256:_add_fuse_mount] glusterfs: 'direct-io-mode' in fuse causes data corruption if O_APPEND is used. disabling 'direct-io-mode'
[2009-07-03 14:40:34] D [glusterfsd.c:1179:main] glusterfs: running in pid 97785 [2009-07-03 14:40:34] D [client-protocol.c:5948:init] gfs-001-afr1: defaulting frame-timeout to 30mins [2009-07-03 14:40:34] D [client-protocol.c:5959:init] gfs-001-afr1: defaulting ping-timeout to 10
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so [2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [client-protocol.c:5948:init] gfs-001-afr2: defaulting frame-timeout to 30mins [2009-07-03 14:40:34] D [client-protocol.c:5959:init] gfs-001-afr2: defaulting ping-timeout to 10 [2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so [2009-07-03 14:40:34] D [client-protocol.c:5948:init] gfs-002-afr1: defaulting frame-timeout to 30mins
[2009-07-03 14:40:34] D [client-protocol.c:5959:init] gfs-002-afr1: defaulting ping-timeout to 10 [2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so [2009-07-03 14:40:34] D [client-protocol.c:5948:init] gfs-002-afr2: defaulting frame-timeout to 30mins
[2009-07-03 14:40:34] D [client-protocol.c:5959:init] gfs-002-afr2: defaulting ping-timeout to 10 [2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so
[2009-07-03 14:40:34] D [transport.c:141:transport_load] transport: attempt to load file /usr/local/gluster/lib/glusterfs/2.0.2/transport/socket.so [2009-07-03 14:40:34] D [read-ahead.c:786:init] readahead: Using conf->page_count = 8
[2009-07-03 14:40:34] D [io-threads.c:2280:init] iothreads: io-threads: Autoscaling: off, min_threads: 32, max_threads: 32 [2009-07-03 14:40:34] D [write-behind.c:1859:init] writeback: disabling write-behind for first 1 bytes
[2009-07-03 14:40:34] D [dict.c:297:dict_get] dict: @this=0x0 @key=0x22dd0b [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr1: got GF_EVENT_PARENT_UP, attempting connect on transport [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr1: got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr2: got GF_EVENT_PARENT_UP, attempting connect on transport [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr2: got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr1: got GF_EVENT_PARENT_UP, attempting connect on transport [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr1: got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr2: got GF_EVENT_PARENT_UP, attempting connect on transport [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr2: got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr1: got GF_EVENT_PARENT_UP, attempting connect on transport [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr1: got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr2: got GF_EVENT_PARENT_UP, attempting connect on transport [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-001-afr2: got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr1: got GF_EVENT_PARENT_UP, attempting connect on transport [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr1: got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr2: got GF_EVENT_PARENT_UP, attempting connect on transport [2009-07-03 14:40:34] D [client-protocol.c:6276:notify] gfs-002-afr2: got GF_EVENT_PARENT_UP, attempting connect on transport
[2009-07-03 14:40:34] N [glusterfsd.c:1198:main] glusterfs: Successfully started [2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-001-afr1: got GF_EVENT_CHILD_UP [2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-001-afr1: got GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-001-afr2: got GF_EVENT_CHILD_UP [2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-001-afr2: got GF_EVENT_CHILD_UP [2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-002-afr1: got GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-002-afr1: got GF_EVENT_CHILD_UP [2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-002-afr2: got GF_EVENT_CHILD_UP [2009-07-03 14:40:34] D [client-protocol.c:6290:notify] gfs-002-afr2: got GF_EVENT_CHILD_UP
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk] gfs-001-afr1: Connected to 10.0.0.30:6996, attached to remote volume 'afr1'. [2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk] gfs-001-afr1: Connected to 10.0.0.30:6996, attached to remote volume 'afr1'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk] gfs-001-afr2: Connected to 10.0.0.30:6996, attached to remote volume 'afr2'. [2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk] gfs-002-afr1: Connected to 10.0.0.32:6996, attached to remote volume 'afr1'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk] gfs-002-afr1: Connected to 10.0.0.32:6996, attached to remote volume 'afr1'. [2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk] gfs-002-afr2: Connected to 10.0.0.32:6996, attached to remote volume 'afr2'.
[2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk] gfs-002-afr2: Connected to 10.0.0.32:6996, attached to remote volume 'afr2'. [2009-07-03 14:40:34] N [client-protocol.c:5551:client_setvolume_cbk] gfs-001-afr2: Connected to 10.0.0.30:6996, attached to remote volume 'afr2'.
[2009-07-03 14:40:35] D [dht-common.c:1405:dht_err_cbk] bricks: subvolume gfs-001-afr1 returned -1 (Invalid argument) [2009-07-03 14:40:51] C [dht-diskusage.c:197:dht_is_subvol_filled] bricks: disk space on subvolume 'gfs-001-afr1' is getting full (97.00 %), consider adding more nodes
[2009-07-03 14:40:51] W [dht-diskusage.c:231:dht_free_disk_available_subvol] bricks: No subvolume has enough free space to create