Hi all,
I have 3 peers (peer-A, peer-B and peer-C). I tried to use add-brick and remove-brick to replace peers.
(version: glusterfs 3.4)
What I did:
- created a distribute volume with two 10-GB bricks (peer-A:/brick and peer-B:/brick. Actually they are 9.7 GB after ext4 formatting).
- mount it and write 16 1-GB files in to it (command: seq 16 | xargs -i dd if=/dev/zero of=/mnt/file-{} bs=1G count=1).
- add peer-C:/brick (also 10-GB) to this volume.
- execute remove peer-A:/brick start.
- check remove status and wait until all of the hosts are completed.
- execute remove peer-A:/brick commit.
After step 6, I lost 2 files in the volume.
I list the files in bricks after step 2 and step 5:
After step 2:
peer-A:/brick:
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-1
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-12
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-14
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-15
-rw-r--r-- 2 root root 1073741824 Mar 4 17:08 file-16
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-3
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-6
peer-B:/brick:
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-10
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-11
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-13
---------T 2 root root 0 Mar 4 17:07 file-15
---------T 2 root root 0 Mar 4 17:07 file-16
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-2
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-4
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-5
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-7
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-8
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-9
After step 5:
peer-A:/brick:
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-15
-rw-r--r-- 2 root root 1073741824 Mar 4 17:08 file-16
peer-B:/brick:
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-10
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-11
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-13
---------T 2 root root 1073741824 Mar 4 17:17 file-15
---------T 2 root root 1073741824 Mar 4 17:17 file-16
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-2
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-4
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-5
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-7
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-8
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-9
peer-C:/brick:
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-1
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-12
-rw-r--r-- 2 root root 1073741824 Mar 4 17:07 file-14
-rw-r--r-- 2 root root 1073741824 Mar 4 17:05 file-3
-rw-r--r-- 2 root root 1073741824 Mar 4 17:06 file-6
After step 6, I can't find file-15 and file-16 in the volume.
Anyone know why file-15 and file-16 are not moved to peer-C?
If it's caused by peer-B is full, why does the status show "completed"?
Node Rebalanced-files size scanned failures skipped status run-time in secs
--------- ----------- ----------- ----------- ----------- ----------- ------------ --------------
localhost 5 5.0GB 21 0 completed 126.00
localhost 5 5.0GB 21 0 completed 126.00
localhost 5 5.0GB 21 0 completed 126.00
localhost 5 5.0GB 21 0 completed 126.00
--
Best regards,
Johnny