Re: [Gluster-devel] unify: other clients cannot read files

[David Mayr - Hetzner Online AG]

Hi all,

I got the solution to the problem and am a bit confused.

The working config can be found at  http://gluster.pastebin.org/19526
I'm now using srv9 only as namespace, not as storage any more.

I used one of the bricks (srv9) as storage brick *and* as namespace brick 
before. After having created a file from a client, I noticed that the file 
was on one of the storage nodes and in the *storage* directory of the 
namespace node (srv9) with zero bytes (and not in the namespace directory).

I seems that it is not possible any more to define / use more than one volume 
per server or rather it works not as expected.

David


Am Freitag 08 Februar 2008 13:12:25 schrieb David Mayr - Hetzner Online AG:
> Hi Avati,
>
> >  Can you upgrade to patch-650 and start with an empty backend? Suspecting
> > that you were effected by the bug ~640 which might have created 0 byte
> > files in the backend (which can cause such a behaviour). We are curious
> > to know if patch-650 (with an empty backend) works for you..
>
> I just updated to patch-650 on all servers and clients, removed all my
> storage and namespace dirs on the servers, restarted servers and clients
> (which recreated the storage/namespace dirs automatically) and
> unfortunately have still the same problem:
> files created by client-1 can not be read by client-2.
>
> client-2 logfile (level 'warning') says now while trying to read a file
> created by client-1:
>
> E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 76: /testfile => -1
> (116) E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 77: /testfile =>
> -1 (116) E [unify.c:849:unify_open] unify: /testfile: entry_count is 3
> E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 78: /testfile => -1 (5)
>
> David
>
> > 2008/2/8, David Mayr - Hetzner Online AG <address@hidden>:
> > > Hi Avati,
> > >
> > > > Is the client spec file same on both clients?
> > >
> > > Yes, the client specs are the same on both clients.
> > >
> > > > can you show the client log of client-2?
> > >
> > > with loglevel DEBUG I see the following on client-2 while reading (with
> > > dd) a
> > > file written by client-1 (timestamps removed):
> > >
> > > D [fuse-bridge.c:492:fuse_lookup] glusterfs-fuse: 2: LOOKUP /testfile-2
> > > D [fuse-bridge.c:354:fuse_entry_cbk] glusterfs-fuse: 2: /testfile-2 =>
> > > 1972
> > > D [inode.c:559:__create_inode] fuse/inode: create inode(1972)
> > > D [inode.c:351:__active_inode] fuse/inode: activating inode(1972),
> > > lru=0/1024
> > > D [inode.c:381:__passive_inode] fuse/inode: passivating inode(1972),
> > > lru=1/1024
> > > D [inode.c:308:__destroy_inode] fuse/inode: destroy inode(0)
> > > address@hidden D [inode.c:351:__active_inode] fuse/inode: activating
> > > inode(1972), lru=0/1024
> > > D [fuse-bridge.c:1470:fuse_open] glusterfs-fuse: 3: OPEN /testfile-2
> > > E [unify.c:875:unify_open] unify: /testfile-2: entry_count is 3
> > > E [fuse-bridge.c:670:fuse_fd_cbk] glusterfs-fuse: 3: /testfile-2 => -1
> > > (5) D [inode.c:381:__passive_inode] fuse/inode: passivating
> > > inode(1972), lru=1/1024
> > >
> > >
> > > when I only do an ls /glusterfs/filename I see the following on
> > > client-2:
> > >
> > > D [inode.c:351:__active_inode] fuse/inode: activating inode(1972),
> > > lru=0/1024
> > > D [fuse-bridge.c:500:fuse_lookup] glusterfs-fuse: 4: LOOKUP
> > > /testfile-2(1972)
> > > E [fuse-bridge.c:431:fuse_entry_cbk] glusterfs-fuse: 4: /testfile-2 =>
> > > -1 (116)
> > > D [fuse-bridge.c:437:fuse_entry_cbk] glusterfs-fuse: unlinking stale
> > > dentry
> > > for `/testfile-2'
> > > D [fuse-bridge.c:354:fuse_entry_cbk] glusterfs-fuse: 4: /testfile-2 =>
> > > 1972
> > > D [inode.c:308:__destroy_inode] fuse/inode: destroy inode(0)
> > > address@hidden D [inode.c:381:__passive_inode] fuse/inode: passivating
> > > inode(1972), lru=1/1024
> > >
> > > > also which version of glusterfs/fuse are you using?
> > >
> > > - debian etch
> > > - kernel 2.6.18-5-686-bigmem
> > > - glusterfs tla 643  (debian packages from
> > > http://projects.nlogy.cz/debian )
> > > - fuse-2.7.2glfs8 from
> > >
> > > http://ftp.zresearch.com/pub/gluster/glusterfs/fuse/fuse-2.7.2glfs8.tar
> > >.g z
> > >
> > >
> > > David
> > >
> > > > 2008/1/29, David Mayr - Hetzner Online AG <address@hidden>:
> > > > > Hi Avati,
> > > > >
> > > > > > > Hi GlusterFS-Team,
> > > > > > >
> > > > > > > first thank you for this great piece of software!
> > > > > > >
> > > > > > > I try glusterfs for testing at the moment on 9 servers and two
> > > > > > > clients with
> > > > > > > unify - for config see
> > > > > > > http://glusterfs.pastebin.com/m7a74bd29(client) and
> > > > > > > http://glusterfs.pastebin.com/m206785bc (server).
> > > > > > >
> > > > > > > I'm using TLA 643 of the packages from
> > > > > > > http://projects.nlogy.cz/debian (camel1cz builds them nightly),
> > >
> > > but
> > >
> > > > > > > saw the following problem already last week with earlier patch
> > > > > > > versions.
> > > > > > >
> > > > > > > When I write files with client-1, I can stat and read these
> > > > > > > files from client-1 without problems. But I get wrong results
> > > > > > > doing an
> > >
> > > "ls
> > >
> > > > > > > -l"
> > > > >
> > > > > (zero
> > > > >
> > > > > > > bytes) on client-2 and cannot read the file on client-2.
> > > > > > >
> > > > > > > See the following example:
> > > > > > >
> > > > > > >
> > > > > > > ===> on client-1:
> > > > > > >
> > > > > > > address@hidden ~ # dd if=/dev/zero of=/glusterfs/testfile bs=1M
> > > > >
> > > > > count=100
> > > > >
> > > > > > > 104857600 bytes (105 MB) copied, 1.9773 seconds, 53.0 MB/s
> > > > > > >
> > > > > > > address@hidden ~ # ls -lah /glusterfs/testfile
> > > > > > > -rw-r--r-- 1 root root 100M 2008-01-28 12:48
> > > > > > > /glusterfs/testfile
> > > > > > >
> > > > > > > address@hidden ~ # dd if=/glusterfs/testfile of=/dev/null
> > > > > > > 104857600 bytes (105 MB) copied, 2.55219 seconds, 41.1 MB/s
> > > > > > >
> > > > > > >
> > > > > > > ===> on client-2:
> > > > > > >
> > > > > > > address@hidden ~ # ls -lah /glusterfs/testfile
> > > > > > > -rw-r--r-- 1 root root 0 2008-01-28 12:47 /glusterfs/testfile
> > > > > > > (=> zero byte size)
> > > > > > >
> > > > > > > address@hidden ~ # dd if=/glusterfs/testfile of=/dev/null
> > > > > > > dd: opening `/glusterfs/testfile': Input/output error
> > > > > > >
> > > > > > > David
> > > > > >
> > > > > > In your client spec I do not see 'ns9' being declared as a
> > > > >
> > > > > protocol/client.
> > > > >
> > > > > > Please declare the global namespace as it is necssary for shared
> > > > > > access. avati
> > > > >
> > > > > Oops, I'm sorry - forgot that while cleaning up (removing comments
> > > > > and unnecessary) my config files.
> > > > > In my original files the ns9-entry exists.
> > > > >
> > > > > I re-posted my config files here:
> > > > > server:   http://glusterfs.pastebin.com/m96b2583
> > > > > client:    http://glusterfs.pastebin.com/m6c8472aa
> > > > >
> > > > > Any ideas why clients cannot read files of other clients?
> > > > >
> > > > > --
> > > > > David
> > >
> > > _______________________________________________
> > > Gluster-devel mailing list
> > > address@hidden
> > > http://lists.nongnu.org/mailman/listinfo/gluster-devel



-- 
Mit freundlichen Grüßen

  David Mayr

Hetzner Online AG
Softwareentwicklung RZ & Technischer Support
90429 Nürnberg
Tel: (0911) 23 42 26 0
Fax: (0911) 23 42 26 928
http://www.hetzner.de


Registergericht Ansbach, HRB 3204
Vorstandsvorsitzender: Dipl. Ing. (FH) Martin Hetzner
Aufsichtsratsvorsitzender: Günther Müller