Re: [Gluster-devel] Bug with group permissions

[Raghavendra G]

Hi Geoff,
Can you please check with glusterfs--mainline--2.5--patch-584 that the bug
still exists?

regards,

On Nov 12, 2007 6:49 PM, Geoff Kassel <address@hidden> wrote:

> Hi Raghavendra,
>   Sorry about the delay - I've now been able to reproduce the error on my
> virtual server setup, where's there's no noise from production use.
>
> Here's the glusterfs.log entries from the actions in the original email:
>
> 2007-11-13 00:38:56 E [afr.c:5154:afr_closedir] shared-namespace-afr:
> afrfdp
> is NULL, returning EBADFD
> 2007-11-13 00:38:56 E [afr.c:5154:afr_closedir] shared-dataspace-afr:
> afrfdp
> is NULL, returning EBADFD
> 2007-11-13 00:39:04 E [afr.c:4412:afr_create_cbk] shared-namespace-afr:
> (path=/test/testfile child=shared-A-namespace-iot) op_ret=-1 op_errno=13
> 2007-11-13 00:39:04 E [afr.c:4412:afr_create_cbk] shared-namespace-afr:
> (path=/test/testfile child=shared-B-namespace-iot) op_ret=-1 op_errno=13
>
> This is with the second-to-latest TLA (552) and fuse-2.7.0-glfs6.
>
> Please advise if there's any tests or particular client-server
> configurations
> you'd like me to try.
>
> Kind regards,
>
> Geoff Kassel.
>
> On Tue, 6 Nov 2007, you wrote:
> > oh.. sorry for that.. I didnt observe that.
> >
> > On Nov 6, 2007 10:04 AM, Geoff Kassel <address@hidden>
> wrote:
> > > Please don't forward the message with my work address onto the devel
> list
> > > -
> > > this was accidental. I'd prefer to keep my work address off the list.
> > >
> > > On Tue, 6 Nov 2007, Geoff Kassel wrote:
> > > > To rule out an interaction with libnss-mysql, I have also tested
> this
> > >
> > > with
> > >
> > > > a passwd/groups setup on both machines, by the way.
> > > >
> > > > On Tue, 6 Nov 2007, Geoff Kassel wrote:
> > > > > Both alpha and bravo have the same group set up in this case -
> it's
> > > > > managed by libnss-mysql, on a MySQL Cluster server.
> > > > >
> > > > > On Tue, 6 Nov 2007, you wrote:
> > > > > > Another question,
> > > > > > Does both alpha and bravo have the user group you are
> experimenting
> > > > > > with?
> > > > > >
> > > > > > regards,
> > > > > >
> > > > > > On Nov 6, 2007 9:34 AM, Geoff Kassel
> > > > > > <address@hidden>
> > > >
> > > > wrote:
> > > > > > > Hi Raghavendra,
> > > > > > >   I'll see if I can do a log run for this when my system is
> > >
> > > quieter
> > >
> > > > > > > i.e. sometime later tonight, Australian Eastern Standard Time.
> I
> > > > > > > don't think I'm
> > > > > > > going to to be able to produce any clearer logs than that
> already
> > > > > > > provided,
> > > > > > > though, since this is a multi-user production system.
> > > > > > >
> > > > > > >   In the meanwhile, I have a virtual server image of this
> system
> > > > > > > pre-deployment that I'm planning on testing to see if the bug
> > >
> > > occurs
> > >
> > > > > > > with the
> > > > > > > 545 patch. If it does, I can provide clearer logs for you now.
> If
> > >
> > > I
> > >
> > > > > > > can't get
> > > > > > > it to recur on this system, that'll narrow down quite
> > >
> > > substantially
> > >
> > > > > > > the circumstances under which this bug occurs.
> > > > > > >
> > > > > > >   I'll let you know how I go.
> > > > > > >
> > > > > > > Cheers,
> > > > > > >
> > > > > > > Geoff Kassel.
> > > > > > >
> > > > > > > On Mon, 5 Nov 2007, you wrote:
> > > > > > > > Hi Geoff,
> > > > > > > >
> > > > > > > > The glusterfs.log you sent me indicates that you've been
> doing
> > >
> > > lots
> > >
> > > > > > > > of operations. Is it possible for you to reproduce the bug
> just
> > > > > > > > using the
> > > > > > >
> > > > > > > set
> > > > > > >
> > > > > > > > of commands you had given in your first mail? i.e,
> > > > > > > >
> > > > > > > > groupmems -a nonrootuser -g randomgroup
> > > > > > > > cd /glusterfsmountpoint
> > > > > > > > mkdir test
> > > > > > > > chown root:randomgroup test
> > > > > > > > chmod 770 test
> > > > > > > >  su - nonrootuser
> > > > > > > > cd /glusterfsmountpoint/test # Succeeds
> > > > > > > > ls # Permission denied error.
> > > > > > > > echo "Hello" > testfile      # Also gives a perm
> > > > > > > >
> > > > > > > > and send the logs generated?
> > > > > > > >
> > > > > > > > On Nov 5, 2007 12:57 PM, Geoff Kassel
> > > > > > > > <address@hidden>
> > > > > > >
> > > > > > > wrote:
> > > > > > > > > Hi Raghavendra,
> > > > > > > > >
> > > > > > > > > > I found two client spec files in both alpha/ and bravo/.
> by
> > >
> > > any
> > >
> > > > > > > chance
> > > > > > >
> > > > > > > > > are
> > > > > > > > >
> > > > > > > > > > you running two clients?
> > > > > > > > >
> > > > > > > > > Yes, that is correct. I'm using GlusterFS to maintain
> shared
> > > > > > > > > storage between
> > > > > > > > > two machines. Both machines run a GlusterFS server,
> providing
> > >
> > > a
> > >
> > > > > > > namespace
> > > > > > >
> > > > > > > > > and
> > > > > > > > > a dataspace, which are subsequently unified and AFR'd in
> the
> > > > > > > > > client
> > > > > > >
> > > > > > > also
> > > > > > >
> > > > > > > > > run
> > > > > > > > > on each machine.
> > > > > > > > >
> > > > > > > > > > #uname -a
> > > > > > > > >
> > > > > > > > > Linux alpha 2.6.20-hardened-r10 #1 SMP Sat Oct 20 05:41:43
> > > > > > > > > EST 2007
> > > > > > >
> > > > > > > i686
> > > > > > >
> > > > > > > > > Dual-Core AMD Opteron(tm) Processor 2218 AuthenticAMD
> > >
> > > GNU/Linux
> > >
> > > > > > > > > and
> > > > > > > > >
> > > > > > > > > Linux bravo 2.6.20-hardened-r10 #1 SMP Sat Oct 20 05:41:43
> > > > > > > > > EST 2007
> > > > > > >
> > > > > > > i686
> > > > > > >
> > > > > > > > > Dual-Core AMD Opteron(tm) Processor 2218 AuthenticAMD
> > >
> > > GNU/Linux
> > >
> > > > > > > > > The two machines are identical in hardware, and the
> software
> > > > > > > > > configuration is
> > > > > > > > > more or less identical, being different only in host name
> and
> > > > > > >
> > > > > > > symmetrical
> > > > > > >
> > > > > > > > > changes for load balancing and hosting purposes. GlusterFS
> > > > > > >
> > > > > > > communicates
> > > > > > >
> > > > > > > > > over
> > > > > > > > > a GigE link between the pair, utilized only by GlusterFS
> and
> > > > > > >
> > > > > > > keepalived
> > > > > > >
> > > > > > > > > for
> > > > > > > > > load-balancing and maintenance of redundant services.
> > > > > > > > >
> > > > > > > > > While the processors are 64-bit, I'm running 32-bit for
> > >
> > > software
> > >
> > > > > > > > > compatibility
> > > > > > > > > reasons - that's one difference between your kernel and
> mine.
> > >
> > > I
> > >
> > > > > > > > > also
> > > > > > >
> > > > > > > use
> > > > > > >
> > > > > > > > > PaX,
> > > > > > > > > which affects memory allocation and access, and have the
> > > > > > > > > grsec
> > > > > > >
> > > > > > > extensions
> > > > > > >
> > > > > > > > > compiled in (as part of the Gentoo hardened-sources) but
> not
> > > > > > > > > enabled.
> > > > > > > > >
> > > > > > > > > Attached is my kernel config. I hope this helps.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > >
> > > > > > > > > Geoff Kassel.
> > > > > > > > >
> > > > > > > > > On Mon, 5 Nov 2007, you wrote:
> > > > > > > > > > Hi Geoff,
> > > > > > > > > >
> > > > > > > > > > I found two client spec files in both alpha/ and bravo/.
> by
> > >
> > > any
> > >
> > > > > > > chance
> > > > > > >
> > > > > > > > > are
> > > > > > > > >
> > > > > > > > > > you running two clients?
> > > > > > > > > >
> > > > > > > > > > I tried with single glusterfs client running on a node
> with
> > >
> > > the
> > >
> > > > > > > > > following
> > > > > > > > >
> > > > > > > > > > configuration
> > > > > > > > > >
> > > > > > > > > > #uname -a
> > > > > > > > > > Linux master-node 2.6.18-8.el5 #1 SMP Thu Mar 15
> 19:46:53
> > >
> > > EDT
> > >
> > > > > > > > > > 2007
> > > > > > > > >
> > > > > > > > > x86_64
> > > > > > > > >
> > > > > > > > > > x86_64 x86_64 GNU/Linux
> > > > > > > > > >
> > > > > > > > > > but still I cannot reproduce your problem. I am using
> > > > > > >
> > > > > > > fuse-2.7.0-glfs5.
> > > > > > >
> > > > > > > > > So
> > > > > > > > >
> > > > > > > > > > the only difference in configuration of the node is the
> > >
> > > kernel.
> > >
> > > > > > > > > > Going through logs If I can find any hints.
> > > > > > > > > > regards,
> > > > > > > > > >
> > > > > > > > > > On Nov 5, 2007 10:31 AM, Geoff Kassel
> > > > > > > > > > <address@hidden
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > > > Hi Raghavendra,
> > > > > > > > > > >
> > > > > > > > > > > I'm not replying to the list because of the size of
> the
> > > > > > > > > > > attached
> > > > > > >
> > > > > > > log
> > > > > > >
> > > > > > > > > > > files.
> > > > > > > > > > >
> > > > > > > > > > > > I tried to reproduce your problem with
> > > > > > > > > > >
> > > > > > > > > > > glusterfs--mainline--2.5--patch-545,
> > > > > > > > > > >
> > > > > > > > > > > > but without success.
> > > > > > > > > > >
> > > > > > > > > > > That's not too surprising - I use a 2.6.20 hardened
> > > > > > > > > > > Gentoo kernel with PaX enabled, so I'm likely to get
> > > > > > > > > > > different errors to
> > > > > > >
> > > > > > > everyone
> > > > > > >
> > > > > > > > > > > else. I found the
> > > > > > > > > > > error occurred with (at least) patches 543 and
> onwards.
> > > > > > > > > > >
> > > > > > > > > > > > Can you send your client/server logs and
> configuration
> > > > > > > > > > > > files? Also what is the fuse version you are using?
> > > > > > > > > > >
> > > > > > > > > > > Sure - they're in the attached ZIP file. I'm using
> FUSE
> > > > > > >
> > > > > > > 2.7.0-glfs5
> > > > > > >
> > > > > > > > > > > on both
> > > > > > > > > > > machines, known as alpha and bravo. I've sent logs and
> > >
> > > spec
> > >
> > > > > > > > > > > files
> > > > > > >
> > > > > > > for
> > > > > > >
> > > > > > > > > > > both machines, in the accordingly named directories.
> > > > > > > > > > >
> > > > > > > > > > > To find the time of my various tests, search for ':
> > > > > > > > > > > /test'
> > >
> > > in
> > >
> > > > > > > > > > > the glusterfs
> > > > > > > > > > > logs. The most relevant test (i.e. the results of
> which I
> > > > > > > > > > > posted
> > > > > > > > >
> > > > > > > > > about)
> > > > > > > > >
> > > > > > > > > > > is ': /test4'.
> > > > > > > > > > >
> > > > > > > > > > > Good luck with the bug hunt!
> > > > > > > > > > >
> > > > > > > > > > > Cheers,
> > > > > > > > > > >
> > > > > > > > > > > Geoff Kassel.
> > > > > > > > > > >
> > > > > > > > > > > On Mon, 5 Nov 2007, Raghavendra G wrote:
> > > > > > > > > > > > Hi Geoff,
> > > > > > > > > > > > I tried to reproduce your problem with
> > > > > > > > > > >
> > > > > > > > > > > glusterfs--mainline--2.5--patch-545,
> > > > > > > > > > >
> > > > > > > > > > > > but without success. Can you send your client/server
> > >
> > > logs
> > >
> > > > > > > > > > > > and
> > > > > > > > > > >
> > > > > > > > > > > configuration
> > > > > > > > > > >
> > > > > > > > > > > > files? Also what is the fuse version you are using?
> > > > > > > > > > > >
> > > > > > > > > > > > regards,
> > > > > > > > > > > >
> > > > > > > > > > > > On Nov 4, 2007 4:39 PM, Geoff Kassel
> > > > > > > > > > > > <address@hidden>
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > > > Hi all,
> > > > > > > > > > > > >   I've think I've found a group permissions bug in
> > > > > > > > > > > > > the latest
> > > > > > > > >
> > > > > > > > > patches
> > > > > > > > >
> > > > > > > > > > > > > committed to the repository. The bug causes
> > > > > > > > > > > > > permission denied
> > > > > > > > >
> > > > > > > > > errors
> > > > > > > > >
> > > > > > > > > > > for
> > > > > > > > > > >
> > > > > > > > > > > > > non-root users where they should have adequate
> > > > > > > > > > > > > access.
> > > > > > > > > > > > >
> > > > > > > > > > > > > The following set of commands under a glusterfs
> mount
> > > > > > >
> > > > > > > reproduces
> > > > > > >
> > > > > > > > > the
> > > > > > > > >
> > > > > > > > > > > bug
> > > > > > > > > > >
> > > > > > > > > > > > > for
> > > > > > > > > > > > > me:
> > > > > > > > > > > > >
> > > > > > > > > > > > >   groupmems -a nonrootuser -g randomgroup
> > > > > > > > > > > > >   cd /glusterfsmountpoint
> > > > > > > > > > > > >   mkdir test
> > > > > > > > > > > > >   chown root:randomgroup test
> > > > > > > > > > > > >   chmod 770 test
> > > > > > > > > > > > >   su - nonrootuser
> > > > > > > > > > > > >   cd /glusterfsmountpoint/test # Succeeds
> > > > > > > > > > > > >   ls # Permission denied error.
> > > > > > > > > > > > >   echo "Hello" > testfile      # Also gives a
> > >
> > > permission
> > >
> > > > > > > denied
> > > > > > >
> > > > > > > > > > > > > error.
> > > > > > > > > > > > >
> > > > > > > > > > > > > To get a successful ls under the non-root user,
> > > > > > > > > > > > > permissions
> > > > > > >
> > > > > > > need
> > > > > > >
> > > > > > > > > to
> > > > > > > > >
> > > > > > > > > > > > > be 775 in
> > > > > > > > > > > > > the test directory. To get a successful file
> > > > > > > > > > > > > creation,
> > >
> > > it
> > >
> > > > > > > needs
> > > > > > >
> > > > > > > > > > > > > to
> > > > > > > > >
> > > > > > > > > be
> > > > > > > > >
> > > > > > > > > > > > > 777.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I have noticed that on the initial cd as the
> non-root
> > > > > > > > > > > > > user,
> > > > > > >
> > > > > > > there
> > > > > > >
> > > > > > > > > > > seems
> > > > > > > > > > >
> > > > > > > > > > > > > to be
> > > > > > > > > > > > > the following in the client logs (I'm using
> > >
> > > AFR/Unify):
> > > > > > > > > > > > > 2007-11-04 22:36:25 E [afr.c:5654:afr_closedir]
> > > > > > > > >
> > > > > > > > > shared-namespace-afr:
> > > > > > > > > > > > > afrfdp
> > > > > > > > > > > > > is NULL, returning EBADFD
> > > > > > > > > > > > > 2007-11-04 22:36:25 E [afr.c:5654:afr_closedir]
> > > > > > > > >
> > > > > > > > > shared-dataspace-afr:
> > > > > > > > > > > > > afrfdp
> > > > > > > > > > > > > is NULL, returning EBADFD
> > > > > > > > > > > > > 2007-11-04 22:36:25 E [fuse-bridge.c
> :654:fuse_fd_cbk]
> > > > > > > > >
> > > > > > > > > glusterfs-fuse:
> > > > > > > > > > > > > 1322194: /test => -1 (13)
> > > > > > > > > > > > >
> > > > > > > > > > > > > This bug doesn't cause the server or client to
> crash.
> > >
> > > (I
> > >
> > > > > > > > > > > > > have
> > > > > > > > >
> > > > > > > > > noticed
> > > > > > > > >
> > > > > > > > > > > a
> > > > > > > > > > >
> > > > > > > > > > > > > lot
> > > > > > > > > > > > > of 'E [unify.c:145:unify_buf_cbk] shared:
> > > > > > > > > > > > > shared-namespace-afr
> > > > > > > > > > >
> > > > > > > > > > > returned
> > > > > > > > > > >
> > > > > > > > > > > > > 107'
> > > > > > > > > > > > > and random server crashes in general lately, but
> this
> > > > > > > > > > > > > appears
> > > > > > > > > > >
> > > > > > > > > > > unrelated.)
> > > > > > > > > > >
> > > > > > > > > > > > > I can provide logs as well as client and server
> spec
> > > > > > > > > > > > > files on request.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Kind regards,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Geoff Kassel.
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > _______________________________________________
> > > > > > > > > > > > > Gluster-devel mailing list
> > > > > > > > > > > > > address@hidden
> > > > > > > > > > > > >
> http://lists.nongnu.org/mailman/listinfo/gluster-deve
> > > > > > > > > > > > >l
>
>


-- 
Raghavendra G

A centipede was happy quite, until a toad in fun,
Said, "Prey, which leg comes after which?",
This raised his doubts to such a pitch,
He fell flat into the ditch,
Not knowing how to run.
-Anonymous