Re: rm && opensolaris && ntfs-3g problem

[Szabolcs Szakacsits]

On Thu, 14 Aug 2008, Andras Barna wrote:

> when I execute rm (6.12) -r directory:
> 
> unique: 202, opcode: ACCESS (34), nodeid: 1, insize: 48
> ACCESS / 011
>    unique: 202, error: -89 (Operation not applicable), outsize: 16
> unique: 203, opcode: LOOKUP (1), nodeid: 1, insize: 42
> LOOKUP /o
>    NODEID: 9
>    unique: 203, error: 0 (Error 0), outsize: 136
> unique: 204, opcode: GETATTR (3), nodeid: 9, insize: 40
>    unique: 204, error: 0 (Error 0), outsize: 112
> unique: 205, opcode: ACCESS (34), nodeid: 1, insize: 48
> ACCESS / 011
>    unique: 205, error: -89 (Operation not applicable), outsize: 16
> unique: 206, opcode: LOOKUP (1), nodeid: 1, insize: 42
> LOOKUP /o
>    NODEID: 9
>    unique: 206, error: 0 (Error 0), outsize: 136
> unique: 207, opcode: ACCESS (34), nodeid: 9, insize: 48
> ACCESS /o 012
>    unique: 207, error: -89 (Operation not applicable), outsize: 16
> unique: 208, opcode: ACCESS (34), nodeid: 1, insize: 48
> ACCESS / 011
>    unique: 208, error: -89 (Operation not applicable), outsize: 16
> unique: 209, opcode: LOOKUP (1), nodeid: 1, insize: 42
> LOOKUP /o
>    NODEID: 9
>    unique: 209, error: 0 (Error 0), outsize: 136
> unique: 210, opcode: UNLINK (10), nodeid: 1, insize: 42
> UNLINK /o
>    unique: 210, error: -93 (Directory not empty), outsize: 16

ntfs-3g correctly returns ENOTEMPTY but this information gets lost and 
arrives as success to gnu rm. 

This indeed looks to be a problem in Solaris.

Thanks,
           Szaka

> unique: 211, opcode: ACCESS (34), nodeid: 1, insize: 48
> ACCESS / 011
>    unique: 211, error: -89 (Operation not applicable), outsize: 16
> unique: 212, opcode: GETATTR (3), nodeid: 1, insize: 40
>    unique: 212, error: 0 (Error 0), outsize: 112
> 
> 
> On Thu, Aug 14, 2008 at 4:04 PM, Szabolcs Szakacsits <address@hidden> wrote:
> >
> > Hi Solaris FUSE Developers,
> >
> > We're back with the GNU rm -rf failing with ntfs-3g on Solaris only.
> > GNU rm on Linux is ok with ntfs-3g. Solaris rm is also ok with ntfs-3g
> > but it works differently.
> >
> > To reproduce the problem:
> >
> >  1. compile the latest coreutils, version 6.12
> >     http://ftp.gnu.org/gnu/coreutils/coreutils-6.12.tar.gz
> >  2. mkdir -p l/l/l/l/l/l
> >  3. path/to/coreutils-6.12/src/rm -rf l
> >
> > Andras reports step 3 succeeds but 'l' is not removed.
> > Truss traces are available in the bottom of the page at
> > http://article.gmane.org/gmane.comp.gnu.coreutils.bugs/14256
> >
> > According to Jim, coreutils maintainer, the problem is that the below
> > syscall shouldn't succeed if 'l' still contains non-removed files:
> >
> >   unlinkat(AT_FDCWD, "l", 0x00000000)             = 0
> >
> > Apparently this seems to be a Solaris FUSE problem unless GNU rm
> > doesn't use the right arguments but something else it believes.
> > There are some more info below.
> >
> > One can monitor what commands ntfs-3g is receving and returning by using
> > the -o debug mount option, e.g. if ntfs-3g receives the UNLINK command at
> > all, and if so, then what it returns.
> >
> > Regards,
> >            Szaka
> >
> > --
> > NTFS-3G:  http://ntfs-3g.org
> >
> > On Thu, 14 Aug 2008, Jim Meyering wrote:
> >
> >> Szabolcs Szakacsits <address@hidden> wrote:
> >> > Jim Meyering <jim <at> meyering.net> writes:
> >> >> "Andras Barna" <andras.barna <at> gmail.com> wrote:
> >> >> > On Tue, Aug 12, 2008 at 6:56 PM, Jim Meyering <jim <at> meyering.net> 
> >> >> > wrote:
> >> >> ...
> >> >> >> That suggests that the opensolaris ntfs support for unlinkat
> >> >> >> doesn't work as documented.  That unlinkat call is succeeding,
> >> >> >> yet I presume there is a non-empty directory named "l" that it
> >> >> >> fails to remove.
> >> >> >>
> >> >> >> There are two differences in how unlinkat is used between
> >> >> >> coreutils and /usr/bin/rm:
> >> >> >>  - coreutils uses "0" as the third argument, and /bin/rm uses 0x1
> >> >> >>    (which is probably AT_REMOVEDIR)
> >> >> >>  - coreutils uses AT_FDCWD as the first argument, and /bin/rm
> >> >> >>    uses a file descriptor.
> >> >> >>
> >> >> >> Since Solaris is where openat-style functions originated, I'm
> >> >> >> surprised that their ntfs implementation would not adhere to the
> >> >> >> documented specification.
> >> >> >
> >> >> > what you mean under "their ntfs implementation"?
> >> >> > i thought we talk about ntfs-3g
> >> >> > hint: http://ntfs-3g.org/
> >> >>
> >> >> Sorry for the imprecision.
> >> >> Obviously, I meant "the ntfs-3g" driver code, since
> >> >> this seems to be ntfs-3g specific.  It'd be good to know
> >> >> if it is also specific to Solaris, and what precise version(s)
> >> >> of ntfs-3g are affected, if only to document the problem
> >> >> for people who encounter this in the future.
> >> >>
> >> >> For the record, can you tell us what versions you know to be affected?
> >> >
> >> > I asked Andras to report the bug here because ntfs-3g works with all
> >> > gnu rm on Linux, and Solaris rm on Solaris.
> >> >
> >> > The only problem is gnu rm on Solaris. Version 6.7 gave ENOTEMPTY
> >> > and the latest 6.12 reports no error when the directory is not
> >> > removed.
> >>
> >> With 6.12, the error is here:
> >> (from Andras' truss output)
> >>
> >>     unlinkat(AT_FDCWD, "l", 0x00000000)             = 0
> >>
> >> i.e., unlinkat succeeds (returns 0), yet fails to remove the directory.
> >> That is not a bug in coreutils.
> >>
> >> FWIW, that is the very first and only unlinkat call.
> >> Once it succeeds in unlinking the command line argument,
> >> GNU rm (6.12) must presume that its job is done.
> >>
> >> > The ntfs-3g driver can return ENOTEMPTY if a removed file/directory
> >> > has an open file descriptor and its parent directory tried to be
> >> > removed (this is a FUSE high-level API issue).
> >> >
> >> > A potential explanation could be for the ENOEMPTY problem if gnu rm
> >> > "leaks" file descriptors during recursive removal.
> >>
> >> GNU rm does not leak anything, afaik.
> >>
> >> >> >> I do not plan to make GNU rm work around this bug.
> >> >> >
> >> >> > sorry for reporting it
> >> >>
> >> >> What I should have said is that working around this
> >> >> system-and-file-system-specific bug in coreutils/gnulib would not be 
> >> >> easy,
> >> >> and would probably have a negative impact all other systems.  However,
> >> >> if someone can come up with a patch that is low-impact and safe looking,
> >> >> I'll be happy to look at it.
> >> >
> >> > Personally I don't see where the problem is. Does gnu rm misinterpret
> >> > something Solaris specific, or is it a problem with the Solaris FUSE
> >> > kernel module?
> >>
> >> unlinkat is the problem.
> >
> 
> 
> 
> -- 
> Andy
> http://blog.sartek.net
> 

--
NTFS-3G:  http://ntfs-3g.org