Re: rm && opensolaris && ntfs-3g problem

[Andras Barna]

szaka,
when I execute rm (6.12) -r directory:

unique: 202, opcode: ACCESS (34), nodeid: 1, insize: 48
ACCESS / 011
   unique: 202, error: -89 (Operation not applicable), outsize: 16
unique: 203, opcode: LOOKUP (1), nodeid: 1, insize: 42
LOOKUP /o
   NODEID: 9
   unique: 203, error: 0 (Error 0), outsize: 136
unique: 204, opcode: GETATTR (3), nodeid: 9, insize: 40
   unique: 204, error: 0 (Error 0), outsize: 112
unique: 205, opcode: ACCESS (34), nodeid: 1, insize: 48
ACCESS / 011
   unique: 205, error: -89 (Operation not applicable), outsize: 16
unique: 206, opcode: LOOKUP (1), nodeid: 1, insize: 42
LOOKUP /o
   NODEID: 9
   unique: 206, error: 0 (Error 0), outsize: 136
unique: 207, opcode: ACCESS (34), nodeid: 9, insize: 48
ACCESS /o 012
   unique: 207, error: -89 (Operation not applicable), outsize: 16
unique: 208, opcode: ACCESS (34), nodeid: 1, insize: 48
ACCESS / 011
   unique: 208, error: -89 (Operation not applicable), outsize: 16
unique: 209, opcode: LOOKUP (1), nodeid: 1, insize: 42
LOOKUP /o
   NODEID: 9
   unique: 209, error: 0 (Error 0), outsize: 136
unique: 210, opcode: UNLINK (10), nodeid: 1, insize: 42
UNLINK /o
   unique: 210, error: -93 (Directory not empty), outsize: 16
unique: 211, opcode: ACCESS (34), nodeid: 1, insize: 48
ACCESS / 011
   unique: 211, error: -89 (Operation not applicable), outsize: 16
unique: 212, opcode: GETATTR (3), nodeid: 1, insize: 40
   unique: 212, error: 0 (Error 0), outsize: 112


On Thu, Aug 14, 2008 at 4:04 PM, Szabolcs Szakacsits <address@hidden> wrote:
>
> Hi Solaris FUSE Developers,
>
> We're back with the GNU rm -rf failing with ntfs-3g on Solaris only.
> GNU rm on Linux is ok with ntfs-3g. Solaris rm is also ok with ntfs-3g
> but it works differently.
>
> To reproduce the problem:
>
>  1. compile the latest coreutils, version 6.12
>     http://ftp.gnu.org/gnu/coreutils/coreutils-6.12.tar.gz
>  2. mkdir -p l/l/l/l/l/l
>  3. path/to/coreutils-6.12/src/rm -rf l
>
> Andras reports step 3 succeeds but 'l' is not removed.
> Truss traces are available in the bottom of the page at
> http://article.gmane.org/gmane.comp.gnu.coreutils.bugs/14256
>
> According to Jim, coreutils maintainer, the problem is that the below
> syscall shouldn't succeed if 'l' still contains non-removed files:
>
>   unlinkat(AT_FDCWD, "l", 0x00000000)             = 0
>
> Apparently this seems to be a Solaris FUSE problem unless GNU rm
> doesn't use the right arguments but something else it believes.
> There are some more info below.
>
> One can monitor what commands ntfs-3g is receving and returning by using
> the -o debug mount option, e.g. if ntfs-3g receives the UNLINK command at
> all, and if so, then what it returns.
>
> Regards,
>            Szaka
>
> --
> NTFS-3G:  http://ntfs-3g.org
>
> On Thu, 14 Aug 2008, Jim Meyering wrote:
>
>> Szabolcs Szakacsits <address@hidden> wrote:
>> > Jim Meyering <jim <at> meyering.net> writes:
>> >> "Andras Barna" <andras.barna <at> gmail.com> wrote:
>> >> > On Tue, Aug 12, 2008 at 6:56 PM, Jim Meyering <jim <at> meyering.net> 
>> >> > wrote:
>> >> ...
>> >> >> That suggests that the opensolaris ntfs support for unlinkat
>> >> >> doesn't work as documented.  That unlinkat call is succeeding,
>> >> >> yet I presume there is a non-empty directory named "l" that it
>> >> >> fails to remove.
>> >> >>
>> >> >> There are two differences in how unlinkat is used between
>> >> >> coreutils and /usr/bin/rm:
>> >> >>  - coreutils uses "0" as the third argument, and /bin/rm uses 0x1
>> >> >>    (which is probably AT_REMOVEDIR)
>> >> >>  - coreutils uses AT_FDCWD as the first argument, and /bin/rm
>> >> >>    uses a file descriptor.
>> >> >>
>> >> >> Since Solaris is where openat-style functions originated, I'm
>> >> >> surprised that their ntfs implementation would not adhere to the
>> >> >> documented specification.
>> >> >
>> >> > what you mean under "their ntfs implementation"?
>> >> > i thought we talk about ntfs-3g
>> >> > hint: http://ntfs-3g.org/
>> >>
>> >> Sorry for the imprecision.
>> >> Obviously, I meant "the ntfs-3g" driver code, since
>> >> this seems to be ntfs-3g specific.  It'd be good to know
>> >> if it is also specific to Solaris, and what precise version(s)
>> >> of ntfs-3g are affected, if only to document the problem
>> >> for people who encounter this in the future.
>> >>
>> >> For the record, can you tell us what versions you know to be affected?
>> >
>> > I asked Andras to report the bug here because ntfs-3g works with all
>> > gnu rm on Linux, and Solaris rm on Solaris.
>> >
>> > The only problem is gnu rm on Solaris. Version 6.7 gave ENOTEMPTY
>> > and the latest 6.12 reports no error when the directory is not
>> > removed.
>>
>> With 6.12, the error is here:
>> (from Andras' truss output)
>>
>>     unlinkat(AT_FDCWD, "l", 0x00000000)             = 0
>>
>> i.e., unlinkat succeeds (returns 0), yet fails to remove the directory.
>> That is not a bug in coreutils.
>>
>> FWIW, that is the very first and only unlinkat call.
>> Once it succeeds in unlinking the command line argument,
>> GNU rm (6.12) must presume that its job is done.
>>
>> > The ntfs-3g driver can return ENOTEMPTY if a removed file/directory
>> > has an open file descriptor and its parent directory tried to be
>> > removed (this is a FUSE high-level API issue).
>> >
>> > A potential explanation could be for the ENOEMPTY problem if gnu rm
>> > "leaks" file descriptors during recursive removal.
>>
>> GNU rm does not leak anything, afaik.
>>
>> >> >> I do not plan to make GNU rm work around this bug.
>> >> >
>> >> > sorry for reporting it
>> >>
>> >> What I should have said is that working around this
>> >> system-and-file-system-specific bug in coreutils/gnulib would not be easy,
>> >> and would probably have a negative impact all other systems.  However,
>> >> if someone can come up with a patch that is low-impact and safe looking,
>> >> I'll be happy to look at it.
>> >
>> > Personally I don't see where the problem is. Does gnu rm misinterpret
>> > something Solaris specific, or is it a problem with the Solaris FUSE
>> > kernel module?
>>
>> unlinkat is the problem.
>



-- 
Andy
http://blog.sartek.net