Re: 'cp -a' does not preserve nocow xattrs on btrfs

[A L]

On 2020-05-17 20:48, mail@lechevalier.se wrote:
> "Pádraig Brady" P@draigbrady.com – 17 maj 2020 kl. 13:22
> > On 16/05/2020 21:01, A L wrote:
> > > On 2020-05-16 13:31, A L wrote:
> > > > Hi,
> > > >
> > > > Normally, 'cp -a' should preserve xattrs when copying files and folders.
> > > >
> > > >  From the man page(*):
> > > > -a, --archive
> > > > same as -dR --preserve=all
> > > > --preserve[=ATTR_LIST]
> > > > preserve the specified attributes (default:
> > > > mode,ownership,timestamps), if possible additional
> > > > attributes: context, links, xattr, all
> > > >
> > > >
> > > > There are two important xattrs(*) on the Btrfs filesystem; 'no
> > > > copy-on-writes' ('+C') and 'compression' (+c). They have specific
> > > > conditions to work.
> > > >
> > > > The nocow '+C' attribute can only be set on empty files.
> > > > The compression '+c' can be set on any file, but only newly written
> > > > data will be compressed.
> > > >
> > > > The problem is that 'cp -a' seems to set the xattrs after creating the
> > > > file and writing the data to it, which means that the nocow '+C' flags
> > > > can't be set and that files would not be fully compressed, even though
> > > > the flag is set.
> > > >
> > > > I think that these xattrs should be preserved when doing 'cp -a'.
> > > >
> > > >
> > > >
> > > > * man7.org/linux/man-pages/man1/cp.1.html
> > > > * man7.org/linux/man-pages/man1/chattr.1.html
> > >
> > > I realise I had confused the difference between fattr (aka ext2
> > > attributes) and xattrs (extended attributes). In Btrfs, the nocow '+C'
> > > is only available as fattr, and not a xattr, which is why it is not
> > > copied with cp. Apart from that, the issue still stands with compression
> > > flag '+c', since that is also implemented as an xattr
> > > ("btrfs.compression") . cp ought to set the xattrs before writing any
> > > data to the file, possibly with an fsync in between.
> > >
> > > Side note: Why is it that cp -a does not copy file attrs and only
> > > xattrs? As a end user, the naming similarity fattr and xattr is somewhat
> > > confusing, and the man page does not clearly state that only the latter
> > > is supported in cp. man7.org/linux/man-pages/man1/cp.1.html
> >
> > Seems the same flags interface is used by ext4, btrfs, and xfs at least:
> > git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git/commit/misc/chattr.1.in
> > Given the generality, it's plausible we might consider preserving these flags.
> >
> > Copying all xattrs before writing the data may impact the ability to write data,
> > so we may need to split which xattrs are written when. Though if we
> > just considered the fattrs it might simplify (though we may have the same issue 
> > there).
> >
> > As for the particular NOCOW flag, is that something you'd always want
> > to set on new files? It seems like it might be more specific to
> > the file being written, than the file being read?
> > This would overlap with the existing cp --reflink flag.
> > Perhaps cp --reflink=never would preserve the NOCOW flag.
> > Notes:
> > one can have NOCOW set on a dir, which causes each file to be NOCOW.
> > one can't reflink a NOWCOW file
> >
> > thanks for the info,
> > Pádraig
>
> Perhaps it is best to limit my request to fattrs for the moment. I don't know 
> how well defined the xattrs are in general.
>
>
> Looking at the list of available fattrs, I think only the immutable attribute 
> can't be set first. There are some attributes that may never be set with 
> chattr. Not sure how to handle those.
>
>
> How should cp react when the target fs does not support one or more attributes?
>
>
> Regarding nocow. I think all fattrs should only be preserved with -a or 
> --preserve=fattr (or all).if you also specify reflink=always/auto then perhaps 
> the -a flag should have priority?
>
>
> Thanks

Hi again,

Why not make a rulebook for when attrs should be set during cp -a? IMHO 
the intent of 'cp -a' is to preserve all source attributes, if possible.

On Btrfs, nocow (+C) files can be reflinked too, but only to other nocow 
files. Thus, in order to enable 'cp -a --reflink=always' for nocow 
files, we need to set '+C' on the destination files before we add actual 
file data to them.

Here is an example where a nocow file is reflinked. It works around 'cp 
-a' by setting +C on the target dir first.

  # mkdir foo bar
  # touch foo/file
  # chattr +C foo/file
  # chattr +C bar
  # cat somefile >> foo/file
  # cp -a --reflink --sparse=auto foo/file bar/file
  (on my distro, cp complained about reflink if sparse=auto was not set)

  # compsize foo bar
  Processed 2 files, 1 regular extents (2 refs), 0 inline.
  Type       Perc     Disk Usage   Uncompressed Referenced
  TOTAL      100%       12K          12K          24K
  none       100%       12K          12K          24K



Can we generally say that all flags except ‘i’ immutable can and should 
be set before filedata is added to new files and that directories should 
have flags set _after_ all containing files have been added?

a       Order does not matter.
A       Order does not matter.
c	On empty file, before data is added. Otherwise file content won’t be 
compressed.
C       Must be set on empty file, before data is added or reflinked.
d       Order does not matter.
D       Only valid dirs. Does order matter?
i       Must be set after file has been written to.
S       Does order matter?
m	On empty file, before data is added. Otherwise file content could be 
compressed due to mount flags or inheritance from parent directory flags.

There are other flags not supported by Btrfs, which may have other 
requirements? https://www.man7.org/linux/man-pages/man1/chattr.1.html

####
# Currently supported Btrfs file attributes:
# https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs(5)
#

a       append only, new writes are always written at the end of the file

A       no atime updates

c	compress data, all data written after this attribute is set will be 
compressed. Please note that compression is also affected by the mount 
options or the parent directory attributes.
	When set on a directory, all newly created files will inherit this 
attribute.

C       no copy-on-write, file modifications are done in-place
	When set on a directory, all newly created files will inherit this 
attribute.
	Note: due to implementation limitations, this flag can be set/unset 
only on empty files.

d	no dump, makes sense with 3rd party tools like dump(8), on BTRFS the 
attribute can be set/unset but no other special handling is done

D	synchronous directory updates, for more details search open(2) for 
O_SYNC and O_DSYNC

i	immutable, no file data and metadata changes allowed even to the root 
user as long as this attribute is set (obviously the exception is 
unsetting the attribute)

S	synchronous updates, for more details search open(2) for O_SYNC and 
O_DSYNC

m	(previously X) no compression, permanently turn off compression on the 
given file. Any compression mount options will not affect this file.
	When set on a directory, all newly created files will inherit this 
attribute.