bug#9500: [RFC/PATCH] cp: Add option to pre-allocate space for files

[Mark]

On Fri, May 11, 2012 20:19, Pádraig Brady wrote:
> On 05/11/2012 06:40 PM, Mark wrote:
>> ...
>> An interesting aside: I tried using cp to pre-allocate space for a very
>> large file on an ext4 partition, much larger than the amount of free
>> space. IMHO it would be best for the filesystem to fail immediately in
>> that case. ext4 does a lot of work (there was a lot of disk activity and
>> it took a long time to fail). ext4 pre-allocates as much of the
>> requested
>> region as possible, rather than succeeding or failing all-or-nothing. So
>> you get a disk-full condition. (Of course that's no worse than what
>> happens when you run cp normally. But it would happen much more quickly
>> with pre-allocation.)
>
> Well that's bad as you get a delay in addition to the normal copy.
> However, I don't see that behavior with 2.6.40.4-5.fc15.x86_64 at least?

I'm using kernel 3.0.0-19-generic #32-Ubuntu here. But probably more
relevant, the partition I tested on was ~1.7TB ext4 on an external USB 2.0
drive which was almost full and probably *very* fragmented, i.e. free
space spread all over the disk in thousands of small chunks. ext4 seems to
be pretty slow at allocating space in that case.

If I were designing a filesystem, I'd have it immediately return failure
if fallocate() is specified with additional size larger than the amount of
free space. Though for the filesystem, determining how much extra space a
fallocate() call would need can be quite involved in some cases and
require a significant amount of disk access...

Imagine a huge sparse file with many thousands of holes, and the requested
region for fallocate() serving to "fill in" many of the holes. But any
non-hole parts within the fallocate() region would reduce the amount of
additional space required for fallocate() to succeed. So it's not as
simple as comparing length of fallocate() region with amount of free
space...

Unless you're creating a new file, which is what cp does most of the time.
So maybe a workaround could be added to cp. If --preallocate is specified,
cp could check the amount of free space before writing to the destination
file and abort without even needing to call fallocate() if there isn't
enough. (In fact, cp could do that anyway in most cases I think?)


-- Mark