Re: [PATCH] sort: use posix_fadvise to announce access patterns on files opened for reading

[Joey Degges]

2010/3/1 Joey Degges <address@hidden>

> Thanks for testing, these results are interesting. I will run some tests on
> various systems tonight and report back. Maybe my earlier results were
> caused by peculiar hardware.
>
>
Results differ once again, this time I've tested on a different system.

Here is pastebin of the code I used to spin. Its some math in a large loop,
should run for ~5 minutes (enough time to do most tests):
http://codepad.org/tCtjLw7H

CPU: Intel(R) Core(TM)2 Quad CPU    Q6600  @ 2.40GHz
RAM: 2G
kernel: 2.6.31-19-generic #56-Ubuntu SMP x86_64
input file: 97M, gensort -a 1000000 file
cmd: umount /mnt && mount /dev/foo /mnt && /usr/bin/time -v sort < file >
/dev/null

mechanical disk @ 80MB/s, ext3 - 4x spinning processes
mode      :   real, %CPU, voluntary/involuntary context switches
NORMAL    : 14.73s,  10%, 408/34
SEQUENTIAL: 10.95s,  13%, 220/52
WILLNEED  : 05.22s,  35%, 026/57

flash key @ 20MB/s, ext4 - 4x spinning processes
NORMAL    : 88.49s,  01%, 1521/42
SEQUENTIAL: 69.76s,  02%, 1131/57
WILLNEED  : 50.29s,  03%, 0174/31

flash key @ 11MB/s, FAT32 - 4x spinning processes
NORMAL    : 106.5s,  01%, 1334/32
SEQUENTIAL: 85.63s,  01%, 0765/53
WILLNEED  : 68.70s,  02%, 0193/59

mechanical disk @ 80MB/s, ext3
NORMAL    : 02.70s,  64%, 748/9
SEQUENTIAL: 02.71s,  61%, 374/8
WILLNEED  : 02.79s,  57%, 026/5

flash key @ 20MB/s, ext4
NORMAL    : 06.25s,  25%, 1505/8
SEQUENTIAL: 06.18s,  26%, 1136/11
WILLNEED  : 06.10s,  26%, 0156/8

flash key @ 11MB/s, FAT32
NORMAL    : 10.23s,  15%, 1320/7
SEQUENTIAL: 10.12s,  16%, 0762/7
WILLNEED  : 10.14s,  16%, 0193/4


Note the large difference in the number of voluntary context switches
between the different modes. This number appears to only be dependent on the
device being read from and the advice mode, not the system load. When the
system is heavily loaded the process that context switches the least will be
able to consume more resources. From these results you can see that
SEQUENTIAL reduces the voluntary context switching from 25-46% and WILLNEED
reduces the voluntary context switching from 85-96%. This also explains why
you won't notice the improvements due to SEQUENTIAL or WILLNEED very much in
unburdened systems.

Thanks,
Joey