[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
fseek is inadequately slow (4 times slower then fread)
From: |
Khamenia, Valery |
Subject: |
fseek is inadequately slow (4 times slower then fread) |
Date: |
Tue, 23 Sep 2003 11:00:51 +0200 |
Hi all and especially drepper, aj and rolland.
the random access in non-cacheable files is very slow.
I've used the test below for benchmarking this issue
and find an interesting fact.
My experiment was like following.
First run this test like that:
% time -v thistest non_cached_big_file1 16 4000 16
(e.g. the wall clock time I got is 1 min 15 sec for reading
1Gb file)
Let's comment out fseek now and run with the same size
noncached file (or just clear the cache if you are
confident with the Linux internals and use the same file).
% time -v thistest non_cached_big_file2 16 4000 16
(and I got 30 sec for reading 1Gb file. BTW, that is about
33Mb/sec even with 16 bytes reading)
Well, what do we have then? The ratio is easy to calculate,
the system of equations is like that:
x+y=75sec & 2*x = 30sec => y:x = 1:4
where
'x' is relative time for fread and
'y' is a relative time for fseek
Thus, we see that fseek is about 4 times slower then fread.
Attention:
I got glibc 2.2 only to test.
However the similar results were obtained with last cygwin
distribution. Hm, it's difficult to say what glibc is used
there, but drepper should know I guess :-)
thanks a priori for any answers!
// -------------- TEST BEGIN
#include <cassert>
#include <cstdio>
#include <cstdlib>
#include <iostream>
using namespace std;
main(int argc, char**argv) {
clog << "usage: "<<argv[0]<<" filename readby vbufsize fseekgap\n";
assert(argc==5);
long sz = atol(argv[2]);
long bufsz=atol(argv[3]);
long fseekgap = atol(argv[4]);
char *data = new char[sz];
char *buf = new char[bufsz+8]; // as show docs +8 is needed :-/
FILE* f=fopen(argv[1], "rb");
assert(f);
assert(!setvbuf(f, buf, _IOFBF, bufsz));
clog << "reading by " << sz << " bytes; buffer size = "<<
bufsz << "bytes... ";
long i=0;
while( fread(data, sz, 1, f) ) {
fseek(f, fseekgap, SEEK_CUR);
i++;
}
clog << "done\n\n";
clog << "fread is called: " << i << endl;
}
// -------------- TEST END
kind regards,
Valery A.Khamenya
---------------------------------------------------------------------------
Bioinformatics Department
BioVisioN AG, Hannover
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- fseek is inadequately slow (4 times slower then fread),
Khamenia, Valery <=