[Gluster-devel] Re; Caching ...

[Gareth Bult]

Hi, 

Can someone tell me, is the IO cache per mount point, or per process ? 

I suspect it's per mount point, in which case running a number of intensive 
processes (i.e. >1) on one mount point can make the IO cache pretty pointless. 
I've not had time to "prove" this just yet, but .. I have one process that runs 
in 30 seconds ... when I run 3 concurrent processes they complete in around 5 
mins .. (each process operates on a different / large file) 

Would it be possible (!) to specify either in the client config or at mount 
time that an IO cache was per-process rather than per mount point ? 
(or would the solution simply to remount the same partition many times ...?) 
Or even better, that cache memory is tagged so each process gets an even share 
of the cache and intensive reads by one process do not wipe out cached entries 
of another process .. i.e. some sort of sharing on a process basis rather than 
an absolute drop the least recently used basis? 

Just as a matter of interest, I've noticed the following in terms of 
performance which has surprised me somewhat; 

a. Running XEN instances off disk images on a mounted gluster filesystem 
provides much better performance than I would have anticipated 
b. Running multi-process / multi-file access off a gluster filesystem is slower 
than anticipated, indeed performance seems to drop-off exponentially with the 
number of processes involved in the IO. 

I am using readahead and IO cache and I'm still tuning .. but if there's 
anything specific performance-wise I should be looking at with regards to 
multi-process access, an advice would be most welcome. 

Regards, 
Gareth.