Re: [Gluster-devel] Re; Load balancing ...

[Gareth Bult]

Sorry, I'm trying to follow this but I'm coming a little unstuck ..

Am I right in thinking the rolling hash / rsync solution would involve syncing 
the file "on open" as per the current system .. and in order to do this, the 
server would have to read through the entire file in order to create the hashes?
(indeed it would need to do this on two servers to create hashes for 
comparison?)

So .. as a rough benchmark .. assume 50Mb/sec for a standard / modern SATA 
drive, opening a crashed 20G file is going to take 400 seconds or six minutes 
... ? (which would also flatten two servers for the duration)

Whereas a journal replay of 10M is going to take < 1s and be effectively 
transparent.
(I'm guessing this could also be done at open time ??)

I know I'm missing something here .. could anyone clarify or detail the rolling 
hash solution for me?

tia

Gareth.



----- Original Message -----
From: "Krishna Srinivas" <address@hidden>
To: "Gordan Bobic" <address@hidden>
Cc: "gluster-devel" <address@hidden>
Sent: Wednesday, April 30, 2008 10:02:00 AM GMT +00:00 GMT Britain, Ireland, 
Portugal
Subject: Re: [Gluster-devel] Re; Load balancing ...

On Wed, Apr 30, 2008 at 1:41 PM, Gordan Bobic <address@hidden> wrote:
> Martin Fick wrote:
>
>
> >
> > > A better solution would be to maintain a list of
> > > dirty blocks and use it during selfheal.
> > >
> >
> > Agreed, but why not make it infinitely granular and
> > keep a list of dirty file spans instead of blocks? This should be
> extremely space efficient.
> >
>
>  Is this complication and extra effort realy worth the benefit over straight
> rolling hash rsync approach? It seems to me that applying the rsync method
> at read-time would be a fairly minor mod that would solve 99% of the
> problem. No extra book-keeping would be required, only a change from copying
> the whole file to rsyncing the file.

correct, rsync algorithm is the best choice right now.

Krishna

>
>  Gordan
>
>
>
>
>  _______________________________________________
>  Gluster-devel mailing list
>  address@hidden
>  http://lists.nongnu.org/mailman/listinfo/gluster-devel
>


_______________________________________________
Gluster-devel mailing list
address@hidden
http://lists.nongnu.org/mailman/listinfo/gluster-devel