Re: [Duplicity-talk] Sliding window backup strategy?

[Timothee Besset]

Lars Marowsky-Bree wrote:
> On 2008-10-05T11:44:36, Serge Wroclawski <address@hidden> wrote:
> 

[..]

> 
> Do you think something likethis would be worthwhile to add to duplicity?
> I like it's simplicity and client-centric / dump-repository approach,
> and of course the encryption etc, which I think would be quite hard to
> add to amanda.
> 
> Basically, it boils down to marking n% of the files dirty regardless of
> whether they have changed or not - how hard can that be?
> 
> (He rhetorically asked, mistakenly ;-)
> 
> I'd welcome some feedback on how difficult more experienced duplicity
> developers think this would be in the terms of the data structures etc.
> 

I am completely on board with the idea that a full backup should only be
performed once, and that the full backup should be maintained with
little to no extra incrementals to re-apply.

As far as implementation, a sliding window as you described is basically
doing a full backup but spreading it over time so it hurts "less" ..

When I suggested this same basic feature (only one full backup ever) ..
some months ago on this list, I was thinking more along the lines of
'reverse delta' .. for instance like subversion does. The basic idea is
that you always keep the latest version of the file and you maintain
deltas going back in time towards earlier versions. That way you can
drop the old stuff as you see fit.

I think however that such an implementation requires some server side
intelligence, and it would be pretty hard to fit in duplicity's approach
of a dumb untrusted remote storage (which was also one of the aspects
that initially drew me to it)

If the remote server is more 'intelligent', and you could run software
on it, how about writing something to build a new full backup from an
old full backup and a set of incrementals? A safe approach would not
touch the old full backup and build a complete new one, if you need to
save some space you could also incrementally turn the old full into the
new one..

TTimo