[Duplicity-talk] A duplicity backend for Amazon S3

[Brian Sutherland]

Hi,

I've written a backend for duplicity that writes to the Amazon S3
service [1]. It uses the bitbucket.py module [2].

It allows you to do something like this:

    duplicity / s3+http://${access_key}:${secret_key}/${bucket}

and so on to store you backups on S3. Which, well is very cheap. If
others start using this, it would be nice for the code to be included in
duplicity.

(Beware, bitbucket.py is a fairly young module and may have some bugs)

[1] http://www.amazon.com/gp/browse.html/103-1203077-2066220?node=16427261
[2] http://www.other10percent.com/?p=18


The relevant code from the duplicity.backend module is very simple:

class BitBucketBackend(Backend):

        def __init__(self, parsed_url):
                import bitbucket
                self.module = bitbucket
                parts = parsed_url.suffix.split('/')
                bucket_name = parts[-1]
                AccessKeyID, SecretAccessKey = '/'.join(parts[:-1]).split(':')
                self.bucket = self.module.BitBucket(bucket_name,
                                                    access_key=AccessKeyID,
                                                    secret_key=SecretAccessKey,
                                                    page_size=100)

        def put(self, source_path, remote_filename = None):
                """Transfer source_path (Path object) to remote_filename 
(string)

                If remote_filename is None, get the filename from the last
                path component of pathname.

                """
                if not remote_filename:
                    remote_filename = source_path.get_filename()
                bits = self.module.Bits(filename=source_path.name)
                self.bucket[remote_filename] = bits

        def get(self, remote_filename, local_path):
                """Retrieve remote_filename and place in local_path"""
                bits = self.bucket[remote_filename]
                bits.to_file(local_path.name)
                local_path.setdata()

        def list(self):
                """Return list of filenames (strings) present in backend"""
                self.bucket.fetch_all_keys() # XXX I don't think this should be 
necessary
                return self.bucket.keys()

        def delete(self, filename_list):
                """Delete each filename in filename_list, in order if 
possible"""
                for file in filename_list:
                    del self.bucket[file]

# Dictionary relating protocol strings to backend_object classes.
protocol_class_dict = {"scp": scpBackend,
                                           "ssh": scpBackend,
                                           "file": LocalBackend,
                                           "ftp": ftpBackend,
                                           "rsync": rsyncBackend,
                                           "s3+http": BitBucketBackend}


-- 
Brian Sutherland

Metropolis - "it's the first movie with a robot. And she's a woman.
              And she's EVIL!!"