Re: [Duplicity-talk] Ubuntu One backend in 0.6.20, ascii codec error

[Paul Barker]

I've found the cause of the ascii decode error, and it has nothing to do
with the oauth library used. The 'url' argument to
OAuthHttpClient.request() is a unicode string which causes httplib2 (or
httplib, which I have also tried) to treat the entire HTTP request as a
unicode string. It then comes to appending the binary data in a POST
request, treats it as ASCII text and complains that it has bytes with
values greater than 0x80 which cannot be converted from ASCII to
unicode.

See http://bugs.python.org/issue12398 and
http://bugs.python.org/issue11898

Unsure why the URL is unicode so I'll leave the decision on what
to do for others on this list. I personally think the bugfix belongs in
Python not in Duplicity as POSTing an ASCII or binary encoded file to
a unicode URL should not cause problems.

I'd also recommend the Ubuntu One backend uses httplib rather than
httplib2. In httplib the response from the server is treated as a file
object you can read() rather than a string, so the entire file for
PUT/GET doesn't have to be read into memory in one go. It can by copied
through in chunks (using shutil in the GET, handled automatically in
the PUT) which may help for large volume sizes. The backend I wrote
takes this approach and I'm happy to convert the new backend to use
this method once the current bug is fixed.

I'll stick to using duplicity-0.6.19 with my Ubuntu One backend for now
as that at least works without bugs for me.

Thanks,

Paul Barker

On Fri, 02 Nov 2012 11:00:11 +0000
Paul Barker <address@hidden> wrote:

> Mike,
> 
> I've encountered the same error, I've modified the backend to use 
> python-oauth2 instead of 'oauth' and I'm currently testing if that
> fixes things, though that module was last updated 11 months ago 
> (https://github.com/simplegeo/python-oauth2).
> 
> If that fixes things, is python-oauth2 a good idea or do you still
> want to aim for python-oauthlib?
> 
> Thanks,
> 
> Paul Barker
> 
> On 2012-11-01 14:49, Michael Terry wrote:
> > Yeah, I hadnt realized that had landed, until I saw the release.  Im
> > going to be doing some testing on it before I upload it to Ubuntu.
> >
> > One thing I noticed is that is uses the unmaintained python-oauth
> > library (Ill be providing a branch that ports to python-oauthlib).
> >
> > I havent encountered your bug yet, but if I do, Id be glad to
> > provide a patch for that too.
> >
> > -mt
> >
> > On 1 November 2012 15:39, Richard <address@hidden [3]> wrote:
> >
> >> The note about the U1 backend in the man page has changed in
> >> 0.6.20, so I tried to check that it still works for me by doing a
> >> full backup of a tiny fileset.
> >>
> >> I get:
> >> Giving up on request after 5 attempts, last exception ascii codec
> >> cant decode byte 0x8c in position 0: ordinal not in range(128)
> >>
> >> The system is Ubuntu 12.04, with python 2.7.3
> >> I used the access token in env FTP_PASSWORD.
> >>
> >> _______________________________________________
> >> Duplicity-talk mailing list
> >> address@hidden [1]
> >> https://lists.nongnu.org/mailman/listinfo/duplicity-talk [2]
> >
> >
> >
> > Links:
> > ------
> > [1] mailto:address@hidden
> > [2] https://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > [3] mailto:address@hidden
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/duplicity-talk



--