duplicity-talk
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Duplicity-talk] Ubuntu One backend in 0.6.20, ascii codec error


From: Paul Barker
Subject: Re: [Duplicity-talk] Ubuntu One backend in 0.6.20, ascii codec error
Date: Tue, 06 Nov 2012 11:55:52 +0000
User-agent: Roundcube Webmail/0.7.1

Mike,

Minor point I just noticed, the variable is already made a bytearray by an earlier line, so just removing str() would have done. Think it's actually more logical on that line anyway.

Would suggest removing the duplication:

--- a/duplicity/backends/u1backend.py   Tue Nov 06 11:48:54 2012 +0000
+++ b/duplicity/backends/u1backend.py   Tue Nov 06 11:49:12 2012 +0000
@@ -199,7 +199,7 @@
log.Info("uploading file %s to location %s" % (remote_filename, remote_full))

         fh=open(source_path.name,'rb')
-        data = bytearray(fh.read())
+        data = fh.read()
         fh.close()

         content_type = 'application/octet-stream'

Also, the oauth signature method is specified in two places, in OAuthHttpClient.__init__() and again in _get_oauth_request_header. The variable signature_method is set but not used, so we could make that used:

--- a/duplicity/backends/u1backend.py   Tue Nov 06 11:50:27 2012 +0000
+++ b/duplicity/backends/u1backend.py   Tue Nov 06 11:53:49 2012 +0000
@@ -60,7 +60,7 @@
             token=self.token,
             parameters=dict(parse_qsl(query))
         )
- oauth_request.sign_request(oauth.OAuthSignatureMethod_HMAC_SHA1(),
+        oauth_request.sign_request(self.signature_method,
                                    self.consumer, self.token)
         return oauth_request.to_header()


Both of these are very minor tidyups I know. I should probably learn how to use bazaar, my experience is all in mercurial.

Thanks,

Paul Barker

On 2012-11-03 21:39, Michael Terry wrote:
I tested with this patch and the backend seems to work for simple use
cases anyway.  See merge filed here to include the patch in trunk:

https://code.launchpad.net/~mterry/duplicity/u1-ascii-error/+merge/132789
[16]

-mt

On 2 November 2012 09:32, Paul Barker <address@hidden [17]>
wrote:

Sorry for the multitude of posts, found a very quick and dirty fix
that
works for me:

diff -r 8adea0b1c193 duplicity/backends/u1backend.py
--- a/duplicity/backends/u1backend.py   Fri Nov 02 12:07:06 2012
+0000
+++ b/duplicity/backends/u1backend.py   Fri Nov 02 13:30:36 2012
+0000
@@ -207,7 +207,7 @@
                    "Content-Type": content_type}
         resp, content = self.client.request(remote_full,
                                           
 method="PUT",
-                                          
 body=str(data),
+                                          
 body=bytearray(data),
                                           
 headers=headers)

     def get(self, filename, local_path):

Still think using httplib rather than httplib2 would be a potential
improvement.

Thanks,

Paul Barker

On Fri, 2 Nov 2012 12:58:16 +0000

Paul Barker <address@hidden [1]> wrote:

> Ive found the cause of the ascii decode error, and it has nothing
to
> do with the oauth library used. The url argument to
> OAuthHttpClient.request() is a unicode string which causes
httplib2
> (or httplib, which I have also tried) to treat the entire HTTP
> request as a unicode string. It then comes to appending the
binary
> data in a POST request, treats it as ASCII text and complains
that it
> has bytes with values greater than 0x80 which cannot be converted
> from ASCII to unicode.
>
> See http://bugs.python.org/issue12398 [2] and
> http://bugs.python.org/issue11898 [3]
>
> Unsure why the URL is unicode so Ill leave the decision on what
> to do for others on this list. I personally think the bugfix
belongs
> in Python not in Duplicity as POSTing an ASCII or binary encoded
file
> to a unicode URL should not cause problems.
>
> Id also recommend the Ubuntu One backend uses httplib rather than
> httplib2. In httplib the response from the server is treated as a
file
> object you can read() rather than a string, so the entire file
for
> PUT/GET doesnt have to be read into memory in one go. It can by
> copied through in chunks (using shutil in the GET, handled
> automatically in the PUT) which may help for large volume sizes.
The
> backend I wrote takes this approach and Im happy to convert the
new
> backend to use this method once the current bug is fixed.
>
> Ill stick to using duplicity-0.6.19 with my Ubuntu One backend
for
> now as that at least works without bugs for me.
>
> Thanks,
>
> Paul Barker
>
> On Fri, 02 Nov 2012 11:00:11 +0000
> Paul Barker <address@hidden [4]> wrote:
>
> > Mike,
> >
> > Ive encountered the same error, Ive modified the backend to use
> > python-oauth2 instead of oauth and Im currently testing if that
> > fixes things, though that module was last updated 11 months ago
> > (https://github.com/simplegeo/python-oauth2 [5]).
> >
> > If that fixes things, is python-oauth2 a good idea or do you
still
> > want to aim for python-oauthlib?
> >
> > Thanks,
> >
> > Paul Barker
> >
> > On 2012-11-01 14:49, Michael Terry wrote:
> > > Yeah, I hadnt realized that had landed, until I saw the
release.
> > > Im going to be doing some testing on it before I upload it to
> > > Ubuntu.
> > >
> > > One thing I noticed is that is uses the unmaintained
python-oauth
> > > library (Ill be providing a branch that ports to
python-oauthlib).
> > >
> > > I havent encountered your bug yet, but if I do, Id be glad to
> > > provide a patch for that too.
> > >
> > > -mt
> > >
> > > On 1 November 2012 15:39, Richard <address@hidden [6]
[3]> wrote:
> > >
> > >> The note about the U1 backend in the man page has changed in
> > >> 0.6.20, so I tried to check that it still works for me by
doing a
> > >> full backup of a tiny fileset.
> > >>
> > >> I get:
> > >> Giving up on request after 5 attempts, last exception ascii
codec
> > >> cant decode byte 0x8c in position 0: ordinal not in
range(128)
> > >>
> > >> The system is Ubuntu 12.04, with python 2.7.3
> > >> I used the access token in env FTP_PASSWORD.
> > >>
> > >> _______________________________________________
> > >> Duplicity-talk mailing list
> > >> address@hidden [7] [1]
> > >> https://lists.nongnu.org/mailman/listinfo/duplicity-talk [8]
[2]
> > >
> > >
> > >
> > > Links:
> > > ------
> > > [1] mailto:address@hidden [9]
> > > [2] https://lists.nongnu.org/mailman/listinfo/duplicity-talk
[10]
> > > [3] mailto:address@hidden [11]
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden [12]
> > https://lists.nongnu.org/mailman/listinfo/duplicity-talk [13]
>
>
>

--

_______________________________________________
Duplicity-talk mailing list
address@hidden [14]
https://lists.nongnu.org/mailman/listinfo/duplicity-talk [15]



Links:
------
[1] mailto:address@hidden
[2] http://bugs.python.org/issue12398
[3] http://bugs.python.org/issue11898
[4] mailto:address@hidden
[5] https://github.com/simplegeo/python-oauth2
[6] mailto:address@hidden
[7] mailto:address@hidden
[8] https://lists.nongnu.org/mailman/listinfo/duplicity-talk
[9] mailto:address@hidden
[10] https://lists.nongnu.org/mailman/listinfo/duplicity-talk
[11] mailto:address@hidden
[12] mailto:address@hidden
[13] https://lists.nongnu.org/mailman/listinfo/duplicity-talk
[14] mailto:address@hidden
[15] https://lists.nongnu.org/mailman/listinfo/duplicity-talk
[16]

https://code.launchpad.net/~mterry/duplicity/u1-ascii-error/+merge/132789
[17] mailto:address@hidden



reply via email to

[Prev in Thread] Current Thread [Next in Thread]