Re: [Duplicity-talk] Re: How to run duplicity with a Cron Job

[Mitchell Perilstein]

Some things to check:
Have you copied a PUBLIC key (~/.ssh/id_rsa.pub or id_dsa.pub) to the
remote machine's authorized_keys file?
Have you chowned his whole .ssh dir to the right user?
Is the authorized_keys file chmod 600?
Are you able to simply "ssh remote_machine -l root" doing the same thing
your cron job will do?

Also, another way to do it is run as root cron on each source machine to
be backed up, but send to a non-root account on the storage machine.  Then
you have only non-root SSH going around.  The downside is you have to
maintain everybody's cron and duplicity running script separately.

> Alms... Alms for a stupid user.
>
> I thought that I had it going on. Alas! I do not.
>
> Keychain starts at start-up. ssh-agent is running.
>
> I have followed 1) Carlos Justiniano's instruction for setting up
> automated backup of a Linux system via a distributed network using secure
> remote access and 2) Dom Hargreaves' suggestion of having the PASSPHRASE
> stored separately. So the remote backup script runs that file, exports the
> PASSPHRASE, then runs the duplicity request.
>
> To no avail, I still am prompted repeatedly:
>
> "Enter passphrase for key '/root/.ssh/id_rsa':"
>
> What am I doing wrong? Does keychain need to be running on the remote
> system, also?
>
> Your help, e.g., larger hammer, bullseye on wall, etc., would be
> appreciated.
>
>  -------------- Original message ----------------------
> From: address@hidden
>> Send Duplicity-talk mailing list submissions to
>>      address@hidden
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>>      http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>> or, via email, send a message with subject or body 'help' to
>>      address@hidden
>>
>> You can reach the person managing the list at
>>      address@hidden
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Duplicity-talk digest..."
>>
>>
>> Today's Topics:
>>
>>    1. Periodic Account Review (Wells Fargo)
>>    2. duplicity fails on gids/uids > 2097151 (address@hidden)
>>    3. V-g?od CI?IS V?AGRRA (Saxon Mack)
>>    4. setup.py options? (Axel)
>>    5. Re: gpg io error (Mitchell Perilstein)
>>    6. Re: ue2 It Works Fine VI?GRRA (Kaisa Racanelli)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Sat, 20 Aug 2005 04:39:50 +0100
>> From: Wells Fargo <address@hidden>
>> Subject: [Duplicity-talk] Periodic Account Review
>> To: address@hidden
>> Message-ID: <address@hidden>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> An HTML attachment was scrubbed...
>> URL:
>> http://lists.gnu.org/pipermail/duplicity-talk/attachments/20050820/1fc63ddc/atta
>> chment.html
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Sun, 21 Aug 2005 22:25:19 +0200
>> From: address@hidden
>> Subject: [Duplicity-talk] duplicity fails on gids/uids > 2097151
>> To: address@hidden
>> Message-ID: <address@hidden>
>> Content-Type: text/plain;    charset=ISO-8859-1
>>
>> I stumbled over a bug in duplicity where files with uids/gids greater
>> than
>> 2097151 (that's 7777777 in octal) will lead to corrupt backup files.
>> Incidently
>> there was a file with gid 4294967295 on my fs. When trying to restore
>> the
>> created backup-set Duplicity failed, saying:
>>
>> Traceback (most recent call last):
>>   File "/usr/bin/duplicity", line 358, in ?
>>     if __name__ == "__main__": main()
>>   File "/usr/bin/duplicity", line 352, in main
>>     else: incremental_backup(sig_chain)
>>   File "/usr/bin/duplicity", line 162, in incremental_backup
>>     bytes_written = write_multivol("inc", tarblock_iter,
>> globals.backend)
>>   File "/usr/bin/duplicity", line 75, in write_multivol
>>     globals.gpg_profile)
>>   File "/usr/lib/python2.3/site-packages/duplicity/gpg.py", line 211, in
>> GPGWriteFile
>>     try: data = block_iter.next(bytes_to_go).data
>>   File "/usr/lib/python2.3/site-packages/duplicity/diffdir.py", line
>> 407, in
>> next
>>     result = self.process(self.input_iter.next(), size)
>>   File "/usr/lib/python2.3/site-packages/duplicity/diffdir.py", line
>> 261, in
>> get_delta_iter_w_sig
>>     for new_path, sig_path in collated:
>>   File "/usr/lib/python2.3/site-packages/duplicity/diffdir.py", line
>> 177, in
>> collate2iters
>>     try: relem2 = riter2.next()
>>   File "/usr/lib/python2.3/site-packages/duplicity/diffdir.py", line
>> 230, in
>> combine_path_iters
>>     range(len(path_iter_list))))
>>   File "/usr/lib/python2.3/site-packages/duplicity/diffdir.py", line
>> 211, in
>> get_triple
>>     try: path = path_iter_list[iter_index].next()
>>   File "/usr/lib/python2.3/site-packages/duplicity/diffdir.py", line
>> 153, in
>> sigtar2path_iter
>>     ropath.init_from_tarinfo(tarinfo)
>>   File "/usr/lib/python2.3/site-packages/duplicity/path.py", line 171,
>> in
>> init_from_tarinfo
>>     else: raise PathException("Unknown tarinfo type %s" % (type,))
>> duplicity.path.PathException: Unknown tarinfo type
>>
>> And, as I later noticed, when creating the archive there was a Warning:
>> /usr/lib/python2.3/site-packages/duplicity/tarfile.py:390:
>> FutureWarning:
>> %u/%o/%x/%X of negative int will return a signed string in Python 2.4
>> and up
>>
>> As I tried to manually extract the archive with tar, it said something
>> about
>> "unexpected end of file".
>>
>> The problem is, that tar files created by tarfile.py can only store
>> uids/gids up
>> to 2097151, so I created a patch that sets uids/gids which are too high
>> to
>> 60001. This is rather a hack and since I'm new to duplicity and python
>> someone
>> more experienced should have a look at this problem.
>>
>> --- tarfile.py.org      2005-08-21 19:14:11.000000000 +0200
>> +++ tarfile.py  2005-08-21 21:40:23.000000000 +0200
>> @@ -385,6 +385,12 @@
>>      def getheader(self):
>>          """Return a tar header block as a 512 byte string.
>>          """
>> +        if self.uid > 2097151 or self.uid < 0:
>> +            sys.stderr.write("uid %i of file %s not in range. Setting
>> uid to
>> 60001\n" % (self.uid,self.name))
>> +            self.uid = 60001
>> +        if self.gid > 2097151 or self.gid < 0:
>> +            sys.stderr.write("gid %i of file %s not in range. Setting
>> gid to
>> 60001\n" % (self.gid, self.name))
>> +            self.gid = 60001
>>          # The following code was contributed by Detlef Lannert.
>>          parts = []
>>          for value, fieldsize in (
>>
>>
>>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Wed, 24 Aug 2005 13:49:15 -0500
>> From: "Saxon Mack" <address@hidden>
>> Subject: [Duplicity-talk] V-g?od CI?IS V?AGRRA
>> To: "Jaden Patterson" <address@hidden>
>> Message-ID: <address@hidden>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> Hello,
>> offered the brunette an open case with a flacon in it.Pushkino called
>> Yalta!
>> Its all clear! He went  there, got drunk, and now     That is  impolite,
>> Nikolai
>> Ivanovich! Im still a woman after all! Itspaper.melted away, how the sky
>> saddened and faded, how the woods turned black.rest of my  days, then I
>> ask the
>> Soviet government  to give me  a job in my     And what was this
>> magicians name?
>> Vassily Stepanovich did not know, hesparkled.     At once there dropped
>> from the
>> ceiling on to the floor a bewildered andthe kitchen, it could be
>> supposed that
>> Behemoth was precisely there, playing     There was movement in the
>> balcony, and
>> a joyful voice said:     Yes,  I do, the woman  said imploringly, and
>> suddenly
>> began repeatingnovel about Pilate.     Pilate looked at the high priest
>> with
>> dead eyes and, baring his teeth,there are things he cannot manage. He
>> cannot
>> manage this spring full moon.made of  stone because he was afraid to
>> move his
>> head, aflame with  infernal
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://lists.gnu.org/pipermail/duplicity-talk/attachments/20050824/94595b82/atta
>> chment.html
>>
>> ------------------------------
>>
>> Message: 4
>> Date: Fri, 26 Aug 2005 03:28:01 +0200
>> From: Axel <address@hidden>
>> Subject: [Duplicity-talk] setup.py options?
>> To: address@hidden
>> Message-ID: <address@hidden>
>> Content-Type: text/plain; charset=ISO-8859-15
>>
>> Hello,
>>
>> I'd like to use duplicity on MacOSX (Tiger) with a fink distribution as
>> base (especially for the librsync) but I have problems
>> getting the setup.py script to find the librsync. Does someone know some
>> options for the script to have it search on a given path
>> for the lib? Or is there a possibility to hack the script? Please help
>> or give me some hints.
>>
>> Thanks.
>>
>> Greets
>>
>> Axel
>>
>>
>>
>>
>> ------------------------------
>>
>> Message: 5
>> Date: Fri, 26 Aug 2005 10:15:04 -0400
>> From: Mitchell Perilstein <address@hidden>
>> Subject: Re: [Duplicity-talk] gpg io error
>> To: address@hidden
>> Cc: Nick Austin <address@hidden>
>> Message-ID: <address@hidden>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> Hi Folks,
>>
>>  >>  File "/usr/lib/python2.3/site-packages/duplicity/gpg.py", line 115,
>> in
>>  >>  close
>>  >>    self.gpg_process.wait()
>>  >>  File "/usr/lib/site-python/GnuPGInterface.py", line 639, in wait
>>  >>    raise IOError, "GnuPG exited non-zero, with code %d" % (e << 8)
>>  >>IOError: GnuPG exited non-zero, with code 131072
>>
>> Just wanted to give an update on this issue.  The bad news for the list
>> is I
>> wasn't able
>> to narrow it down to help solve the problem.  The good news for me is by
>> various
>> workarounds and hardenings I was able to get backups running again and
>> have not
>> seen the
>> error recur:
>>
>>   1. Clean up the source area.  There may have been some old or
>> inappropriate
>> symlinks
>> hanging around and there was some junk that didn't need backing up.  I'm
>> pretty
>> sure this
>> solved the I/O error ultimately, but just in case, read on...
>>
>>   2. Partition the source area into smaller chunks to be performed on
>> different
>> nights
>> (cron + scripty bits).
>>
>>   3. Manually perform a full backup to a local file:/// target, then
>> copy that
>> to the
>> remote backup host for each chunk.
>>
>>   4. Restart the cron+scripty backups going to the remote host scp://
>> target,
>> which holds
>> the newly uploaded full chunks.
>>
>>   5. A major improvement we've seen at some of our sites is a wrapping
>> layer
>> around ssh
>> and scp which performs a fallback and retry in the case of a slow or
>> unreliable
>> internet
>> connection acting up during backups.  Instead of failing the whole
>> backup, my
>> ssh wrapper
>> will keep retrying, timing out, and waiting longer times until it
>> succeeds.
>> From
>> duplicity's perspective, it just runs scp once and waits until
>> eventually it
>> happens or
>> fails after a lot of trying.
>>
>> The SSH wrappers are an effective hack, and we plan to share them with
>> the list
>> shortly,
>> but ultimately, this points out a few duplicity improvement wishes for a
>> new
>> thread.
>>
>> Best regards,
>> --
>> Mitchell Perilstein
>> Partner
>> ACE Technology Group, LLC
>> http://www.acetechgroup.com
>> (866) 229-1543 x11
>>
>> -------------- next part --------------
>> A non-text attachment was scrubbed...
>> Name: mitch.vcf
>> Type: text/x-vcard
>> Size: 301 bytes
>> Desc: not available
>> Url :
>> http://lists.gnu.org/pipermail/duplicity-talk/attachments/20050826/99c45e21/mitc
>> h.vcf
>>
>> ------------------------------
>>
>> Message: 6
>> Date: Tue, 30 Aug 2005 22:04:56 -0500
>> From: "Kaisa Racanelli" <address@hidden>
>> Subject: [Duplicity-talk] Re: ue2 It Works Fine VI?GRRA
>> To: "Petrica Bough" <address@hidden>
>> Message-ID: <address@hidden>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>>  On awakening, Margarita did not weep, as she often did, because she
>> Well, its
>> as I was saying, the administrator replied, sucking as if time at the
>> word
>> Attendant. The cylinder rang quietly in response, Ready! in his eyes, as
>> if
>> blocking his way, as if cautioning him. A window on the imaginable.
>> Anyone
>> entering Griboedovs first of all became involuntarily All the while the
>> waiters
>> were tying up the poet with napkins, a I was mistaken! Levi cried in a
>> completely hoarse voice. You are a jockeys cap on his little head, a
>> short
>> checkered jacket also made of air. knows where, sobbing with grief. No
>> brick,
>> the stranger interrupted imposingly, `will ever fall on you, then, how
>> man can
>> govern, if he is not only deprived of the opportunity Oh, my! Ivan
>> whispered
>> enviously. was trying to step out of the suns way. nonplussed by such a
>> telegram. If someone sends a telegram saying he has of the comic
>> masterpieces of
>> Gogol and Cervantes coincided with the writing
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> http://lists.gnu.org/pipermail/duplicity-talk/attachments/20050830/eaa8f805/atta
>> chment.html
>>
>> ------------------------------
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>
>>
>> End of Duplicity-talk Digest, Vol 27, Issue 4
>> *********************************************
>
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>


-- 
  Mitchell Perilstein
  ACE Technology Group, LLC
  address@hidden
  866 229 1543 x11