Re: [Duplicity-talk] duplicity on mac failing without a trace

[John Covici]

I use script and am still doing so, exactly the same, but under
launchd I seemed to get more output -- lots more.  I am not sure
whether it was some standard out problem, or the fact that I used
launchd instead of cron.  I may experiment more just for curiosity's
sake.  Also, the lock file was released which leads me to believe that
cron did not work correctly under Sierra.

On Fri, 30 Dec 2016 11:06:42 -0500,
address@hidden wrote:
> 
> John,
> 
> how did you retrieve the output before? is it possible that you redirected it 
> to a log file but forgot redirect STDERR as well?
> 
> ..ede/duply.net
> 
> On 30.12.2016 16:55, John Covici via Duplicity-talk wrote:
> > I have solved this problem.  I did two things: I was using crontab to
> > launch the shell scriptand I switched to using a launchd agent instead
> > -- I am not sure whether this was necessary, but I will experiment
> > with that later.
> > 
> > When I did just that, it gave me much more output, went all the way to
> > last full backup date and got some kind of gpg error which seemed to
> > have something to do with the  encoding.  So, I then exported the LANG
> > and PATH variables in the shell script and that made it work.  This
> > seems to be only necessary in Osx 12.
> > 
> > I hope this helps someone.
> > 
> > On Thu, 29 Dec 2016 11:00:28 -0500,
> > Kenneth Loafman wrote:
> >>
> >> [1  <text/plain; UTF-8 (7bit)>]
> >> [2  <text/html; UTF-8 (quoted-printable)>]
> >> It's been awhile since I installed on MacOS. My librsync comes from 
> >> Homebrew, not Macports. Not sure if it matters at this point, just FYI. 
> >> Plus, I check pip for any python modules since that seems to be the 
> >> standard way to go.
> >>
> >> I know you will need the modules librsync-dev and python-dev, lockfile, 
> >> and maybe others. Check pip, then brew, then port.
> >>
> >> I'll have 0.7.11 out in a day or so. Maybe Macports will be quick with an 
> >> install.
> >>
> >> On Thu, Dec 29, 2016 at 9:47 AM, John Covici via Duplicity-talk 
> >> <address@hidden> wrote:
> >>
> >>  SO, I am trying to build trunk without success. First thing it says
> >>  is to build the rsync module. It could not find librsync.h. I solved
> >>  by putting CCFLAGS=/opt/local/include before the python compilec.py
> >>  line and that made it worked. Now, when I try to back out one level
> >>  and ron with PYTHONPATH=duplicity bin/duplicity -V I get the following
> >>  traceback. First it had no python2, so I had to put a link between
> >>  /opt/local/bin/python and /opt/local/bin/python2
> >>
> >>  Traceback (most recent call last):
> >>  File "bin/duplicity", line 40, in <module>
> >>  import threading
> >>  File
> >>  
> >> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
> >>  line 13, in <module>
> >>  from collections import deque as _deque
> >>  File "/usr/local/src/duplicity/duplicity/collections.py", line
> >>  30, in <module>
> >>  from duplicity import log
> >>  File
> >>  
> >> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/log.py",
> >>  line 28, in <module>
> >>  import logging
> >>  File
> >>  
> >> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py",
> >>  line 201, in <module>
> >>  _lock = threading.RLock()
> >>  AttributeError: 'module' object has no attribute 'RLock'
> >>
> >>  I used macports to install python and duplicity 0.7.10. Should I have
> >>  done this some other way?
> >>
> >>  On Mon, 26 Dec 2016 07:32:13 -0500,
> >>  edgar.soldin--- via Duplicity-talk wrote:
> >>  >
> >>  > John,
> >>  >
> >>  > did you already find and check the OSX system logs?
> >>  >
> >>  > ..ede
> >>  >
> >>  > On 25.12.2016 22:49, John Covici wrote:
> >>  > > How do I do that now that Osx has restrictions on where things go. I
> >>  > > installed from macports, anny way to match that up. Duplicity is
> >>  > > in/opt/bin if memory serves.
> >>  > >
> >>  > > On Sun, 25 Dec 2016 15:05:15 -0500,
> >>  > > Kenneth Loafman via Duplicity-talk wrote:
> >>  > >>
> >>  > >> [1 <multipart/alternative (7bit)>]
> >>  > >> [1.1 <text/plain; UTF-8 (7bit)>]
> >>  > >> [1.2 <text/html; UTF-8 (quoted-printable)>]
> >>  > >> It was tested under MacOS, Yakety, and Zesty. So, it should work 
> >> under any of those without stdin. It would require gpg-agent, but that's 
> >> normal.
> >>  > >>
> >>  > >> I do my dev under MacOS, but test on Ubuntu VMs, and again under the 
> >> Launchpad Build system for the daily PPA. All good.
> >>  > >>
> >>  > >> @John, if you don't mind installing from the trunk, please do so and 
> >> try it out. All of our tests pass now.
> >>  > >>
> >>  > >> ...Thanks,
> >>  > >> ...Ken
> >>  > >>
> >>  > >> On Sun, Dec 25, 2016 at 12:58 PM, <address@hidden> wrote:
> >>  > >>
> >>  > >> hey Ken,
> >>  > >>
> >>  > >> you changed some gpg parameter and seem to have debugged it on OSX. 
> >> could you really recreate the stackless dying? what confuses me is that it 
> >> works for John as root, but not as another user.
> >>  > >>
> >>  > >> ..ede
> >>  > >>
> >>  > >> On 25.12.2016 16:17, Kenneth Loafman wrote:
> >>  > >> > I am going to release 0.7.11 in the next day or so. There are 
> >> specific fixes for MACOS. That should fix your problems.
> >>  > >> >
> >>  > >> > ...Ken
> >>  > >> >
> >>  > >> >
> >>  > >> > On Sun, Dec 25, 2016 at 6:47 AM, edgar.soldin--- via 
> >> Duplicity-talk <address@hidden <mailto:address@hidden>> wrote:
> >>  > >> >
> >>  > >> > smells OSX specific. try to find out where OSX saves log files and 
> >> see if some info ends up in there. your -v9 output holds no error or stack 
> >> or such.
> >>  > >> >
> >>  > >> > btw. i remember there was some file limit that needed setting 
> >> under OSX in the past
> >>  > >> > https://www.google.com/search?q=ulimit+duplicity 
> >> <https://www.google.com/search?q=ulimit+duplicity>
> >>  > >> > .
> >>  > >> >
> >>  > >> > ..ede/duply.net <http://duply.net>
> >>  > >> >
> >>  > >> > On 22.12.2016 20:10, John Covici via Duplicity-talk wrote:
> >>  > >> > > Well I did a -v 9 and disabled asyncronous uploads and here is 
> >> what I
> >>  > >> > > got. If its too big for the list, I will send a link.
> >>  > >> > >
> >>  > >> > > Script started on Thu Dec 22 14:06:00 2016
> >>  > >> > > command: /opt/local/bin/duplicity -v 9 --exclude-filelist 
> >> /Users/covici/exclude_duplicity.txt --name backup_home_directory 
> >> --encrypt-key address@hidden <mailto:address@hidden> --volsize 300 
> >> /Users/covici
> >>  azure://mac
> >>  > >> > > ^D Using archive dir: 
> >> /var/root/.cache/duplicity/backup_home_directory
> >>  > >> > > Using backup name: backup_home_directory
> >>  > >> > > Import of duplicity.backends.acdclibackend Succeeded
> >>  > >> > > Import of duplicity.backends.azurebackend Succeeded
> >>  > >> > > Import of duplicity.backends.b2backend Succeeded
> >>  > >> > > Import of duplicity.backends.botobackend Succeeded
> >>  > >> > > Import of duplicity.backends.cfbackend Succeeded
> >>  > >> > > Import of duplicity.backends.copycombackend Succeeded
> >>  > >> > > Import of duplicity.backends.dpbxbackend Failed: No module named 
> >> dropbox
> >>  > >> > > Import of duplicity.backends.gdocsbackend Succeeded
> >>  > >> > > Import of duplicity.backends.giobackend Succeeded
> >>  > >> > > Import of duplicity.backends.hsibackend Succeeded
> >>  > >> > > Import of duplicity.backends.hubicbackend Succeeded
> >>  > >> > > Import of duplicity.backends.imapbackend Succeeded
> >>  > >> > > Import of duplicity.backends.lftpbackend Succeeded
> >>  > >> > > Import of duplicity.backends.localbackend Succeeded
> >>  > >> > > Import of duplicity.backends.mediafirebackend Succeeded
> >>  > >> > > Import of duplicity.backends.megabackend Succeeded
> >>  > >> > > Import of duplicity.backends.multibackend Succeeded
> >>  > >> > > Import of duplicity.backends.ncftpbackend Succeeded
> >>  > >> > > Import of duplicity.backends.onedrivebackend Succeeded
> >>  > >> > > Import of duplicity.backends.par2backend Succeeded
> >>  > >> > > Import of duplicity.backends.pydrivebackend Succeeded
> >>  > >> > > Import of duplicity.backends.rsyncbackend Succeeded
> >>  > >> > > Import of duplicity.backends.ssh_paramiko_backend Succeeded
> >>  > >> > > Import of duplicity.backends.ssh_pexpect_backend Succeeded
> >>  > >> > > Import of duplicity.backends.swiftbackend Succeeded
> >>  > >> > > Import of duplicity.backends.sxbackend Succeeded
> >>  > >> > > Import of duplicity.backends.tahoebackend Succeeded
> >>  > >> > > Import of duplicity.backends.webdavbackend Succeeded
> >>  > >> > > Reading globbing filelist /Users/covici/exclude_duplicity.txt
> >>  > >> > > Main action: inc
> >>  > >> > > 
> >> ================================================================================
> >>  > >> > > duplicity 0.7.10 (August 20, 2016)
> >>  > >> > > Args: /opt/local/bin/duplicity -v 9 --exclude-filelist 
> >> /Users/covici/exclude_duplicity.txt --name backup_home_directory 
> >> --encrypt-key address@hidden <mailto:address@hidden> --volsize 300 
> >> /Users/covici
> >>  azure://mac
> >>  > >> > > Darwin John-Covicis-MacBook-Pro-2.local 16.3.0 Darwin Kernel 
> >> Version 16.3.0: Thu Nov 17 20:23:58 PST 2016; 
> >> root:xnu-3789.31.2~1/RELEASE_X86_64 x86_64 i386
> >>  > >> > > 
> >> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
> >>  2.7.13 (default, Dec 18 2016, 05:36:03)
> >>  > >> > > [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)]
> >>  > >> > > 
> >> ================================================================================
> >>  > >> > > Using temporary directory /tmp/duplicity-4Ml9sV-tempdir
> >>  > >> > > Registering (mkstemp) temporary file 
> >> /tmp/duplicity-4Ml9sV-tempdir/mkstemp-HxgXAa-1
> >>  > >> > > Temp has 322646118400 available, backup will use approx 
> >> 408944640.
> >>  > >> > > Local and Remote metadata are synchronized, no sync needed.
> >>  > >> > > 3640 files exist on backend
> >>  > >> > > 262 files exist in cache
> >>  > >> > > Extracting backup chains from list of files: 
> >> [u'duplicity-full-signatures.20160504T211416Z.sigtar.gpg', 
> >> u'duplicity-full.20160504T211416Z.manifest.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol10.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol100.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1000.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1001.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1002.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1003.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1004.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1005.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1006.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1007.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1008.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1009.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol101.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1010.difftar.gpg', u!
> >>  > >> > 'd!
> >>  > >> > > uplicity-full.20160504T211416Z.vol1011.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1012.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1013.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1014.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1015.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1016.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1017.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1018.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1019.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol102.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1020.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1021.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1022.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1023.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1024.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1025.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1026.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1!
> >>  > >> > 02!
> >>  > >> > > 7.difftar.gpg', u'duplicity-full.20160504T211416Z.vol1028.diff!
> >>  > >> > > tar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1029.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol103.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1030.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1031.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1032.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1033.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1034.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1035.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1036.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1037.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1038.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1039.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol104.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1040.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1041.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1042.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1043.difftar.gpg', 
> >> u'duplicity-full.20160504T!
> >>  > >> > 21!
> >>  > >> > > 1416Z.vol1044.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1045.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1046.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1047.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1048.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1049.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol105.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1050.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1051.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1052.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1053.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1054.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1055.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1056.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1057.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1058.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1059.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol106.difftar.gpg', u'duplic!
> >>  > >> > it!
> >>  > >> > > y-full.20160504T211416Z.vol1060.difftar.gpg', 
> >> u'duplicity-full.20160504!
> >>  > >> > > T211416Z..vol1061.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1062.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1063.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1064.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1065.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1066.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1067.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1068.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1069.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol107.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1070.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1071.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1072.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1073.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1074.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1075.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1076.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1077.difftar.gpg', u'd!
> >>  > >> > up!
> >>  > >> > > licity-full.20160504T211416Z.vol1078.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1079.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol108.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1080.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1081.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1082.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1083.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1084.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1085.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1086.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1087.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1088.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1089.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol109.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1090.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1091.difftar.gpg',
> >>  > >> u'duplicity-full.20160504T211416Z.vol1092.difftar.gpg', 
> >> u'duplicity-full.20160504T211416Z.vol1093!
> >>  > >> > .d!
> >>  > >> > > ifftar.gpg', u'duplicity-full.20160504T211416Z.vol1094.
> >>
> > 

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         address@hidden