Re: [Duplicity-talk] duplicity on mac failing without a trace

[John Covici]

I have solved this problem.  I did two things: I was using crontab to
launch the shell scriptand I switched to using a launchd agent instead
-- I am not sure whether this was necessary, but I will experiment
with that later.

When I did just that, it gave me much more output, went all the way to
last full backup date and got some kind of gpg error which seemed to
have something to do with the  encoding.  So, I then exported the LANG
and PATH variables in the shell script and that made it work.  This
seems to be only necessary in Osx 12.

I hope this helps someone.

On Thu, 29 Dec 2016 11:00:28 -0500,
Kenneth Loafman wrote:
> 
> [1  <text/plain; UTF-8 (7bit)>]
> [2  <text/html; UTF-8 (quoted-printable)>]
> It's been awhile since I installed on MacOS. My librsync comes from Homebrew, 
> not Macports. Not sure if it matters at this point, just FYI. Plus, I check 
> pip for any python modules since that seems to be the standard way to go.
> 
> I know you will need the modules librsync-dev and python-dev, lockfile, and 
> maybe others. Check pip, then brew, then port.
> 
> I'll have 0.7.11 out in a day or so. Maybe Macports will be quick with an 
> install.
> 
> On Thu, Dec 29, 2016 at 9:47 AM, John Covici via Duplicity-talk 
> <address@hidden> wrote:
> 
>  SO, I am trying to build trunk without success. First thing it says
>  is to build the rsync module. It could not find librsync.h. I solved
>  by putting CCFLAGS=/opt/local/include before the python compilec.py
>  line and that made it worked. Now, when I try to back out one level
>  and ron with PYTHONPATH=duplicity bin/duplicity -V I get the following
>  traceback. First it had no python2, so I had to put a link between
>  /opt/local/bin/python and /opt/local/bin/python2
> 
>  Traceback (most recent call last):
>  File "bin/duplicity", line 40, in <module>
>  import threading
>  File
>  
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py",
>  line 13, in <module>
>  from collections import deque as _deque
>  File "/usr/local/src/duplicity/duplicity/collections.py", line
>  30, in <module>
>  from duplicity import log
>  File
>  
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/duplicity/log.py",
>  line 28, in <module>
>  import logging
>  File
>  
> "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/__init__.py",
>  line 201, in <module>
>  _lock = threading.RLock()
>  AttributeError: 'module' object has no attribute 'RLock'
> 
>  I used macports to install python and duplicity 0.7.10. Should I have
>  done this some other way?
> 
>  On Mon, 26 Dec 2016 07:32:13 -0500,
>  edgar.soldin--- via Duplicity-talk wrote:
>  >
>  > John,
>  >
>  > did you already find and check the OSX system logs?
>  >
>  > ..ede
>  >
>  > On 25.12.2016 22:49, John Covici wrote:
>  > > How do I do that now that Osx has restrictions on where things go. I
>  > > installed from macports, anny way to match that up. Duplicity is
>  > > in/opt/bin if memory serves.
>  > >
>  > > On Sun, 25 Dec 2016 15:05:15 -0500,
>  > > Kenneth Loafman via Duplicity-talk wrote:
>  > >>
>  > >> [1 <multipart/alternative (7bit)>]
>  > >> [1.1 <text/plain; UTF-8 (7bit)>]
>  > >> [1.2 <text/html; UTF-8 (quoted-printable)>]
>  > >> It was tested under MacOS, Yakety, and Zesty. So, it should work under 
> any of those without stdin. It would require gpg-agent, but that's normal.
>  > >>
>  > >> I do my dev under MacOS, but test on Ubuntu VMs, and again under the 
> Launchpad Build system for the daily PPA. All good.
>  > >>
>  > >> @John, if you don't mind installing from the trunk, please do so and 
> try it out. All of our tests pass now.
>  > >>
>  > >> ...Thanks,
>  > >> ...Ken
>  > >>
>  > >> On Sun, Dec 25, 2016 at 12:58 PM, <address@hidden> wrote:
>  > >>
>  > >> hey Ken,
>  > >>
>  > >> you changed some gpg parameter and seem to have debugged it on OSX. 
> could you really recreate the stackless dying? what confuses me is that it 
> works for John as root, but not as another user.
>  > >>
>  > >> ..ede
>  > >>
>  > >> On 25.12.2016 16:17, Kenneth Loafman wrote:
>  > >> > I am going to release 0.7.11 in the next day or so. There are 
> specific fixes for MACOS. That should fix your problems.
>  > >> >
>  > >> > ...Ken
>  > >> >
>  > >> >
>  > >> > On Sun, Dec 25, 2016 at 6:47 AM, edgar.soldin--- via Duplicity-talk 
> <address@hidden <mailto:address@hidden>> wrote:
>  > >> >
>  > >> > smells OSX specific. try to find out where OSX saves log files and 
> see if some info ends up in there. your -v9 output holds no error or stack or 
> such.
>  > >> >
>  > >> > btw. i remember there was some file limit that needed setting under 
> OSX in the past
>  > >> > https://www.google.com/search?q=ulimit+duplicity 
> <https://www.google.com/search?q=ulimit+duplicity>
>  > >> > .
>  > >> >
>  > >> > ..ede/duply.net <http://duply.net>
>  > >> >
>  > >> > On 22.12.2016 20:10, John Covici via Duplicity-talk wrote:
>  > >> > > Well I did a -v 9 and disabled asyncronous uploads and here is what 
> I
>  > >> > > got. If its too big for the list, I will send a link.
>  > >> > >
>  > >> > > Script started on Thu Dec 22 14:06:00 2016
>  > >> > > command: /opt/local/bin/duplicity -v 9 --exclude-filelist 
> /Users/covici/exclude_duplicity.txt --name backup_home_directory 
> --encrypt-key address@hidden <mailto:address@hidden> --volsize 300 
> /Users/covici
>  azure://mac
>  > >> > > ^D Using archive dir: 
> /var/root/.cache/duplicity/backup_home_directory
>  > >> > > Using backup name: backup_home_directory
>  > >> > > Import of duplicity.backends.acdclibackend Succeeded
>  > >> > > Import of duplicity.backends.azurebackend Succeeded
>  > >> > > Import of duplicity.backends.b2backend Succeeded
>  > >> > > Import of duplicity.backends.botobackend Succeeded
>  > >> > > Import of duplicity.backends.cfbackend Succeeded
>  > >> > > Import of duplicity.backends.copycombackend Succeeded
>  > >> > > Import of duplicity.backends.dpbxbackend Failed: No module named 
> dropbox
>  > >> > > Import of duplicity.backends.gdocsbackend Succeeded
>  > >> > > Import of duplicity.backends.giobackend Succeeded
>  > >> > > Import of duplicity.backends.hsibackend Succeeded
>  > >> > > Import of duplicity.backends.hubicbackend Succeeded
>  > >> > > Import of duplicity.backends.imapbackend Succeeded
>  > >> > > Import of duplicity.backends.lftpbackend Succeeded
>  > >> > > Import of duplicity.backends.localbackend Succeeded
>  > >> > > Import of duplicity.backends.mediafirebackend Succeeded
>  > >> > > Import of duplicity.backends.megabackend Succeeded
>  > >> > > Import of duplicity.backends.multibackend Succeeded
>  > >> > > Import of duplicity.backends.ncftpbackend Succeeded
>  > >> > > Import of duplicity.backends.onedrivebackend Succeeded
>  > >> > > Import of duplicity.backends.par2backend Succeeded
>  > >> > > Import of duplicity.backends.pydrivebackend Succeeded
>  > >> > > Import of duplicity.backends.rsyncbackend Succeeded
>  > >> > > Import of duplicity.backends.ssh_paramiko_backend Succeeded
>  > >> > > Import of duplicity.backends.ssh_pexpect_backend Succeeded
>  > >> > > Import of duplicity.backends.swiftbackend Succeeded
>  > >> > > Import of duplicity.backends.sxbackend Succeeded
>  > >> > > Import of duplicity.backends.tahoebackend Succeeded
>  > >> > > Import of duplicity.backends.webdavbackend Succeeded
>  > >> > > Reading globbing filelist /Users/covici/exclude_duplicity.txt
>  > >> > > Main action: inc
>  > >> > > 
> ================================================================================
>  > >> > > duplicity 0.7.10 (August 20, 2016)
>  > >> > > Args: /opt/local/bin/duplicity -v 9 --exclude-filelist 
> /Users/covici/exclude_duplicity.txt --name backup_home_directory 
> --encrypt-key address@hidden <mailto:address@hidden> --volsize 300 
> /Users/covici
>  azure://mac
>  > >> > > Darwin John-Covicis-MacBook-Pro-2.local 16.3.0 Darwin Kernel 
> Version 16.3.0: Thu Nov 17 20:23:58 PST 2016; 
> root:xnu-3789.31.2~1/RELEASE_X86_64 x86_64 i386
>  > >> > > 
> /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python
>  2.7.13 (default, Dec 18 2016, 05:36:03)
>  > >> > > [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.38)]
>  > >> > > 
> ================================================================================
>  > >> > > Using temporary directory /tmp/duplicity-4Ml9sV-tempdir
>  > >> > > Registering (mkstemp) temporary file 
> /tmp/duplicity-4Ml9sV-tempdir/mkstemp-HxgXAa-1
>  > >> > > Temp has 322646118400 available, backup will use approx 408944640.
>  > >> > > Local and Remote metadata are synchronized, no sync needed.
>  > >> > > 3640 files exist on backend
>  > >> > > 262 files exist in cache
>  > >> > > Extracting backup chains from list of files: 
> [u'duplicity-full-signatures.20160504T211416Z.sigtar.gpg', 
> u'duplicity-full.20160504T211416Z.manifest.gpg', 
> u'duplicity-full.20160504T211416Z.vol1.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol10.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol100.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1000.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1001.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1002.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1003.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1004.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1005.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1006.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1007.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1008.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1009.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol101.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1010.difftar.gpg', u!
>  > >> > 'd!
>  > >> > > uplicity-full.20160504T211416Z.vol1011.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1012.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1013.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1014.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1015.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1016.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1017.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1018.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1019.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol102.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1020.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1021.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1022.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1023.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1024.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1025.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1026.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1!
>  > >> > 02!
>  > >> > > 7.difftar.gpg', u'duplicity-full.20160504T211416Z.vol1028.diff!
>  > >> > > tar.gpg', u'duplicity-full.20160504T211416Z.vol1029.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol103.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1030.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1031.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1032.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1033.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1034.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1035.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1036.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1037.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1038.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1039.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol104.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1040.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1041.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1042.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1043.difftar.gpg', 
> u'duplicity-full.20160504T!
>  > >> > 21!
>  > >> > > 1416Z.vol1044.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1045.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1046.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1047.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1048.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1049.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol105.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1050.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1051.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1052.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1053.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1054.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1055.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1056.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1057.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1058.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1059.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol106.difftar.gpg', u'duplic!
>  > >> > it!
>  > >> > > y-full.20160504T211416Z.vol1060.difftar.gpg', 
> u'duplicity-full.20160504!
>  > >> > > T211416Z..vol1061.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1062.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1063.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1064.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1065.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1066.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1067.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1068.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1069.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol107.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1070.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1071.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1072.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1073.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1074.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1075.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1076.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1077.difftar.gpg', u'd!
>  > >> > up!
>  > >> > > licity-full.20160504T211416Z.vol1078.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1079.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol108.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1080.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1081.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1082.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1083.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1084.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1085.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1086.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1087.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1088.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1089.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol109.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1090.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1091.difftar.gpg',
>  > >> u'duplicity-full.20160504T211416Z.vol1092.difftar.gpg', 
> u'duplicity-full.20160504T211416Z.vol1093!
>  > >> > .d!
>  > >> > > ifftar.gpg', u'duplicity-full.20160504T211416Z.vol1094.
> 

-- 
Your life is like a penny.  You're going to lose it.  The question is:
How do
you spend it?

         John Covici
         address@hidden