Re: [Duplicity-talk] Fwd: GPG taking many ressources

[Henrik Olsen]

I have the following in my python library path (I use export 
PYTHONPATH="[prefix]/lib/python":$PYTHONPATH):

l GnuPGInterface.*
-rw-r--r--  1 holsen  wheel  22000 Jan 11  2002 GnuPGInterface.py
-rw-r--r--  1 holsen  wheel  22022 Aug  9 22:00 GnuPGInterface.pyc
Henrik-Olsens-MacBook:python holsen$ l duplicity/GnuPGInterface.*
-rw-r--r--  1 holsen  wheel  22914 Jul 25 18:14 duplicity/GnuPGInterface.py
-rw-r--r--  1 holsen  wheel  23217 Aug  9 22:01 duplicity/GnuPGInterface.pyc

The last one in lib/python/duplicity is the one including 'threaded_waitpid'. 

Does this look correct? Or can I risk duplicity is using the on in lib/python 
and not lib/python/duplicity with the PYTHONPATH I use?

/Henrik

On Aug 9, 2010, at 8:17 PM, Kenneth Loafman wrote:

> Henrik,
> 
> Not sure how this is packaged on the Mac side, but you need to check to make 
> sure that the GnuPGInterface.py module is in the duplicity source.  I had to 
> make some mods to harvest the child processes and the changes were not 
> propagated.  You can tell if it is the correct module by looking for the 
> function 'threaded_waitpid' starting on line 660 of the module.
> 
> If the module does not coexist with duplicity, or the module is the wrong 
> one, just copy the correct one to the duplicity library and your problem 
> should go away.  The correct one is in the duplicity tarball.
> 
> ...Thanks,
> ...Ken
> 
> Henrik Olsen wrote:
>> Bump... No one with similar issues or comments? It's starting to become so 
>> much an issue on my installation, that I'm slowly being "forced" to drop my 
>> otherwise excellent duplicity solution. The CPU usage is getting out of 
>> hand, unfortunately.
>> 
>> Best regards
>> Henrik
>> 
>> Begin forwarded message:
>> 
>>   
>> 
>>> From: Henrik Olsen <address@hidden>
>>> 
>>> Date: August 3, 2010 8:19:11 PM GMT+02:00
>>> To: Discussion of the backup program duplicity 
>>> <address@hidden>
>>> 
>>> Subject: GPG taking many ressources
>>> 
>>> Using Duplicity 0.6.0.9 on Mac OS X 10.6.4 on a 2010 Macbook Pro.
>>> 
>>> I'm noticing that GPG is using a lot of CPU ressources. Seems to take 99% 
>>> of CPU for loooong period of times when running something like this
>>> 
>>> /usr/bin/python /usr/local/duplicity/bin/duplicity --name=[name] 
>>> --archive-dir=/var/cache/duplicity --full-if-older-than 3M 
>>> --asynchronous-upload --verbosity 4 --num-retries 5 --volsize=50 
>>> --include=/Users/[username] --exclude=/** / 
>>> ftp://
>>> [cut]
>>> 
>>> I see many many GPGs running.
>>> 
>>> ps aux | grep gpg
>>> root      2995  92.6  0.0  2448072   1160   ??  RN   Mon10AM 875:02.33 gpg 
>>> --status-fd 112 --passphrase-fd 116 --logger-fd 109 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      3041   0.0  0.0  2436168    868   ??  SN   Mon10AM   0:05.08 gpg 
>>> --logger-fd 118 --passphrase-fd 122 --batch --no-tty --no-secmem-warning 
>>> --force-mdc --symmetric
>>> root      2993   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.73 gpg 
>>> --status-fd 108 --passphrase-fd 112 --logger-fd 105 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2990   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:04.08 gpg 
>>> --status-fd 104 --passphrase-fd 108 --logger-fd 101 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2970   0.0  0.0  2447048   1076   ??  SN   Mon10AM   0:01.92 gpg 
>>> --status-fd 100 --passphrase-fd 104 --logger-fd 97 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2967   0.0  0.0  2447048   1072   ??  SN   Mon10AM   0:00.46 gpg 
>>> --status-fd 96 --passphrase-fd 100 --logger-fd 93 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2964   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:02.84 gpg 
>>> --status-fd 92 --passphrase-fd 96 --logger-fd 89 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2941   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:00.39 gpg 
>>> --status-fd 88 --passphrase-fd 92 --logger-fd 85 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2939   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:00.44 gpg 
>>> --status-fd 84 --passphrase-fd 88 --logger-fd 81 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2937   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.52 gpg 
>>> --status-fd 80 --passphrase-fd 84 --logger-fd 77 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2935   0.0  0.0  2447048   1076   ??  SN   Mon10AM   0:00.57 gpg 
>>> --status-fd 76 --passphrase-fd 80 --logger-fd 73 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2927   0.0  0.0  2447048   1072   ??  SN   Mon10AM   0:00.43 gpg 
>>> --status-fd 72 --passphrase-fd 76 --logger-fd 69 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2923   0.0  0.0  2447048   1084   ??  SN   Mon10AM   0:00.44 gpg 
>>> --status-fd 68 --passphrase-fd 72 --logger-fd 65 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2920   0.0  0.0  2448072   1076   ??  SN   Mon10AM   0:00.55 gpg 
>>> --status-fd 64 --passphrase-fd 68 --logger-fd 61 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2917   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.72 gpg 
>>> --status-fd 60 --passphrase-fd 64 --logger-fd 57 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2893   0.0  0.0  2447572   1092   ??  SN   Mon10AM   0:01.20 gpg 
>>> --status-fd 56 --passphrase-fd 60 --logger-fd 53 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2884   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.68 gpg 
>>> --status-fd 52 --passphrase-fd 56 --logger-fd 49 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2879   0.0  0.0  2447048   1076   ??  SN   Mon10AM   0:01.01 gpg 
>>> --status-fd 48 --passphrase-fd 52 --logger-fd 45 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2869   0.0  0.0  2448072   1076   ??  SN   Mon10AM   0:01.61 gpg 
>>> --status-fd 44 --passphrase-fd 48 --logger-fd 41 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2857   0.0  0.0  2448072   1076   ??  SN   Mon10AM   0:01.95 gpg 
>>> --status-fd 40 --passphrase-fd 44 --logger-fd 37 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2849   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:00.56 gpg 
>>> --status-fd 36 --passphrase-fd 40 --logger-fd 33 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2830   0.0  0.0  2437832   1080   ??  SN   Mon10AM   0:01.86 gpg 
>>> --status-fd 32 --passphrase-fd 36 --logger-fd 29 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2822   0.0  0.0  2448072   1080   ??  SN   Mon10AM   0:01.47 gpg 
>>> --status-fd 28 --passphrase-fd 32 --logger-fd 25 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2815   0.0  0.0  2447048   1076   ??  SN   Mon10AM   0:00.45 gpg 
>>> --status-fd 24 --passphrase-fd 28 --logger-fd 21 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2813   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:00.58 gpg 
>>> --status-fd 20 --passphrase-fd 24 --logger-fd 17 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2803   0.0  0.0  2448072   1084   ??  SN   Mon10AM   0:00.48 gpg 
>>> --status-fd 16 --passphrase-fd 20 --logger-fd 13 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2801   0.0  0.0  2448072   1084   ??  SN   Mon10AM   0:00.68 gpg 
>>> --status-fd 12 --passphrase-fd 16 --logger-fd 9 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> root      2797   0.0  0.0  2447048   1080   ??  SN   Mon10AM   0:11.20 gpg 
>>> --status-fd 7 --passphrase-fd 12 --logger-fd 5 --batch --no-tty 
>>> --no-secmem-warning --decrypt
>>> 
>>> Is this amount and ressource usage to be expected? Can anything be done to 
>>> lower it? It's getting quite annoying. I'm launching my backup scripts via 
>>> Launch Daemons. I've set nice and low priority i/o on the launch daemon 
>>> plist.
>>> 
>>> /Henrik
>>>     
>>> 
>> 
>> 
>> _______________________________________________
>> Duplicity-talk mailing list
>> 
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>> 
>> 
>>   
>> 
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk