Re: [Duplicity-talk] OSError: [Errno 24] Too many open files

[Kenneth Loafman]

I cannot reproduce this on a 32bit machine at all.  I'll be able to test
on a 64bit machine sometime this week and, hopefully, find the problem.

...Ken

address@hidden wrote:
> i seem to get closer to it ...
> the more incrementals (regardless if file: or ftp: backend) the more
> defunct gpg processes appear .. but dissappear again while the backup is
> still running
> 
> I did
> initial backup of 3GB
> touched a new test file to have a change & did a 2nd backup
> touched a new test2 file to have a change & did a 3nd backup
> ...
> 
> and since the 3rd backup the defunct gpg processes keep appearing
> 
> essentially I say: Take any more than 3 incrementals in size backup and
> do a new incremental and you will see those zombie gpg processes ..
> 
> while I am not sure that this is what happens on the ulimit problem I
> thought I should report these observations ... ede
> 
> PS:
> 
> here a 'ps xf -u user' shortly after starting the sixth run .. the
> number of gpg zombies seems to be number of sets (full + incr's) minus 2 ..
> 
> 9122 pts/0    S+     0:00          \_ /bin/bash
> /srv/www/user//release/ftplicity_1.4.2/ftplicity test backup
> 19128 pts/0    S+     0:00              \_ /bin/bash
> /srv/www/user//release/ftplicity_1.4.2/ftplicity backup
> 19193 pts/0    S+     0:00                  \_ /bin/bash
> /srv/www/user//release/ftplicity_1.4.2/ftplicity backup
> 19194 pts/0    R+     0:03                      \_ /usr/bin/python
> /srv/www/user/_apps/duplicity-0.5.10/bin/duplicity --verbosity 4
> --encrypt-key 00000000--sign-key 00000000--gpg-options=--always-trust
> --volsize 1 --exclude-globbing-filelist /srv/www/user//.ftplicity/test/excl
> 19196 pts/0    SL+    0:00                          \_ gpg --logger-fd 4
> --passphrase-fd 8 --batch --no-tty --default-key 00000000--recipient
> 00000000--no-secmem-warning --always-trust --encrypt --sign
> 19197 pts/0    SL+    0:00                          \_ gpg --status-fd 6
> --passphrase-fd 11 --logger-fd 5 --batch --no-tty --default-key
> 00000000--no-secmem-warning --always-trust --decrypt
> 19199 pts/0    Z+     0:00                          \_ [gpg] <defunct>
> 19200 pts/0    Z+     0:00                          \_ [gpg] <defunct>
> 19201 pts/0    Z+     0:00                          \_ [gpg] <defunct>
> 19202 pts/0    Z+     0:00                          \_ [gpg] <defunct>
> 19203 pts/0    SL+    0:00                          \_ gpg --logger-fd
> 23 --passphrase-fd 28 --batch --no-tty --default-key 00000000--recipient
> 00000000--no-secmem-warning --always-trust --encrypt --sign
> 
> 
>> hmm .. you are right these processes are zombie (they appear in top as
>> such) .. later on the processes disappear, while duplicity is still
>> running ..
>> but I don't feel this is normal.. or is it?
>> just to be more sure, I'll try a 1MB chunk full backup of my 3GB home
>> dir .. let's see with how many zombies I'll end up ...
>>
>> interestingly my ulimit = unlimited ... seems to be default in the old
>> SUSE 10.2, btw also on my debian 5.0 box both 32bit
>> can't remember to ever have touched this setting
>>
>> ..ede
>> -- 
>>> These are not open processes, they are zombie processes, so no real
>>> resources taken.  That simplifies it a bit.  What normally causes this
>>> is the failure of a parent process to properly retrieve its exit status,
>>> but this is not the case or more people would have this problem.
>>>
>>> As far as I can tell, the only systems this is happening on are 64bit,
>>> and not even all of those.  Some of my systems are 64bit and they don't
>>> show this, so I'm wondering if this could be limited to a particular
>>> version of GnuPG, or what.
>>>
>>> Edgar, from your note I could not tell, is this happening to you too?
>>>
>>> ...Ken
>>>
>>> address@hidden wrote:
>>>  
>>>> just my quick observation .. a simple incr backup opens and leaves over
>>>> 20 gpg processes open. I don't think this is healthy and also can
>>>> imagine that this multiplies if the volumes get smaller (mine are
>>>> 50MB).
>>>>
>>>> regards ede
>>>>
>>>>
>>>> address@hidden:~> ps -u user xf
>>>>  PID TTY      STAT   TIME COMMAND
>>>> 32731 ?        S      0:00 sshd: address@hidden/0
>>>> 32732 pts/0    Ss     0:00  \_ -bash
>>>> 1167 pts/0    S      0:00      \_ /bin/bash /usr/local/bin/ftplicity
>>>> profile backup
>>>> 1168 pts/0    S      0:00      |   \_ /bin/bash
>>>> /srv/www//release/ftplicity_1.4.2/ftplicity profile backup
>>>> 1174 pts/0    S      0:00      |       \_ /bin/bash
>>>> /srv/www//release/ftplicity_1.4.2/ftplicity backup
>>>> 1239 pts/0    S      0:00      |           \_ /bin/bash
>>>> /srv/www//release/ftplicity_1.4.2/ftplicity backup
>>>> 1240 pts/0    S      0:05      |               \_ /usr/bin/python
>>>> /srv/www/user/_apps/duplicity-0.5.10/bin/duplicity --verbosity 4
>>>> --encrypt-key XXXXXXX --sign-key B59ECD99 --gpg-options=--always-trust
>>>> --full-if-older-than 1M --volsize 50 --exclude-globbing-filelist
>>>> /srv/www/...
>>>> 1249 pts/0    SL     0:00      |                   \_ gpg --logger-fd 4
>>>> --passphrase-fd 8 --batch --no-tty --default-key XXXXXXXX --recipient
>>>> XXXXXXXX --no-secmem-warning --always-trust --encrypt --sign
>>>> 1265 pts/0    SL     0:00      |                   \_ gpg --status-fd 6
>>>> --passphrase-fd 11 --logger-fd 5 --batch --no-tty --default-key
>>>> XXXXXXXX
>>>> --no-secmem-warning --always-trust --decrypt
>>>> 1267 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1272 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1275 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1278 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1282 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1286 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1289 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1292 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1296 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1299 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1302 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1305 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1308 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1311 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1314 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1317 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1320 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1324 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1327 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1330 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1333 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
>>>> 1334 pts/1    Ss+    0:00      |                   \_ /usr/bin/ncftp -u
>>>> ******* backup.server.de
>>>> 1335 pts/0    R+     0:00      \_ ps -u user xf
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Duplicity-talk mailing list
>>>> address@hidden
>>>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>>
>>>>     
>>>
>>>
>>>  
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Duplicity-talk mailing list
>>> address@hidden
>>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>>>   
>>
>>
>>
>> _______________________________________________
>> Duplicity-talk mailing list
>> address@hidden
>> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> 
> 
> 
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk
>

signature.asc
Description: OpenPGP digital signature