Re: [Duplicity-talk] OSError: [Errno 24] Too many open files

[edgar . soldin]

i seem to get closer to it ...
the more incrementals (regardless if file: or ftp: backend) the more 
defunct gpg processes appear .. but dissappear again while the backup is 
still running

I did
initial backup of 3GB
touched a new test file to have a change & did a 2nd backup
touched a new test2 file to have a change & did a 3nd backup
...

and since the 3rd backup the defunct gpg processes keep appearing

essentially I say: Take any more than 3 incrementals in size backup and 
do a new incremental and you will see those zombie gpg processes ..

while I am not sure that this is what happens on the ulimit problem I 
thought I should report these observations ... ede

PS:

here a 'ps xf -u user' shortly after starting the sixth run .. the 
number of gpg zombies seems to be number of sets (full + incr's) minus 2 ..

9122 pts/0    S+     0:00          \_ /bin/bash 
/srv/www/user//release/ftplicity_1.4.2/ftplicity test backup
19128 pts/0    S+     0:00              \_ /bin/bash 
/srv/www/user//release/ftplicity_1.4.2/ftplicity backup
19193 pts/0    S+     0:00                  \_ /bin/bash 
/srv/www/user//release/ftplicity_1.4.2/ftplicity backup
19194 pts/0    R+     0:03                      \_ /usr/bin/python 
/srv/www/user/_apps/duplicity-0.5.10/bin/duplicity --verbosity 4 
--encrypt-key 00000000--sign-key 00000000--gpg-options=--always-trust 
--volsize 1 --exclude-globbing-filelist /srv/www/user//.ftplicity/test/excl
19196 pts/0    SL+    0:00                          \_ gpg --logger-fd 4 
--passphrase-fd 8 --batch --no-tty --default-key 00000000--recipient 
00000000--no-secmem-warning --always-trust --encrypt --sign
19197 pts/0    SL+    0:00                          \_ gpg --status-fd 6 
--passphrase-fd 11 --logger-fd 5 --batch --no-tty --default-key 
00000000--no-secmem-warning --always-trust --decrypt
19199 pts/0    Z+     0:00                          \_ [gpg] <defunct>
19200 pts/0    Z+     0:00                          \_ [gpg] <defunct>
19201 pts/0    Z+     0:00                          \_ [gpg] <defunct>
19202 pts/0    Z+     0:00                          \_ [gpg] <defunct>
19203 pts/0    SL+    0:00                          \_ gpg --logger-fd 
23 --passphrase-fd 28 --batch --no-tty --default-key 00000000--recipient 
00000000--no-secmem-warning --always-trust --encrypt --sign


> hmm .. you are right these processes are zombie (they appear in top as 
> such) .. later on the processes disappear, while duplicity is still 
> running ..
> but I don't feel this is normal.. or is it?
> just to be more sure, I'll try a 1MB chunk full backup of my 3GB home 
> dir .. let's see with how many zombies I'll end up ...
>
> interestingly my ulimit = unlimited ... seems to be default in the old 
> SUSE 10.2, btw also on my debian 5.0 box both 32bit
> can't remember to ever have touched this setting
>
> ..ede
> --
> > These are not open processes, they are zombie processes, so no real
> > resources taken.  That simplifies it a bit.  What normally causes this
> > is the failure of a parent process to properly retrieve its exit status,
> > but this is not the case or more people would have this problem.
> >
> > As far as I can tell, the only systems this is happening on are 64bit,
> > and not even all of those.  Some of my systems are 64bit and they don't
> > show this, so I'm wondering if this could be limited to a particular
> > version of GnuPG, or what.
> >
> > Edgar, from your note I could not tell, is this happening to you too?
> >
> > ...Ken
> >
> > address@hidden wrote:
> >  
> > > just my quick observation .. a simple incr backup opens and leaves over
> > > 20 gpg processes open. I don't think this is healthy and also can
> > > imagine that this multiplies if the volumes get smaller (mine are 
> > > 50MB).
> > >
> > > regards ede
> > >
> > >
> > > address@hidden:~> ps -u user xf
> > >  PID TTY      STAT   TIME COMMAND
> > > 32731 ?        S      0:00 sshd: address@hidden/0
> > > 32732 pts/0    Ss     0:00  \_ -bash
> > > 1167 pts/0    S      0:00      \_ /bin/bash /usr/local/bin/ftplicity
> > > profile backup
> > > 1168 pts/0    S      0:00      |   \_ /bin/bash
> > > /srv/www//release/ftplicity_1.4.2/ftplicity profile backup
> > > 1174 pts/0    S      0:00      |       \_ /bin/bash
> > > /srv/www//release/ftplicity_1.4.2/ftplicity backup
> > > 1239 pts/0    S      0:00      |           \_ /bin/bash
> > > /srv/www//release/ftplicity_1.4.2/ftplicity backup
> > > 1240 pts/0    S      0:05      |               \_ /usr/bin/python
> > > /srv/www/user/_apps/duplicity-0.5.10/bin/duplicity --verbosity 4
> > > --encrypt-key XXXXXXX --sign-key B59ECD99 --gpg-options=--always-trust
> > > --full-if-older-than 1M --volsize 50 --exclude-globbing-filelist
> > > /srv/www/...
> > > 1249 pts/0    SL     0:00      |                   \_ gpg --logger-fd 4
> > > --passphrase-fd 8 --batch --no-tty --default-key XXXXXXXX --recipient
> > > XXXXXXXX --no-secmem-warning --always-trust --encrypt --sign
> > > 1265 pts/0    SL     0:00      |                   \_ gpg --status-fd 6
> > > --passphrase-fd 11 --logger-fd 5 --batch --no-tty --default-key 
> > > XXXXXXXX
> > > --no-secmem-warning --always-trust --decrypt
> > > 1267 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1272 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1275 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1278 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1282 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1286 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1289 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1292 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1296 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1299 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1302 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1305 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1308 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1311 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1314 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1317 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1320 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1324 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1327 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1330 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1333 pts/0    Z      0:00      |                   \_ [gpg] <defunct>
> > > 1334 pts/1    Ss+    0:00      |                   \_ /usr/bin/ncftp -u
> > > ******* backup.server.de
> > > 1335 pts/0    R+     0:00      \_ ps -u user xf
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Duplicity-talk mailing list
> > > address@hidden
> > > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> > >
> > >     
> >
> >
> >   
> > ------------------------------------------------------------------------
> >
> > _______________________________________________
> > Duplicity-talk mailing list
> > address@hidden
> > http://lists.nongnu.org/mailman/listinfo/duplicity-talk
> >   
>
>
>
> _______________________________________________
> Duplicity-talk mailing list
> address@hidden
> http://lists.nongnu.org/mailman/listinfo/duplicity-talk