Re: [Duplicity-talk] OSError: [Errno 24] Too many open files

[Kenneth Loafman]

Kenneth Loafman wrote:
> Yang Zhang wrote:
>> Kenneth Loafman wrote:
>>> Yang Zhang wrote:
>>>> I'm running into this exception (running on Ubuntu 9.04). Anybody know
>>>> what's up with this?
>>>>
>>>> $ uname -a
>>>> Linux yang-xps410 2.6.28-11-generic #42-Ubuntu SMP Fri Apr 17 01:58:03
>>>> UTC 2009 x86_64 GNU/Linux
>>>>
>>>> $ duplicity --version
>>>> duplicity 0.5.17
>>>>
>>>> $ duplicity /home/yang/personal/
>>>> OSError: [Errno 24] Too many open files
>>> This line says it all.  I just looked and Jaunty has the default of 1024
>>> Max Open Files.  duplicity will have open up about 150 of those at any
>>> one time and each task running will have their own files open, often 50
>>> or more, so its easy to deplete the resource.
>>>
>>> The fix is easy: "sudo ulimit -n 4096" will up the count to 4096 Max
>>> Open Files.  This should solve the problem.
>>>
>>> ...Ken
>> But this limit is per-process, not a shared pool across all processes.
>> It means *duplicity* (via GPG, apparently) is opening more than 1024
>> files at a time.
>>
>> See also this closed bug:
>>
>> http://savannah.nongnu.org/bugs/?21792
> 
> It's possible there's a bug in gpg that's keeping files open on Jaunty.
>  I don't have that system to check against, so if you would, please do:
> 
> 1) run duplicity in a fairly long job, or with a small --volsize
> 2) while its running, do:
>      # lsof -n | grep ^duplicity
> and save the last output you get.  I think it will show the same problem
> as the other bugs have, lots of gpg processes not closed.  I thought I
> had this problem fixed.   If that's not the case, then zip the listing
> and post it to the bug report above.  I'll reopen it.

Maybe just a ps will do, 'ps auxf' if you would.  That will tell me what
I need to know.

Plus, run duplicity with a -vINFO option and dump the first few lines.

...Thanks,
...Ken

signature.asc
Description: OpenPGP digital signature