Re: [Duplicity-talk] gpg: too many files open

[Kenneth Loafman]

A couple of answers...

You need to up the ulimit on the system, or

You need to upgrade to the latest version.  This bug has been fixed for a long time.

On Thu, Feb 14, 2019 at 4:45 PM Nate Eldredge via Duplicity-talk <address@hidden> wrote:

On Thu, 14 Feb 2019, Wolfgang Rohdewald via Duplicity-talk wrote:

> On Do, 2019-02-14 at 22:29 +0100, Wolfgang Rohdewald via Duplicity-talk wrote:
>> I am getting this on restoring. Looking at /proc I find
>> 66 gpg processes, and each of them has about 260 open pipes.
>>
>> So that opened about 17000 pipes.
>>
>> Any ideas?
>
> since I urgently needed the data, I drastically increased ulimit -n.
>
> I believe the problem is that I have a LOT of small incremental
> backups (one every work hour), 392 incrementals since last full backup.
>
> And I believe duplicity calls GnuPG.run() for each and every inc
> backup and releases all of them only when ending the process.
> I added some debug output and got:

Yeah, I think this is expected behavior.  When restoring, duplicity wants
to read through (and decrypt) all the incrementals simultaneously.  The
idea is that since the diffs to the files are stored in sorted order, by
doing this you get to apply all the diffs to get the final version of a
single file before moving on to the next.

The alternative would be to do a full restore of one incremental, and then
patch all the files to get the next one, and so on.  One disadvantage
would be that you would need enough disk space for the largest of all the
incrementals, instead of just for the final one.

The solution is as you found: if you have a very long chain of
incrementals, you'll have to arrange for a higher file limit when you
restore.

--
Nate Eldredge
address@hidden


_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk