Re: [Sks-devel] SKS RAM usage gone haywire

[Ari Trachtenberg]

Phil Pennock wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: RIPEMD160
> I restarted recon to pick up a couple of admin changes (ulimits, reopen
> fds 0,1,2 on /dev/null so they're not revoked in the process) and got to
> see the memory and RSS usage shoot up.  CPU was pegging out.  When it
> finished, the memory/RSS stayed high but the CPU dropped back down to 0.
>
> This maxed out at 811MB memory, so is much lower than before.
>
> While this was happening, I hit the recon process with lsof and saw that
> it was connected to [217.66.26.140], which is bazon.ru.  This is the
> peer who only has a few tens of thousands of keys.

This could cause a problem.  Remember that the memory scales with delta 
... so if delta is high, memory usage will be as well.  This is usually
not a problem for hard disk usage, but it could be a problem for RAM.

The simplest solution would be to switch modes when the number of 
differences is astronomical and simply do a wholesale transfer of data.

Best,
        -Ari