Re: [Sks-devel] SKS RAM usage gone haywire

[Phil Pennock]

On 2009-02-14 at 16:50 -0500, Daniel Kahn Gillmor wrote:
> Yikes!  Thanks for pointing that out, because you made me check up on a
>  keyserver i'm responsible for.  It looks like zimmermann.mayfirst.org
> is doing the same thing.  the sks recon process in particular now has an
> RSS of 3.3g.

:-(

So, that's two servers doing the same thing at the same time, so is
there causation to the correlation?

Is anyone else seeing this?  And running with enough RAM in the box that
you can afford to investigate?

So, the options behind it, if the correlation is more than coincidence,
seem to be:
 1 bad SKS update
 2 bad query hitting pool.sks-keyservers.net
 3 someone hitting the servers individually deliberately
 4 someone doing full sync of a keyserver without an initial DB behind it
 5 sudden really *really* heavy query load

I don't have enough experience with SKS to know how badly case 4 would
hit other keyservers.

I'd check the www.sks-keyservers.net website for signs of that, but the
DNS servers at ns1.kfwebs.net and ns2.kfwebs.net have gone lame -- one
is down, the other returning SERVFAIL.  So, there's no DNS for
sks-keyservers.net.  Thus the explicit CC to Kristian.

This means pool.sks-keyservers.net is also currently a dead name.

My suspicion is aroused by this entry just before the restart:

2009-02-14 03:14:07 Page not found: /var/sks/web/robots.txt

A misbehaving web-spider crawling the entire PGP data-set following
links across signing uids, despite the ? URI query and the non-standard
port, would be ... bad.

I'm about to deploy a robots.txt in my webroots for both port 80 and
11371.  Something like:

  User-agent: *
  Disallow: /pks/

should do it.

-Phil

pgpzecWAnf8ST.pgp
Description: PGP signature