[Sks-devel] simple DoS against SKS's HKP interface :/

[Daniel Kahn Gillmor]

Hey SKS folks--

It appears that SKS 1.1.1's hkp interface is vulnerable to an ugly DoS
attack by a client holding open a network connection without completing
an HTTP request.

Demonstration
-------------

This is pretty easy to demonstrate using two terminal windows: use
netcat in one to connect to SKS on the hkp port, and start a request but
leave it open.  In the other window, use curl or wget to fetch a
standard hkp request from the same server
(http://$keyserver:11371/pks/lookup?op=get&search=0x$keyid).  The curl
or wget will block until the netcat connection is closed.  (obviously,
you should run this test against single keyservers, not pools)

I've only tested this against the 1.1.1 version of SKS in debian stable
(i don't believe it's a problem due the debian packaging).  I haven't
tested it against more recent versions of SKS.

Fix?
----

I'm afraid i don't know ocaml at all, so i don't have a proposed fix.
It seems to be related to the event loop model on the sks db process,
though.  Looking at it from a system call level: either sks should be
multi-threaded, or reads from network sockets should be non-blocking,
and bundled into an aggregate select() statement so that concurrent
requests can be properly interleaved.

Accidental DoS
--------------

Note that a bandwidth-constrained client is capable of executing this
DoS attack inadvertently.  If its TCP session to the SKS server is
throttled into retransmission, and it doesn't transmit all of its HTTP
header within a single packet, then it can effectively block the server
From responding to other requests until the client's bandwidth frees up
(or the server decides to terminate the TCP session at the TCP timeout).

In investigating the issue, i believe i've seen exactly this accidental
server DoS from a bandwidth-constrained client in the wild (gpgkeys_hkp
when built against curl appears to send a single HTTP HKP request across
multiple TCP packets) :(

Workaround
----------

My current workaround for keys.mayfirst.org is to limit sks's db process
to only listen on the loopback, and then run a multi-request-capable
HTTP reverse proxy frontend on the public interface.  (i'm using nginx)

This makes it so that the sks instance communicates only with the
well-behaved reverse proxy, which buffers requests from the outside
world before handing them off in full to sks.  This shields sks from any
vagaries of network delay, retransmission, or malicious clients.

Recommendation for SKS operators
--------------------------------

I recommend that anyone operating a public SKS 1.1.1 instance use an
HTTP reverse proxy for their db process as a workaround.  Ensure that
/etc/sks/sksconf contains:

  hkp_address: 127.0.0.1 

I'm happy to share additional configuration advice if folks have
questions, and i'd also be happy to help evaluate and test any
alternative workarounds.

Recommendation for pool operators
---------------------------------

Any round-robin pools that have "high-availability" among their goals
should probably limit themselves to SKS installations using a similar
HTTP reverse proxy.

A simple test might be to retrieve the Server: header from the HTTP
response to a GET http://$keyserver/ -- if Server: contains sks_www,
it's probably not a reverse proxy.  keys.mayfirst.org shows Server:
nginx now.  Any other suggested mechanisms to detect this?

---

I'd be very happy to learn that i'm wrong about all of this, or that my
sks instance is somehow misconfigured.  Please let me know if that's the
case!

Regards,

     --dkg

pgpCVpXDGTIt7.pgp
Description: PGP signature