Re: [Sks-devel] Proposal: Start verifying self-signatures

[Daniel Roesler]

On Tue, May 19, 2015 at 4:04 AM, address@hidden <address@hidden> wrote:
> But there was also the question on how to deal with the situation on
> a more conceptual level and I don't know what this would entail in a
> technical sense,

Here's a proposed phased technical rollout of verifying self signatures.

NOTE: Verifying self-signatures is the only thing in scope of this
proposal. As others have mentioned, there are many other issues, but
those should be discussed in other threads.

==Phase 1 (2015)==

a) Verify self-signatures on POST /pks/add. This can be done via
openssl, gnupg, or via home-rolled signature verification function (if
we don't want to add openssl or gnupg as a dependency). This could
even be done in sub-phases where we only start verify algorithms one
at a time.

b) If someone uploads a key, the server will drop the packets that do
not have valid self-signatures.

c) Nothing changes with gossip, storage, or download. So existing
invalid self-signatures remain in the database and can still be
gossiped around and uploaded to non-updated keyservers in the pool.
This phase is mostly to start the ball rolling and introduce a bit of
verification into the codebase and making sure it works on upload. No
database or infrastructure changes are needed.

==Phase 2 (2016)==

a) Stop including packets with invalid self-signatures on keys when
clients download keys from the keyserver. This will require that
keyservers either dynamically verify self-signatures upon each
request, or that they keep track of which keys. The first option is
more compute intensive, but doesn't require database changes. The
second option means you only have to verify once, but requires
database changes. I'd recommend the second option since it will help
lay the ground work for Phase 3.

b) Invalid self signed packets are still kept in the keyserver
databases. This is so that the gossip protocol doesn't have to change
much. Hashes of the keys (including those with invalid self-sigs) will
still be the same and gossiped the same. However, I think that the
gossip protocol currently uses the normal download endpoint for
retrieving changed public keys. If that's the case, we would have to
create a special gossip-only download endpoint that would still let
the other keyservers download public keys with invalid packets still
in them so they will hash correctly.

c) This phase is mostly about starting to change the key database to
prepare for Phase 3. It does not prevent non-updated keyservers from
still adding and gossiping around invalid self-signed packets, but
this phase does prevent clients from downloading invalid packets from
updated keyservers (making it much more of a pain in the ass to
distribute invalid packets via the pool).

==Phase 3 (2017)==

a) Change the gossip protocol to only use the hash of the verified
self-signed packets in a public key.

b) This means a re-building of the database index for all keyservers.
It also means a likely updated keyservers will not be able to gossip
with non-updated keyservers. Thus, this is a non-backwards compatible
change. I'd love to hear ideas on how we could make this more
backwards compatible or a smoother transition.

c) With the adoption of the new gossip hashing system, keyservers
could begin to optionally purge invalid packets in their own
databases. This would close the hole where a troll could bloat the
size the database for keyserver operators.

==Recap==

2015 - Start dropping packets with unverified self-signatures
2016 - Stop including packets with unverified self-signatures in downloads
2017 - Stop gossiping packets with unverified self-signatures

Thoughts?
Daniel