Re: [Sks-devel] ams.sks.heypete.com migrating to new facility, changing IP addresses

[Pete Stephenson]

On 9/11/2014 4:10 PM, Pete Stephenson wrote:
[snip
> Later today I will be migrating ams.sks.heypete.com to this new
> facility. This will entail a period of downtime. When it comes back
> online, the system will have new IPv4 and IPv6 addresses.
> 
> After things are setup, tested, and working, I will update the DNS
> records pointing to the server. If I understand things correctly, SKS
> should automatically reload the membership file and re-query DNS names
> of peers every 6 hours or so, so peering should be reestablished
> automatically.

Hi all,

ams.sks.heypete.com is back online in the new facility in Amsterdam and
has native (rather than tunneled) IPv6 connectivity.

The migration has gone off without any problems. There was only about 10
minutes of downtime while the database was rsynced between the two
servers: apologies to those who tried connecting during that time period.

After changing the DNS to point at the new server, I left the old server
running for several hours until it was clear that all peers were syncing
with the new server, the pool crawler had removed the server from the
pool.sks-keyservers.net zone, and there was no client traffic on ports
11371/80/443 for at least one hour.

Those conditions were met about 20 minutes ago, and the old server has
been turned off and its VPS container destroyed.

For those who are curious, the process went like this:
1. Turn off the old server. Take a snapshot of the VPS container. Turn
the old server back on. (This resulted in a few minutes of downtime.)

2. Move the snapshot to the new facility. (Inter-facility transfers of
snapshots is something that my host, DigitalOcean, offers.)

3. Create the new server using the snapshot. Make appropriate changes to
network configuration, hostname in SKS, etc. As this was a snapshot of a
working server, only minimal changes were required.

4. (optional) Stop the SKS process on both the old and new servers.
rsync the /var/lib/sks/ directory from the old server to the new one,
set permissions correctly, and re-start the SKS process on both servers.
This is optional, but useful in my case because step #2 took several
hours longer than expected due to a large number of people transferring
their snapshots to the new facility.

5. Change the DNS for ams.sks.heypete.com to point at the new server.

6. Add the new server to the SKS membership file on the old server.

7. Add an entry in the /etc/hosts file on the new server pointing to the
old server using a non-publicly-resolvable DNS name. Add that DNS name
to the SKS membership file on the new server.

This way, the old and new servers can stay in sync (so users connecting
to the old server still get fresh data) but the pool crawler can't find
the old server anymore and will then remove the old server from the pool.

8. Wait several hours. Peers will eventually re-resolve the DNS name and
connect to the new server. The old server is no longer in the pool, but
it will take several hours for the pool DNS changes to propagate and for
users to stop connecting to the old server.
9. When there's no SKS traffic to the old server, turn it off and enjoy
a cold beer.

Cheers!
-Pete

signature.asc
Description: OpenPGP digital signature