|
From: | Jeff Johnson |
Subject: | Re: [Sks-devel] sks-keyserver Fedora Linux |
Date: | Mon, 25 Apr 2011 09:00:35 -0400 |
On Apr 25, 2011, at 8:29 AM, Robert Hinson wrote:
You need to adjust the -n and -cache parameters. This is what I have done to get past "crap out": The three operations performed while initializing an SKS server from a dump are in /usr/bin/sks_build.sh: echo === Running fastbuild... === if ! sks fastbuild -n 10 -cache 100; then fail; fi echo === Cleaning key database... === if ! sks cleandb; then fail; fi echo === Building ptree database... === if ! sks pbuild -cache 20 -ptree_cache 70; then fail; fi Run each of those operations individually and track progress by putting a tail on the logs produced for each operation. The resource intensive operations are "fastbuild" and "pbuild" and each has slightly different parameter sets and tweaks needed. I'd tell you exactly what parameters I've used but I have forgotten (and I've done different things on different platforms) Most definitely adding more memory did not solve the issues (for me). Then double (or scale by 10x initially) the -n and -cache parameters and repeat, monitoring the progress using top and paying particular attention to the VIRT/RSS memory usage. At "crap out" the process progress ceases (no CPU cycles used) and you can tell how close you are/were to success by examining the statistics in the logs. Another path to an initial load is to copy the database from an existing running SKS server. This is a bit tricky because of Berkeley DB, the very obscure command that is needed for success is db_load -r lsn ... on each of the tables (Berkeley DB calls these "databases" but they are tables): These are the "tables" in KDB/* run "db_load -r lsn" on each on a working SKS server: # ls -al total 2778072 drwx------ 3 jbj jbj 4096 Apr 24 20:17 . drwxr-xr-x 5 jbj root 4096 Apr 24 20:29 .. ... -rw------- 1 jbj jbj 1504172032 Apr 24 20:09 key -rw------- 1 jbj jbj 143142912 Apr 24 20:09 keyid ... -rw------- 1 jbj jbj 8192 Oct 1 2010 meta -rw------- 1 jbj jbj 134213632 Apr 24 20:09 subkeyid -rw------- 1 jbj jbj 118718464 Apr 24 20:09 time -rw------- 1 jbj jbj 77824 Mar 20 18:42 tqueue -rw------- 1 jbj jbj 758652928 Apr 24 20:09 word The "db_load -r lsn" removes "log sequence numbers" so that the tables are portable. The tables index into the dump files so you also need to move all the dump/* files. If you are prepared to download multiple Gb's of information, I could do these operations and hand you an "image" that you could jumpstart a running SKS instance. Almost certainly it would take a few attempts, but I have succeeded in setting up a SKS server from an image when I get annoyed at diddling -n and -cache parameters at 3am. There's a third path to "works" is loading a truncated dump. There are approximately 200 files (depending on chosen per-dump-file-size) used in initializing a SKS server. Removing some of those files is another way to achieve a load in spite of difficulties with "crap out" related (apparently, I do not know) to the amount of memory resources available. Please note that removing files from the dump in order to achieve an initial load WILL lead to a very large update (I've seen >24 hours needed to "catch up") when you start peering. Note also that gossip protocols like in SKS key servers are NOT designed for bulk transport aand are hugely inefficient. Short answer: A partial load achieved by removing files from the dump is nothing more than an expedient hack. I do have private SKS key servers in VM's that could be used if you are forced to the "expediant hack" route in order to get a SKS keyserver initially loaded. hth 73 de Jeff |
smime.p7s
Description: S/MIME cryptographic signature
[Prev in Thread] | Current Thread | [Next in Thread] |