Re: [Sks-devel] Big amount of updated keys yesterday?

[Jeff Johnson]

On Apr 13, 2011, at 8:40 AM, Jeff Johnson wrote:

I should underscore where to fix "reliability" issues,
and where to try for "performance"

> 
> Caveat:
> I spend way too much time fussing with Berkeley DB issues (in RPM) to try
> to tune SKS keyservers for absolute maximum performance. So take the above 
> values as
> nominal and "gud ebuf" rather than any serious attempt to tune for 
> performance.
> 
> hth
> 
> 73 de Jeff
> ==========================================================================
> set_lg_dir              ./log
> set_tmp_dir             ./tmp
> 
> set_flags               DB_LOG_AUTOREMOVE
> 
> # -- thread_count must be >= 8
> #set_thread_count        64
> 
> # ================ Logging
> set_lg_regionmax        1048576
> set_lg_max              10485760
> set_lg_bsize            2097152
> 
> # ================ Memory Pool
> ##set_cachesize         0 67108864 4
> set_cachesize           0 134217728 1
> set_mp_mmapsize         268435456
> 

The above 2 are the main "performance" tunables. mmapsize in
particular is quite important: if you have the memory, use it,
and mmap(2) used by Berkeley DB is fastest. I've personally never
seen too much performance gains tuning cachesize (but all depends on
the types of access, my experience is with RPM, not SKS, stores so YMMV,
and almost certainly WILL vary).

There's a pagesize tunable too, but I doubt that will help seriously
with tuning SKS because the records are too small. Changing the pagesize
also changes the locking granularity (BDB locks are per-page) so you
likely will also affect "reliability" if you change the pagesize.


> # ================ Locking
> set_lk_max_locks        4096
> set_lk_max_lockers      4096
> set_lk_max_objects      4096
> set_lk_detect           DB_LOCK_DEFAULT
> mutex_set_max           163840
> 

These are the main "reliability" tunables that MUST be increased for
an SKS keyserver, the defaults aren't anyplace close to being able to
handle a ~3Gb store like in SKS keyservers.

I typically just double the values until a problem goes away, and
scale "mutex_set_max" as ~10x the size of the other values. I
can tell from the settings above that I was (rather idly) looking at
the minimum locks needed for a SKS keyserver because the numbers don't
follow my usual heuristic of
        mutex_set_max is ~10x the value of "set_lk_max_locks"

hth

73 de jeff

smime.p7s
Description: S/MIME cryptographic signature