Re: [Sks-devel] Issue Importing Database Dump (Dents in my forehead)

[Brian D Heaton]

John,

Short version:  Made the suggested changes and I'm currently bringing in
keys via "build."  Fingers crossed.

Long version:
I went back to the 1.1.3 release I tried earlier and implemented your
other suggestions.  I copied my sksconf from /etc/sks/sksconf into the
working directory (/var/lib/sks).  I also added the options you
suggested.  I went with "-n 6" to be a bit more conservative.  This box
is also a small mailserver.  After seeing some swapping when the
spam/virus scanners kicked off, I restarted the process with "-n 4".

The DB_CONFIG file I was using was an amalgamation of the sample in the
source distro and some information I found in the sks-devel archives.  I
missed that I had the same value twice (set_mp_mmapsize).  Just shows
the evil that can happen when you've been up too long.  I've moved it
aside for the moment.  I will revisit that part later.

Thanks for the guidance.  I will let this run and post again with
success or failure.  Once it completes, would you suggest adding the
DB_CONFIG from the 1.1.3 "sampleConfig" directory to "KDB" before doing
the cleandb and PTree builds?

THX/BDH



On 6/16/2012 10:28 AM, John Clizbe wrote:
> Brian D Heaton wrote:
>> Howdy,  I'm Brian and I'm trying to get a keyserver running on a
>> Rackspace hosted cloud server.  Reading the archives about brokenness in
>> the Ubuntu packages, I've compiled both BDB and SKS from source. 
>> Attempting the full-build, the system deadlocks after reading in the 3rd
>> of 4th dump file.  I'm able to build the DB with a single dump file
>> (testing).  If I then manually do the cleandb and pbuild, those steps
>> succeed.    I then attempt to do a merge one at a time with the
>> remaining dump files, by the 3rd or 4th file, I get a mutex error.  I've
>> provided a lot of information below based on my reading of the archives
>> and the questions usually asked when someone posts an issue.
>>
>> I'm sure I'm missing a simple option somewhere.  I'm open to hints,
>> whacks over the head, or even simple solutions. :)
>>
>> THX-Brian
>> ---------------------------------------------------------
>>
>> OS: Ubuntu 12.04 (LTS)
>> ARCH:  X86-AMD64
>> RAM: 1GB
>> BDB Version: 5.3.21
>> SKS Version: 1.1.3 and 1.1.4 (cloned repository)
> ¡Hola, Brian!
>
> I'd suggest going with 1.1.3 until you get a working install. If you must
> clone a dev repo, stick with Yaron's (trunk), it's saner than Kristian's or
> mine. We both (usually) test things before committing and pushing, but I
> wouldn't recommend the dev repos until you're up and running. Another VERY
> import thing you should remember, the SKS source currently is not LSB FHS
> aware. Everything is expected in one directory, typically /var/sks.
>> BDB Detail:
>> -- Configured with with pthread and posixmutex.  I've also built it
>> without posixmutex with no change.
>> -- Confirmed that sks is linked against libdb-5.3.so via ldd.
>> -- Confirmed that only libraries and headers for v5.3 are present.
> I maintain SlackBuilds for everything past 4.4, For db53 I used
> ../dist/configure \
>   --enable-shared --enable-cxx --enable-compat185 --enable-sql \
>   --enable-sql_compat --enable-amalgamation --enable-sql_codegen
>
> Solaris' OpenCSW freeware project uses
>   --enable-compat185 --enable-o_direct --enable-cxx --enable-java
>
> MacPorts defaults to --enable-cxx with java, sql & tcl as options
>
> The Mac Fink project uses:
>   --disable-static --enable-compat185 --enable-dump185 --enable-cxx \
>   --enable-stl --enable-sql --enable-sql_codegen
>
> Hint: Check what Ubuntu use[d/s] and stick as close to that as you can.
>
>> Output of initial build:
>> ----
>> address@hidden:/var/lib/sks# sks build /var/lib/sks/dump/sks-dump-0001.pgp
>> -debug -debuglevel 10 -stdoutlog -cache 100
> For 1GB of RAM, you should be safe with -n 7 with build. When I built on 1GB,
> I ran into swapping with n >= 8.
>
>> 2012-06-16 04:42:35 Opening KeyDB database
>> Loading keys...done
>> DB time:  0.03 min.  Total time: 0.04 min.
>> ---
>>
>> Output of PTree Build:
>> ---
>> address@hidden:/var/lib/sks# sks pbuild -cache 20 -ptree_cache 70
>> -diskptree -debug -debuglevel 10 -stdoutlog
>> 2012-06-16 04:45:21 Opening PTree database
>> 2012-06-16 04:45:22 Opening dbs...
>> 2012-06-16 04:45:22 Opening KeyDB database
>> 2012-06-16 04:45:22 5000 hashes processed
>> 2012-06-16 04:45:22 10000 hashes processed
>> 2012-06-16 04:45:23 15000 hashes processed
>> 2012-06-16 04:45:23 Cleaning Tree.
>> ---
>>
>> Failing output of merge:
>> --
>> sks merge /var/lib/sks/dump/sks-dump-0002.pgp -debug -debuglevel 10
>> -stdoutlog -cache 100
>> ...
>> ....
>> 2012-06-16 04:48:35 Adding hash 03C0D889F24E3D48DB6B0572504947F3
>> 2012-06-16 04:48:36 Fatal database error: Bdb.DBError("BDB2034 unable to
>> allocate memory for mutex; resize mutex region")
>> 2012-06-16 04:48:36 closing database...
> Never been a fan of merge myself. I think there are other better ways around
> this than merging in a single file at a time (read on). But this error leads
> me to suggest db_recover, first without -e, then a second time with -e to
> recreate the DB environment space AFTER putting in place a sane DB_CONFIG.
>
>   db53_recover  -h /var/sks/KDB
>   db53_recover -eh /var/sks/KDB
>
> That's how regions get resized. (or just run the first and then sks will
> recreate the regions)
>
>> DB_CONFIG file present in both KDB and PTree directories:
>> Note:  Yes, I got frustrated and cranked up the mutex_set_max.  Looking
>> at "dbstat -x" it doesn't appear mutexes are being released between
>> merge runs.
>> ----
>> set_mp_mmapsize         268435456
>> set_cachesize    0      134217728 1
>> set_flags               DB_LOG_AUTOREMOVE
>> set_lg_regionmax        1048576
>> set_lg_max              10485760
>> set_lg_bsize            2097152
>> set_lk_detect           DB_LOCK_DEFAULT
>> set_tmp_dir             /tmp
>> set_lock_timeout        1000
>> set_txn_timeout         1000
>> mutex_set_max           655360000
>> set_mp_mmapsize         16777216
> You have two values for set_mp_mmapsize. A rule of thumb is ~ 1/4 avail
> memory. mmap can speed performance on reads. Pick one size, OK?
>
> I've never exceeded an inuse mutex count of ~42k, you shouldn't need that high
> of a number either. Tunables in DB_Config won't help you on build, the BDB env
> isn't created until 'sks clean'. Likewise the environment isn't created in
> PTree until it's used after 'sks pbuild'.
>
> I /have/ seen things hang while building the DB, but it has always been in
> pbuild for me, never build. The reason it's happened for me is that Berkeley
> DB takes out a mutex lock /per page/. Small pagesize ==> more locks required.
> Thus a Catch-22, since you can't have a directory containing DB_CONFIG in
> place before you create the DB.
>
> Do you have sksconf present in the same directory you are in when running 'sks
> build' (and clean, etc...)? Do you set pagesize and ptree_pagesize?
> If not, I would recommend setting the following two in sksbuild:
>
>   pagesize:       128
>   ptree_pagesize: 16
>
> They'll both do you more good when building than mucking about with DB_CONFIG.
>
> Good luck,
>
> -John
>