Re: [Sks-devel] sks-peer.spodhuis.org catching back up

[Phil Pennock]

On 2012-05-29 at 14:20 -0400, Jeffrey Johnson wrote:
> Sure there's a way to use Berkeley DB without transactional logs.
[snip]

Okay, pedantically you're correct, but since I was trying to *just*
modify sksclient and not the server, most of what's possible with other
modes of operation is irrelevant, and obviously so.  The idea was to
query existing DB files, not spend 10 hours rebuilding.

> Its also possible to open the databases RDONLY without a dbenv to access 
> information
> independently of the actual running daemon, without locks, and without
> guarantees that the client won't occasionally (and its _VERY__ occasionally
> because of the sheer number of records and accesses) if a client attempts
> a read simultaneously with a server attempting a write. (a trickle writer
> to move cached data in __db* files into the backing store might be needed
> here: its not hard coding even if difficult QA).

Ah.  Everything I tried was using a dbenv, as the context for opening
the db.  I saw nothing in the docs suggesting that opening a db designed
for use with a dbenv was possible without a dbenv.  *sigh*

> > Apparently the locking is not what it could be, as I ended up with a DB
> > that needed recovery.  Moral 1: do not run sksclient against a serving
> > DB.
> > 
> 
> There's no engineering diagnosis in your statement that indicates anything
> other than opinion with
>       … the locking is not what it could be, …

For clarity: I meant the locking in how SKS uses BDB is not what it
could be.  I see that I was ambiguous here and that it's possible to
interpret it as a general comment on BDB locking; sorry.

Yes, it was opinion, covering the state of affairs.  Since SKS is
normally going to run as the *only* user of the DB files, and we know
that even using the "sks dump" command to dump current keys requires
stopping sks, I stand by my opinion that the locking is currently not
what it could be.

> The recovery was likely needed for other reasons. Note that modern Berkeley DB
> supplies DB_RUNRECOVERY during open and its not too hard to automate the 
> recovery
> as part of daemon startup, rather than forcing manual intervention.

How does this help when the problem is a daemon started first, and then
another tool running which accesses the backing files so that the daemon
starts PANICing to the logs?

-Phil