listhelper spamassassin database munged

[Bob Proulx]

Bob Proulx wrote:
> Doing more investigation of things since the last message I think the
> SpamAssassin Bayes database has become munged.  I posted a question to
> the SA mailing list with more details but basically the nspam number
> was only 1k not 60k and sizes don't match.  Something bad happened
> causing it to be broken.  Probably the root cause of the recent false
> negatives.
> 
> I am continuing to investigate this.  My next plan of action as soon
> as I can squeeze the time in is to pull older snapshots from backup
> and see what it was and when it went bad.  I will probably leave it
> with an older version and see if it happens again soon to see if we
> are rolling over some counter or some such.  We get enough mail
> processed through it that I could see some bug in a rollover as
> possible.  But it could have also been some file lock problem too.

I did this and found what looks to be interesting data.  I won't bore
you with the raw data but here are the boiled down numbers.

  01/15: nspam:25735 nham:47082 ntokens:118870 bayes_seen-size:43421696
  01/22: nspam:26480 nham:50182 ntokens:123870 bayes_seen-size:43663360
  01/29: nspam:276 nham:53159 ntokens:145782 bayes_seen-size:77619200

Clearly something blew up between 1/22 and 1/29.  Looks good.  Looks
good.  Looks bad.

I will restore the backup from 01/22 and then watch it closely.  If it
blows up again then I suspect some rollover problem in the db that
should be reported upstream.  If not then I suspect some transient
file lock glitch problem of some sort munged the database.

Bob