[Pan-users] Re: Keeping older headers around longer

[Duncan]

jdonner posted <address@hidden>, excerpted below,  on Sun,
18 Apr 2004 00:43:52 -0600:

> Hi,  I'm now using Pan 0.14.2 but used to use 0.14.0, and,
> now it seems headers get cleaned much more aggressively; ie
> they disappear after only a few days.  I'd like to see all
> from a month or more maybe, and in the older version it
> worked that way.
> 
> My cache is 60MB, I can't believe I've got that filled,
> I only read maybe 15 newsgroups.  What setting can I change
> to keep headers around longer?

The cache is for messages, not message overviews, which I don't believe
are size-regulated.  (Overviews are often incorrectly called headers.  It
would seem PAN continues this unfortunate trend, likely because OE and
similar news clients use it.  "Headers" include a number of headers only
found in the message itself, not the "overviews available when one pulls
the list of messages off the server.  Compare the headers displayed when
show all headers is on, to those displayed in the "header" pane, and you
see the limited set of "headers" that are actually included in "overviews".)

PAN, as many news clients, keeps overviews in sync with those available
from the news server.  Unless you have a rule deleting old overviews
sooner (and you should know it if you do), old ones are removed as soon as
one syncs to the news server.

There are several ways around this.  The one I use on some groups is not
syncing every day (I have the option to sync on entering group turned off
in preferences).  I get overviews, delete the spam and other stuff I'm not
interested in, download the messages for the rest, and then process it
over several days, not downloading again until I finish processing the
messages I have.  That keeps the old ones from expiring until I get them
processed.  Fortunately, my server has long enough retention I usually get
back to d/l the next batch before any I haven't yet seen start expiring
off the server.

Another, if your news provider doesn't offer long enough retention, is to
get a different news provider.  Unfortunately, many ISPs, unless they
outsource news to a third party news provider, have very poor retention.
Supernews is known to have VERY long retention (6 months or more) on text
groups, but only middling retention (5-ish days) on single-part binary
groups (most of the jpeg pix groups), and poor retention (2-ish days) on
multi-part binary groups.  They are a good alternative, particularly with
their text group only subscription, for text group users.  Other dedicated
news providers generally target a minimum of a month retention on the text
groups, with many offering 90-day, and 5-14 days on the binary groups. 
Those that separate multi-part from single-part binaries can offer up to
30 days on the single-part binary groups, at the expense of the multi-part
groups.  Thus, it's up to you how you go.

A third option is to move the overviews/posts you wish to keep out of the
newsgroup in question, to a local PAN folder.  Be aware that if your cache
fills up, the messages may be lost, leaving only overviews that you can't
get the messages for.  At least, back when I tried that option, I lost
some messages when the cache filled up, tho I'm not sure local folders
are still part of the cache. (Of course, if the overviews WERE kept in the
groups, you'd STILL not be able to get the messages, once they
expired off the server, one reason it simply makes sense to keep them in
sync as PAN does.)

A forth option is to run your own news server, INN or the like, setting
its retention policies as you desire, and then run PAN with a tiny cache
since it'd be getting messages locally.  This might be the best low-cost
option, as it would allow you to set your own server retention policies
entirely independent of those of your provider.  As well, if you were
keeping the messages in PAN's cache anyway, it'd not take up much
additional space, and by only serving the few groups you are interested
in, particularly if they are all text groups, you could keep a LONG
retention, even YEARS worth (text groups only), while only using a
moderate amount of hard drive space, in modern terms, anyway.  Consider
how long a retention you could get with a 250 gig hard drive spool,
especially if set up using ReiserFS or something similarly efficient with
small files.

-- 
Duncan - List replies preferred.   No HTML msgs.
"They that can give up essential liberty to obtain a little
temporary safety, deserve neither liberty nor safety." --
Benjamin Franklin