[Pan-users] Re: Any tool/script/internal feature planned to facilitate migration to new(er) version?

[Duncan]

Chris Metzler <address@hidden> posted
address@hidden, excerpted below, on  Sun, 29
Oct 2006 01:09:30 -0500:

> Hi.  I run Debian testing, and this evening pan was upgraded to 0.113.
> Apparently the format for configuration files has changed, and no attempt
> is made to get useful data from the old-style files.  I'm wondering if
> there's any plan to make available any kind of tool to facilitate
> migrating over one's configuration info, or if any capability for doing so
> is to be added to Pan itself?  I've enjoyed using Pan for many years now;
> but as it stands I have to downgrade because my configuration is too
> elaborate to reproduce by memory and too detailed to copy over by-hand
> without a lot of annoyance.  And there's a lot of newsgroups on a lot of
> different servers for which I've got killfiles (or rather their equivalent
> in Pan), read message lists, etc. that it would truly suck to lose.

FWIW there are debs for 0.117 out.  It fixed a few bugs.  If you want to
try CVS, last I updated, it has a fix for one more bug (an i18n/l10n bug)
passed 0.117. Right now we are in a holding period with 1.0 stable likely
the next release, expected perhaps this week just starting, if nothing to
derail it is reported.

As for upgrading from 0.14.x... some stuff is the same or similar format,
but enough is different that they use different data dirs by default
(old-pan using ~/.pan, new-pan using ~/.pan2).

Your score files should transfer over, perhaps with a bit of touchup, more
on that below.  The read-message and subscribed group tracking should also
transfer over, provided you do it right.  Again, more below.  Pretty much
everything else the format has changed enough that transferring it over
must be done manually.

The two biggest changes are a DRASTIC reduction in memory usage, thus a
HUGE increase in scaling efficiency, and more automated multi-server
handling, both of which will be of most interest to those doing lots of
binaries on multiple servers. New-pan can handle groups with millions of
posts without falling over or taking forever to load. Try that in old-pan
and you'd probably need about a week and several gigs of memory for it
just to load!  In terms of server handling, pan now presents a unified
group list instead of separate lists for each server, and automatically
queues downloads using connections on all the servers (well, all set at
the same priority, you can set a backup priority if you wish, which won't
be used unless it's not on the other servers).  The biggest problem with
this approach is that the list of subscribed groups can get pretty long,
and there's no way to categorize your groups.  However, there's a
workaround for that.  Again, see below.  There are a fair amount of other
changes but they generally are the result of new multiserver handling and
how that translates into preferences and the GUI.

Scorefiles:  As mentioned, the scorefile is basically the same format as
it was.  It's still based on the venerable slrn scorefile format.  The
difference is that the old scorefile allowed a couple xnews variations on
the format, while the new one is more strictly (with one exception) slrn
format.  Specifically, xnews treated the newsgroup entries as regular
expressions, while slrn treats them as * wildcard expressions.  New-pan
requires those * wildcard expressions.  If that's what you were using
before, you should be able to simply copy the old file into the new
location, no changes at all.  If you were using regexs for the newsgroup
entries, you'll have to do a bit of manual conversion, but IMO it can
easily be worth it -- I took the opportunity to clean out and consolidate
my scorefile, simplifying it DRASTICALLY, so according to pan, it now has
only six rules in two sections -- but those are COMPOUND rules with
sometimes dozens of entries apiece.  The single exception to slrn's format
is that pan's format is normally case insensitive in both
newsgroup/wildcard and regex matching.  (I'm also not sure if pan supports
the fancy stuff like includes, and I don't believe it matches on any
header yet, only the overview headers.  However, I've not tried, and I
don't believe that's a chance from old-pan in any case.)

For reference, here's the scorefile documentation:
slrn:   http://www.slrn.org/docs/score.txt
xnews:  http://xnews.remarqs.net/scoring.txt

Again, pan now uses very little of the xnews variations.  The only
exception is it uses the default case insensitivity from xnews, with the
same way to make the regex matches case sensitive, if desired.  Thus,
you'll want to pay most attention to the slrn doc.

Again, it was well worth the time to do the manual fixing for me, as I WAS
using regex matches in the newsgroups lines, so had to do so, but I took
the opportunity while I was there (and since I had the documentation right
in front of me) to actually organize my score file, and I'm very glad I
did.

I should mention that new-pan doesn't have anything equivalent to
old-pan's rules.  From posts, most folks used them to automate four
things, auto-expiration before stuff expired on the server, auto-download
of bodies for all or only watched messages, auto-delete of ignored
messages, and auto-mark-read of negative-scored messages.   New-pan
already handles auto-expiration, and while the others haven't been
implemented yet, Charles has indicated he plans on it post-1.0, which
probably means pre-1.1 (next planned stable series after 1.0).  In the
mean time, the easiest way to handle those tasks is to display the score
column (preferences) and click on it to sort by score (unthreading if
necessary), then select the group of posts matching what you want and
delete/mark-read/download them as desired.

Subscribed group and read-message tracking:  You'll want to use old-pan's
newsrc export ability for this, setting the preferences to write the
newsrcs. You'll need to ensure one is created for each server. 
New-pan uses newsrc files directly, so once you have them created in
old-pan, all you have to do is move them into correct place in new-pan. 
Basically, what that means is that you'll want to create a server record
(in new-pan, set the address, number of connections, etc) for each server
you use, then quit pan. You'll then want to open up the servers.xml file
(in ~/.pan2, again, by default), and note the server-ids for each server
you configured. Normally it'll be the same order you configured them in. 
Once you have that information, replace the blank newsrc-X (where X
corresponds to the ID number of each server) files pan created with the
ones you exported from old-pan.  You should then be able to startup
new-pan once again and have all your subscribed groups and read message
tracking transfered over from old-pan.

With the unified group view, pan needed a way to track which of your
configured servers (that carry the group) you want to post to for each
group.  That's handled with the posting profiles.  You choose a posting
profile for each group, and the posting profile has a posting server
linked to it, so by choosing the profile, you choose not only your sig and
header set, but also the server you will be posting to when using that
profile.

Keyboard accels:  New-pan allows customized keyboard accels just as
old-pan did, either setting them in pan, if you have the appropriate
config entry in your gtkrc file to allow it, or by editing the accels.txt
dumpfile.  As with old-pan, new-pan dumps its current settings at close,
and it's anything but organized, so if you want to keep an organized
version for direct manual editing, save it as something else so as to keep
it organized, and simply copy it over the accels.txt dump file (with pan
closed of course) every time you wish to make changes.

There are a number of settings not available as choices from the GUI
(which Charles wanted to keep fairly simple) but none-the-less configurable
by directly editing the configuration.  Among them:

1) The PAN_HOME environmental variable tells pan what config/data dir to
use. If it's not set, ~/.pan2 is the the default.  Should you wish to
categorize your groups or use different settings for binaries vs text
groups, for example, this makes it very convenient to set up multiple pan
instances, each with its own settings, by simply pointing each instance at
a different PAN_HOME.  Here, for instance, I have three instances, set to
subdirs of ~/pan (I prefer the dir /not/ be hidden), bin, text, and test.
There's also a fourth subdir called global, that had the common files such
as the scorefile, which is symlinked from the three instances.  The
separate text and bin instances also allow me separate cache settings (see
below), among other things, as well as keeping the subscribed group list
manageable in each one.  This is of course the way around the problem I
mentioned above.

FWIW, I use stub-scripts to setup the environment as desired for each
instance, so for instance:

$cat ~/bin/pan.bin
#!/bin/bash
export 
GTK2_RC_FILES="/etc/gtk-2.0/gtkrc:~/.gtkrc-2.0:~/kde3.5/share/config/gtkrc-2.0"
export PAN_HOME=~/pan/bin
cd ~/pan/scraps
exec /usr/bin/pan $*

I then have separate kmenu entries for pan.bin and pan, which is my normal
text instance, and naturally have khotkeys assigned for each of them,
giving me two-key launch.  =8^)  Of course, you can make whatever
arrangements you like in your desktop environment of choice.

2) In servers.xml, the per-server expiry, connection limit, and rank, can
be set with more precision than in the GUI.  Due to GNKSA requirements, the
GUI limits connections to four per server, but for servers that allow more
(one of my pay servers allows eight), you can set the connections
accordingly here.  The GUI has only a couple of choices for expiry,
keeping things simple, but you can set an arbitrary number of days here. 
And finally, again for simplicity, the GUI has only two server ranking
levels, primary and backup, but you can configure any arbitrary number of
ranking levels desired, by editing servers.xml directly.

3) In preferences.xml, you can set cache size as desired, as the
"cache-size-megs" setting.  The default is IIRC 10MB.  I like to keep text
messages around for awhile, however, so have my text instance cache set to
5 GB.  That should hold for awhile.  =8^)  For binaries, I prefer to
download to cache, then work from there, sorting and saving files after
they are downloaded but while I still have all the info from the posts
they came in, date, who posted them, what they said in the subject line,
etc.  Thus, I have my binary instance cache dir symlinked to a dedicated
12 gig partition, with the cache size set accordingly.  I've tested
upwards of 10 gig actually in the cache, with no issues, so know it works
with a 12+ gig cache size and at least 10 gigs of posts cached, no problem.

That should get you up and running.  Any more questions, just ask.  It's a
big change, certainly, but particularly for heavy binary users that
download from multiple servers, it's a huge improvement, once you get used
to it.  If you do mostly text, it's not such a big deal, but you'll want
to switch eventually, as the old-pan development tree is now abandoned and
will fall farther and farther behind, and will eventually no longer be
shipped or compile.  I know of no one still using the old gnome-1 based
pan-0.11.x, for instance, and 0.14.x will in a few years be just as stale,
particularly as it hasn't had any serious development in over two years
already.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman