[Pan-users] Re: Duplicate detection

[Duncan]

Robert Krig <address@hidden> posted
address@hidden, excerpted below, on  Mon, 20 Nov 2006 04:01:50
+0100:

> In some newsgroups I sometimes get multiple copies of one and the same 
> attachment. Sometimes both postings contain intact attachments, 
> sometimes one is complete while the other is incomplete.
> 
> Pan seems to download them regardless, though. So in my download 
> directory I get a bunch of files with a *.1 ending in addition to the 
> normal filename.
> 
> Is there any way to get around this?

Not at this time, except at the BKAC (between keyboard and chair, that is,
human) level.  

Note that earlier in the new-pan series (thru maybe 0.110 or so, IDR the
exact release the bug was fixed), there was a bug that caused pan to
duplicate headers and download the same thing multiple times.  It was
related to multi-post attachments of the form (x/y), where pan was taking
the first x it saw as the subject line, so if headers were redownloaded
again and another x happened to appear first, it would look to pan like a
different multi-post attachment set and pan would download it again.  The
bug was fixed by omitting the x, so now those subjects appear as (/y)
(where y is the number of individual posts that form the whole).  However,
existing headers remained, so unless you deleted them manually, they'd
continue to exist and create problems until they expired and were purged
via pan's normal mechanism.  If you are still using an affected version or
just upgraded and still have duplicate headers, you'll continue to see
that problem.  That was long enough ago (IIRC) tho that if you have pan
configured for a 60-day header purge and upgraded past the buggy versions
shortly after the fix was released, it should be pretty much worked out of
the system by now.

So for awhile pan was making the duplicate problem much worse of its own
accord.  It shouldn't be doing so any more, and if you've been keeping up
and have a 60-day or less header purge set, that problem should be about
gone, but you'll still have the group's naturally occurring dup posting to
worry about.

One easy way to cope with the issue is to check if the main file (without
the *.n attached) is complete (if desired), then simply go thru and delete
all the *.n files.  In bash:

rm *.[1-9]

or some slightly stricter variation (if you are worried about a runaway
delete), like

rm *.mpeg.[1-9]

should work fairly well.  If in doubt, try it with the -i (interactive),
making the remove prompt for each one, just to be safe, as in

rm -i *.[1-9]

Note that some distributions alias rm to rm -i by default, so you always
get the -i behavior unless you deliberately turn it off with -f (force). 
Of course, that carries its own risks, particularly when using the -r
(recursive) switch to delete recursively into subdirectories as well. 
Many a *ix user has learned that the hard way. =8^(

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman