Re: [Pan-users] Feature request: additional download option

[Duncan]

On Fri 14 Feb 2003 10:56, Jim Reiss posted as excerpted below:
> Haven't generated any interest with my past feature requests, but
> undaunted I present my latest suggestion: a new download option to
> select a range of recent headers instead of just the XXXXX most recent.
> For example, instead of the 50000 most recent, download the first 10000
> that you would get from a "50000" most recent.  This might look like
> 50000-40000 as a range, or it might be implemented as "sample 10000
> headers starting 50000 back".
>
> What is this good for?  My ISP's news server (giganews) has a fairly
> small download limit, so I only use it when I can't fill missing posts
> elsewhere.  With their retention many groups may have hundreds of
> thousands of articles, and I usually only want the past couple of days
> worth.  So I grab, say, 50000 most recent.  Then I find out I needed the
> 70000 most recent.  So I ask for that and then abort the download when
> it looks like I've filled in the headers I need.  A new download option
> would be more precise and allow downloading smaller samples to get a
> feel for where in those thousands of headers is the group I'm looking
> for, so maybe I'd only need to sample 5000 headers.

I like this idea, and could have used it myself a number of times.  However, 
perhaps I'm mistaken, but I thought that unless you deleted the current batch 
of overviews, once it hit the ones already downloaded, it stopped, or at 
least skipped over the ones already d/led. such that if you had started to 
d/l 50K, and had aborted after 20K (@ the 30K mark), then set it to d/l from 
70K, it would d/l 70K-50K, and 30K-0, skipping the overview sequence numbers 
it already had.

Still, the ranging would make life much less difficult, in your example above, 
as you could d/l say 50K-49,900, see which side of those 100 overviews it 
was, adjust accordingly, d/l another 100, etc, likely only having to pull 500 
or so overviews, b4 you found the batch you needed.

Perhaps far more helpful, however, would be what I think I've seen referred to 
as the X-over function.  (I may have the name of the function wrong, but 
there is such a function as described here, correctly named here or not.)  
This, as I understand it, allows server side searching of overviews to 
retrieve specifically the ones requested.  Retrieval by Msg-ID is one useful 
possibility, then, such that the message-IDs one finds in the attribution 
lines could then be made clickable to retrieve the full message, if full 
context is needed.  (The same could be done with the references header line, 
for even MORE usefulness.)  Searching by subject, date, or author, is also 
possible, and if the server has configured for it, even such things as 
organization, posting host, abuse-to, etc, that don't normally come up in 
overviews.

The bad think about this is that because it DOES require server-side searching 
and therefore CPU and memory resources, many news providers turn this feature 
off, or limit it to commonly used headers like date and msg-id, while not 
allowing fetching by say, posting-host, useful for tracking an individual 
user's posting history (even if they are changing from lines in an attempt to 
avoid filters, as they can unfortunately do quite easily with PAN's author 
and subject only filters at present).

-- 
Duncan
"They that can give up essential liberty to obtain a little
temporary safety, deserve neither liberty nor safety." --
Benjamin Franklin