[Pan-users] Re: Can creating task list be optimized?

[Duncan]

Darren <address@hidden>
posted address@hidden, excerpted below, on  Sat,
26 Aug 2006 20:31:22 -0400:

> Duncan wrote:
>> Try this with 0.109.
>> 
>> Go to a group with many single-part binaries, say a picture group, a few
>> tens of thousands of posts.  Here, I tried one with 23,820 unread posts
>> according to pan, plus some downloaded/read already.
>> 
>> Select-all headers and try to download.
>> 
>> Here, pan sits for over an hour
> 
> I feel strange answering something that Duncan posts  ;-)

About like I felt hitting new-post instead of reply. =8^)

> This was filed as a bug and resolved for 0.110
> http://bugzilla.gnome.org/show_bug.cgi?id=352170

Hmm...  0.109 was over a week ago... I've been expecting 0.110.  Maybe
we are back to Sundays?  This gives me something to look forward too.
 
> Interesting, I guess most of my downloads haven't been a large number of
> smaller binaries and I generally don't add much to the Queue so I have
> never noticed this but it is obviously a problem.  Right now I am
> downloading at almost my full 10meg speed.

Yeah, when I tried a multi-part binary group, CD-ISO size, I got full
speed too.

>> Additionally, pan needs to be multithreaded, at least UI in one thread,
>> while everything else is in another, giving the UI back some
>> responsiveness.  With dual CPUs, I'm not /used/ to having a non-responsive
>> app unless it's crashed, any more, at least not unless the system is in
>> i/o-wait, not only running the disk momentarily every few seconds.
>> 
> 
> I agree with this, but I can imagine that it would be a lot of work at
> this point.

Certainly so.  For 1.0, no way.  It would be too destabilizing this late
in the game especially since threads are a about the hardest thing to get
right and prevent deadlock or data corruption races there is. However, it's
going to need to be done, particularly with GHz at a ceiling and CPUs
going multi-core now to increase performance.  Again, at a minimum, a
browsing/ui thread and a network thread.  It could be made four or five
threads without splitting networking, with one for adding to the tasks
(if it can't be made faster by at least 10-fold), one for UI, one for
networking, one for saving to disk, and one for rendering text and images.
Five on a four-way (quad-core or dual-CPU/dual-core) would work, as one
would almost certainly be idle at any point, and another not at 100%,
which would leave room for the rest of the system to run.  That's assuming
some significant CPU bottlenecks remain, tho I home that's a bad
assumption.

>> Maybe I better remerge klibido, if it's going to be doing
/this/.
> 
> <cringe>

I just saw how smoothly it was going with the big-post multi-part group,
and having just bought 6 months worth of newshosting, when I entered a
thousands of small-posts single-part group, I saw that money I just spent
on it going up in smoke, or more precisely, in bottlenecked CPU cycles. 
That's /extremely/ frustrating, and it's absolutely true, klibido was
managing full speed in a similar group (tho with less posts as I had only
Cox at the time) and taking less than half a CPU to do it, so pan
bottlenecking on one CPU and as a result doing barely dialup some of the
time... very very ungood!

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman