Re: [Mldonkey-users] New source management

[wh]

  Goswin Brederlow wrote:

>Lets throw in my 2 c of thougths:
>
>1. new source should be tried within a reasonable time limit but they
>should not flood out older sources. Each file could get a next_new
>counter. Every time a source is added the earliest time it might be
>tried would be the next_new counter and the counter is increased. If
>the next_new is too low it should be advanced to say a second from now
>or something.
>
Therefore, you would need two (or three) seperate queues: One for known
good sources, one for new and untried ones and another one maybe for bad
sources that have been deleted from the normal sources and that should
be retried very seldomly.

>2. each file should have its own queue. Otherwise popular files would
>kill sources of sparse files.
>
Yes, an own queue per file is definitly necessary

>3. Each file should have a priority queue.
>- The priority should be initialised with some small offset depending
>  on the IP. Known dynamic IPs get a penalty.
>
>
Are you sure that this would work?

>- When a connection is made or failes the priority gets addjusted up
>  or down.
>- When the client is identified the client and version give some small
>  adjustment to the priority. Nothing big but some.
>- If a client has chunks I need it gets a bonus for each. _Otherwise_
>  for each chunk it needs from me it gets a malus. (its more likely
>  the client will get a chunk I already have than a new one).
>  Sparseness of chunks could be considered here.
>
Yes, this way of dynamic source priorization should improve our download
speed problems, although it has to be tested very much...

>- If I get some upload from a client the source should get a
>  bonus. Maybe the speed could be factored in too.
>
This factor shouldn't be very high, else we would get a credit system
again. And I don't think a credit system is a good thing for many reasons.

>- The time since the last upload and last connect should be factored
>  in. A successful connect without a download isn't neccessarily a
>  good thing.
>
>- The queue should have a high and low watermark (as it has now). If
>  the high mark is reached the worst sources are purged one by one
>  until the low mark is reached. If that means puring new sources you
>  never connected to so be it. One could just purge the worst purge
>  whenever the queue is full but then it might allways be the newfound
>  super fast but untred source. With a high/low mark system there is a
>  certain time between finding more sources and purging where new
>  connects can be tested.
>
ACK

>
>Additionally I think the penalty for a connection failure for known
>dynamic IPs should be very high. Its quite unlikely that a dialin user
>has its donkey down for an hour and the start it up again. Its far
>more likely he got a new IP.
>
This could perhaps be tried, but I don't think the positive effect would
be too big. But as you say, this could be one of the priority factors
after a restart of Mldonkey

>
>4. All file queues should be in a priority queue. Its priority there
>should depend on the top entry of each queue and the file itself. X
>connects/second are then tried from that queue.
>- each file should have a next_try counter.
>- the priority of a file should be factored in. High priority files
>  should be tried more often.
>- the priority of the top element of each queue should be factored in
>
>The easiest way would be to increase the next_try counter according to
>the files priority and the priority of the top element of that queue
>each time the queue is tried.
>
ACK, but there is one problem: The first file (with low priority) has
1000 sources. The second file (high priority, very rare) has 20 sources.
The 20 sources of the second file would be flooded with requests. There
should be a min_retry_delay for each source, but I think this feature is
already existant in MLDonkey right now.

>
>5. To prevent a source flooding a queue with only new sources between
>the high and low watermark or in other words no new source has been
>tried since the last purge of the queue could be locked to not accept
>any new sources till all existing once have been tried. Sources that
>get a malus on their first try would be removed from the queue, other
>kept. When all new sources have been tried and the queue is still at
>the high watermark the normal purging is done.
>
MLdonkey proposed, that every new source (no matter how many there are)
should be tried once, before it is rated. I don't really know which way
is better...

>
>6. the next_new counter should probably be used for other sources
>too. Every time a source disconnects it should get a next_retry_time
>from the next_new counter if thats greater than the normal retry time
>or the average of both. Alternatively two queues could be used. One
>for existing sources and one for new ones and the two could be used
>alternating (unless the retry time is not yet reached). Actually I
>like that more.
>
ACK

>
>7. a source that has multiple files should be added to the file with
>the highest priority. This could be just the user set priority or
>derived from the user set priority, the sparseness of the file/blocks,
>the age of the file, the number of active connects and sources for the
>file, ... The user set priority should probably be the main facor here
>and the other factors should only make up for a bit, say +-10 points
>on the user set priority. Setting a user set priority of 5 would mean
>prever this but also get others. Setting it to 100 would mean
>absolutely get this.
>
How does emule banning work? Does it ban for too frequently asking
sources at all or does it ban for too frequently asking per source

>
>May the Source be with you.
>
>
I can already feal the spirit of the sources... ;)