Re: [Mldonkey-users] New source management

[MLdonkey]

>  >I hope that if any such scheme replaces the actual connection backoff + LRU,
>  >the two will be heavyly tested one against the other, because more complex
>  >algorithm don't always mean better performance.
>  >
>  >In fact, complex algorithms can fail in many new and strange ways...
>  >
>  Of course it has to be tested VERY MUCH, and there would be also other 
>  problems:
>  Would there be one global queue or one queue for each file?
>  If there is a global queue, it could happen that one of the files has 
>  10000 sources and all sources of other files fall out off the queue and 
>  have not one single source anymore.
>  If there is a queue each file, the retry_delays wouldn't be respected, 
>  because a client could be in more than one queue because he has 
>  different files. And if the queue is too long, the number of sources, 
>  that have never been asked before could get too big and a new source is 
>  connected 4 hours after it was "new" the first time. And if you make too 
>  much difference between the types of clients (Emule, Overnet,...), it 
>  could happen that no single Emule is asked but Edonkeys are kind of 
>  flooded.

I agree with all that. The goal is not to find a complex algorithm,
but on the contrary a simple one. Moreover, it must be fair: popular
files should not take all the connected slots if there are some rare
files being downloaded too.

A bad scenario is when you are both downloading a rare file and a
popular one: the rare file has its few sources in "old sources"
because they are not always online; the popular file keeps receiving
new sources, which have a better priority. 

What we can do is to affect some connection slots to each queue: 5 for
"good sources", 3 for "emerging source", 2 for "concurrent sources", 2
for "bad sources", and finally 1 for "old sources" . So, depending on
its current state (counter modulo 13), the algorithm will try to peek
a source from the given queue. If all the queues have at least one
affected slot, each source will eventually be tested ...

>  But I think that it is important that NOT EVERY new source is being 
>  asked, because this really generates too much traffic overhead, no 
>  matter which method is being used.

Well, I think you have to test all the sources until the download is
finished. Some sources might be tested more often than other ones, but
all have to be. Just suppose you have 300 good sources, you are queued
in all of them, and you don't want to test other sources, just imagine
that the 301st has a free slot and a high bandwidth ... you still
don't want to connect it ?

For the overhead, 5 connections/second is 1.5 KB/s (up and down) if you 
connect, 
160 B/s if you don't. You don't connect to most sources, so it does
not connect too much. Most of the time, I have 20 KB/s
max_hard_upload_rate, but I'm a priviledged cable user...

Finally, we should not worry to much about the ratio between the
bandwidth used to get sources and the bandwidth used for uploading our
shared files. There is some psychological effects to take into
account. If mldonkey does not download fast enough, new users won't
like it, and they will use other peer-to-peer systems (gnutella,
direct-connect,kazaa, ...). So, the faster it downloads, the more
users we have. And these users we have addicted to edonkey, sooner or
later, they will start thinking about sharing more. Most of us
download a lot at the beginning, because they are many vacation movies
we want to see, and then, we have seen most of them, and the upload
starts being greater than the download ...

- MLDonkey