Re: wget2 | HTTP Response 0 flooding. (#609)

[marcel dope (@marceldope)]

marcel dope commented:


Could you hop on irc?

Thanks for the suggestions, already implemented them. I got the `--stats-all` 
from the wiki. `-R 'index.html*'` does save the indexes on disk too.

The `--stats-site` output currently isn't saved to disk until wget2 finishes. 
It could be sorted after it finishes, although I found a way to deal with that, 
I just do `grep -v "     200" stats, that produces a short, clean list. I'm 
only looking at the last line of stdout output and the `--site-stats` output to 
detect any unsuccessful, partial or otherwise incorrect downloads, as going 
through the stdout output is insurmountable.

The best I came up so far is high `--max-threads` and high `-w` without 
`--random-wait`, that gives me correct downloads. Counter-intuitively, a high 
number of threads seems to help much more than a high -w.

-- 
Reply to this email directly or view it on GitLab: 
https://gitlab.com/gnuwget/wget2/-/issues/609#note_1045633422
You're receiving this email because of your account on gitlab.com.