bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug-wget] [wget => Kernel message] Too many open files


From: Minato Namikaze
Subject: [Bug-wget] [wget => Kernel message] Too many open files
Date: Thu, 15 Jul 2010 00:31:16 +0200

First of all : sorry for displaying the french version of wget log messages.

Bug (?) type : Kernel message
Reproduce : wget -nc -r -l inf --no-remove-listing http://website.com/
(launched TWICE in different terminals, and with a lot of files already 
downloaded)

Explanation : The two instances will pass over the numerous files already 
downloaded. After that, they will try to download the first not downloaded file.
Then, one of thoses cases appear :
Case 1 : one will be faster and they will download one file out of two each 
(during approximatively 10~20 minutes max until case 2 appears)
Case 2 : one will refuse to download and will display the next error message 
for all files.

Log :
--2010-07-13 14:04:35--  http://www.website.com/folders/
Connexion vers site|ip|:80...connecté.
requête HTTP transmise, en attente de la réponse...200 OK
Longueur: 11886 (12K) [text/html]
www.website.com/folders/file.html: Too many open files

Impossible d'écrire dans <<www.website.com/folders/file.html>> (Too many open 
files).
/endlog


This is the message displayed for all files by ONE of the wgets running
Information: if I stop the failing one, relaunch it, and wait until all files 
have been not-clobbered, I have great chances that the other one will fail with 
the same message(s).

Information: /proc/sys/fs/file-max is set at 405540.
Information: the folder where the website is mirrored contains currently 525000 
html files and as much folders (total: 6.6 GB).
Information: When I have a (BIG) network problem, and whet cancels every single 
file because the address can't be resolved, the elapsed time indicated is 
always 0s. (and thus the average speed is enormous)


If this mailing list was not the place where I should have posted this, please 
let me know.
                                          
_________________________________________________________________
Vous voulez regarder la TV directement depuis votre PC ? C'est très simple avec 
Windows 7
http://clk.atdmt.com/FRM/go/229960614/direct/01/

reply via email to

[Prev in Thread] Current Thread [Next in Thread]