bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug-wget] Memory Leakage


From: thomas hood
Subject: [Bug-wget] Memory Leakage
Date: Mon, 14 Mar 2016 10:59:25 +0000 (UTC)

  Hey,

Using wget recently to handle large amounts of data. Finding it frequently eats 
into memory quite a bit.Mar 12 06:32:08 ubuntu kernel: [1439386.493486] Out of 
memory: Kill process 63125 (wget) score 661 or sacrifice childMar 12 06:32:08 
ubuntu kernel: [1439386.494595] Killed process 63125 (wget) total-vm:4206452kB, 
anon-rss:3161380kB, file-rss:0kBThese are the parameters I'm using;
wget -c -nd -b -r -e robots=off --header="Accept: text/html" 
--user-agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 
Firefox/40.1" http://$URL/$3 2>&1 | grep '^--' 2>&1 | awk '{ print $3 }' | grep 
-v '\.\(css\|js\|png\|gif\|jpg\|JPG\)$' &Doing a bit of research on Google I've 
found other people have experienced this in the past one or two years. Can 
anyone shed light on the cause? Is it just wget storing the relative links it 
has to traverse in main memory? Obviously there's ways to manage that but I'm 
unsure if it's an issue of memory management. 

Regards,Thomas Hood

  

reply via email to

[Prev in Thread] Current Thread [Next in Thread]