bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-wget] -e robots=off problem


From: Micah Cowan
Subject: Re: [Bug-wget] -e robots=off problem
Date: Mon, 20 Jun 2011 11:20:57 -0700
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.17) Gecko/20110516 Thunderbird/3.1.10

On 06/12/2011 06:34 AM, david chou wrote:
> If there is a robots file ,
> how could I retrieve file from
> that site? I 've used the -e robots=off instruction ,
> but it still can not mirror a web site.

Use the --debug flag to diagnose, and compare the output with

http://wget.addictivecode.org/FrequentlyAskedQuestions#Why_isn.27t_Wget_downloading_all_the_links.3F_I_have_recursive_mode_set

-- 
Micah J. Cowan
http://micah.cowan.name/



reply via email to

[Prev in Thread] Current Thread [Next in Thread]