bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug-wget] can't get wget to not download


From: phil curb
Subject: [Bug-wget] can't get wget to not download
Date: Fri, 16 Mar 2012 21:49:19 +0000 (GMT)

i've made a file robots.txt but wget doesn't seem to be responding to it. it 
always downloads.
http://pastebin.com/raw.php?i=kt1mV2af 


C:\r>wget 127.0.0.1:56
--2012-03-16 19:45:32--  http://127.0.0.1:56/
Connecting to 127.0.0.1:56... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3 [text/html]
Saving to: `index.html' 100%[======================================>] 3         
  --.-K/s   in 0s 2012-03-16 19:45:32 (20.0 KB/s) - `index.html' saved [3/3] 
C:\r>wget 127.0.0.1:56/robots.txt
--2012-03-16 19:45:43--  http://127.0.0.1:56/robots.txt
Connecting to 127.0.0.1:56... connected.
HTTP request sent, awaiting response... 200 OK
Length: 26 [text/plain]
Saving to: `robots.txt' 100%[======================================>] 26        
  --.-K/s   in 0s 2012-03-16 19:45:43 (175 KB/s) - `robots.txt' saved [26/26] 
C:\r>type robots.txt
User-agent: *
Disallow: /
C:\r>




reply via email to

[Prev in Thread] Current Thread [Next in Thread]