[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Bug-wget] can't get wget to not download
From: |
phil curb |
Subject: |
[Bug-wget] can't get wget to not download |
Date: |
Fri, 16 Mar 2012 21:49:19 +0000 (GMT) |
i've made a file robots.txt but wget doesn't seem to be responding to it. it
always downloads.
http://pastebin.com/raw.php?i=kt1mV2af
C:\r>wget 127.0.0.1:56
--2012-03-16 19:45:32-- http://127.0.0.1:56/
Connecting to 127.0.0.1:56... connected.
HTTP request sent, awaiting response... 200 OK
Length: 3 [text/html]
Saving to: `index.html' 100%[======================================>] 3
--.-K/s in 0s 2012-03-16 19:45:32 (20.0 KB/s) - `index.html' saved [3/3]
C:\r>wget 127.0.0.1:56/robots.txt
--2012-03-16 19:45:43-- http://127.0.0.1:56/robots.txt
Connecting to 127.0.0.1:56... connected.
HTTP request sent, awaiting response... 200 OK
Length: 26 [text/plain]
Saving to: `robots.txt' 100%[======================================>] 26
--.-K/s in 0s 2012-03-16 19:45:43 (175 KB/s) - `robots.txt' saved [26/26]
C:\r>type robots.txt
User-agent: *
Disallow: /
C:\r>
- [Bug-wget] can't get wget to not download,
phil curb <=