[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-wget] Question regarding robots option

From: Ángel González
Subject: Re: [Bug-wget] Question regarding robots option
Date: Mon, 16 Sep 2013 23:47:38 +0200
User-agent: Thunderbird

On 16/09/13 13:35, Tim Ruehsen wrote:
How comes that there is no --robots / --no-robots option ?
Instead we have to use -e robots=on/off.
This is ok, but a little bit inconsistent and hard to find (wget --help won't
show it).

I would create a patch if there is any interest...

I think that's on purpose for making hard to ignore the robots.txt specification.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]