[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: lynx-dev Deny access for non standard browsers
|
From: |
Larry W. Virden |
|
Subject: |
Re: lynx-dev Deny access for non standard browsers |
|
Date: |
Thu, 29 Jul 1999 10:21:08 -0400 (EDT) |
From: David Woolley <address@hidden>
> (I pointed out that he should use robots.txt), so I suspect that
Does lynx's crawl mode now respect rebots.txt ? I thought that it still
didn't, which was a complaint the list had gotten in the past.
--
Larry W. Virden <URL: mailto:address@hidden>
<URL: http://www.purl.org/NET/lvirden/> <*> O- "No one is what he seems."
Unless explicitly stated to the contrary, nothing in this posting should
be construed as representing my employer's opinions.
-><-