[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: lynx-dev Oh well... it's been nice... :-|
|
From: |
David Woolley |
|
Subject: |
Re: lynx-dev Oh well... it's been nice... :-| |
|
Date: |
Wed, 29 Apr 1998 08:47:37 +0100 (BST) |
> space for under $500/yr? I wonder if the devillish MSIE
> "robots" had anything to do with it...
According to the maintainer of the imdb site, the MSIE robots are well
behaved and can be squashed by robots.txt, and generate a modified user
agent string, which also allows them to be rejected. The problem is with the
small players, who find it in their commercial interest to not allow
sites to filter them.
Lynx is, of course a problem here, as it doesn't support robots.txt and
allows the user agent string to be forged. Forging the user agent string
to get round a grabbing/web accelerator block just leads to escalation of
the war, with other tests being added to recognise the browsers.
- lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, for Jason White <address@hidden>, 1998/04/29
- Message not available
- Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Michael Warner, 1998/04/29
- Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Heather Stern, 1998/04/29
- Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Jason F. McBrayer, 1998/04/29
- Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Heather Stern, 1998/04/29
- Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Jason F. McBrayer, 1998/04/29
- Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Jan Hlavacek, 1998/04/30
- lynx-dev No Dots., John, 1998/04/30
- Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Michael Warner, 1998/04/29
Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Philip Webb, 1998/04/29
Re: lynx-dev Lynx 2.7.1 and 2.8 refuse to render certain HTML documents, Philip Webb, 1998/04/29