Re: [Lynx-dev] Lynx fails on http://politiken.dk

[Walter Ian Kaye]

At 04:52p -0400 10/29/2004, David Combs didst inscribe upon an 
electronic papyrus:

> On Mon, Oct 11, 2004 at 04:21:22PM +0000, Thorsten Glaser wrote:
>
>  > => Lynx
>  > GET /VisArtikel.iasp?PageID=1 HTTP/1.0
[snip]
>  > User-Agent: Mozilla/5.0 (UNIX; U; OpenBSD; C; compatible) 
> Lynx/2.8.6dev.5-MirOS libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.7d 
> Revision/MirOS:BSD (https://MirBSD.BSDadvocacy.org:8890/) fake (MSIE 
> 5.5)
[snip]
>  >   => (404)
>  >
>  > => Links+
>  > GET /VisArtikel.iasp?PageID=1 HTTP/1.1
>  > User-Agent: Links (2.1pre15; MirBSD 7 i386; 113x20)
[snip]
>  >   => (works)
>  >
>  > => Netcat
>  > GET /VisArtikel.iasp?PageID=1 HTTP/1.0
[snip]
>  > User-Agent: Links (2.1pre15; MirBSD 7 i386; 113x20)
[snip]
>  >   => (works)
>  >
>  >
>  > They are actively blocking anyone with libwww-FM/2.14 in their user agent
>  > string (I tested for that). (Actually, libwww is enough.)
>  >
>  > That's not a bug in lynx, and you can change the user agent in the
>  > (O)ptions page.
>
>
>
> Naive questions:
>
> (1) What in the above trace tells us that they
>     are blocking libwww in user-agent strings?
>
>     I do see "accept-" this or that, but no "reject"
>     or the like.
>
>    Obviously, I'm missing something that's obvious
>    to everyhone else.  Just what is i

The "(404)" and "(works)" lines, which he typed into his email in 
lieu of the actual HTML pages returned. (Thankfully.)

Of course his UA strings differed by more than just the "libwww" 
portion, so that may not have been the most scientifically valid 
test; however, the result would be the same. You could test it more 
correctly yourself if you like... ;)


> (2) where did the libwww in his UA-string come *from*?
>
>     I mean, did he have it there explicitly in his .lynxrc
>     or lynx.cfg, or perhaps typed-in by hand via the option page?

Probably, but it doesn't matter. :)


-Walter