Re: [Lynx-dev] Lynx fails on http://politiken.dk

[Thorsten Glaser]

Morten Bo Johansen dixit:

>The URL above redirects to http://politiken.dk/VisArtikel.iasp?PageID=1
>but the server replies 404. Typing in the redirect URL directly produces
>the same result.

   Internet Information Services

Sure it's lynx? :-)


/me tcpdumps

Judge yourself:

=> Lynx
GET /VisArtikel.iasp?PageID=1 HTTP/1.0
Host: politiken.dk
Accept: text/html, text/plain, text/sgml, */*;q=0.01
Accept-Language: en
Accept-Charset: utf-8, iso-8859-1;q=0.01, us-ascii;q=0.01
User-Agent: Mozilla/5.0 (UNIX; U; OpenBSD; C; compatible) Lynx/2.8.6dev.5-MirOS 
libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.7d Revision/MirOS:BSD 
(https://MirBSD.BSDadvocacy.org:8890/) fake (MSIE 5.5)
Cookie2: $Version="1"
Cookie: ASPSESSIONIDSQRBRARD=NEGNMMOAIEBAMINAINIBIFGJ
        => (404)

=> Links+
GET /VisArtikel.iasp?PageID=1 HTTP/1.1
Host: politiken.dk
User-Agent: Links (2.1pre15; MirBSD 7 i386; 113x20)
Accept: */*
Accept-Charset: us-ascii, ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4, 
ISO-8859-5, ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10, 
ISO-8859-13, ISO-8859-14, ISO-8859-15, ISO-8859-16, windows-1250, windows-1251, 
windows-1252, windows-1256, windows-1257, cp437, cp737, cp850, cp852, cp866, 
x-cp866-u, x-mac, x-mac-ce, x-kam-cs, koi8-r,koi8-u, koi8-ru, TCVN-5712, 
VISCII, utf-8
Connection: Keep-Alive
Cookie: ASPSESSIONIDSQRBRARD=AIGNMMOAHNDPFBBOLMHAAOCG
        => (works)

=> Netcat
GET /VisArtikel.iasp?PageID=1 HTTP/1.0
Host: politiken.dk
Accept: text/html, text/plain, text/sgml, */*;q=0.01
Accept-Language: en
Accept-Charset: utf-8, iso-8859-1;q=0.01, us-ascii;q=0.01
User-Agent: Links (2.1pre15; MirBSD 7 i386; 113x20)
Cookie2: $Version="1"
Cookie: ASPSESSIONIDSQRBRARD=NEGNMMOAIEBAMINAINIBIFGJ
        => (works)


They are actively blocking anyone with libwww-FM/2.14 in their user agent
string (I tested for that). (Actually, libwww is enough.)

That's not a bug in lynx, and you can change the user agent in the
(O)ptions page.
On the other hand, I would consider contacting them and asking if
they are still all in their right mind...

bye,
//mirabile