Re: [Lynx-dev] Standard headers

[David Woolley]

On 28/11/2019 06:10, address@hidden wrote:
> I also want you to change Useragent to standard
> format like "Mozilla/5.0 (Lynx; Win64)".

That is very definitely not a standard format.  That claims the browser 
really is version 5.0 of Mozilla, and just adds Lynx and Win64 as comments.

The standard format is a list of tokens and versions, and, if you have a 
web site crawled by a crawler, I think you will still see user agents in 
that format.

The reason that people use the non-standard format you describe was 
discrimination against Internet Explorer in its early days.  Microsoft 
spoofed Netscape as its identity (but kept the proper identity as a 
comment) to trick web sites into thinking it was Netscape.

The original mobile browsers used to follow the proper standard, and 
did, actually, have multiple token/version strings in their identities.

If you send Mozilla/*  as the user agent, you are likely to get unusable 
pages, even from sites that try to be universal.