lynx-dev Set-cookie requests omitting "GMT" from 'expires'

[David Mosher]

Using Lynx Version 2.8.3rel.1, I recently began experiencing difficulty in
accessing the New York Times (NYT) site.  I believe I have identified the
cookie-related problem and, even though it appears to be the "fault" of that
site and not of Lynx, I would like to describe it, since it may be
informative, and also pose a couple of questions. 

The main NYT page, <http://www.nytimes.com/>, is openly accessible, but
actually reading the articles listed there requires a login (after a free
registration process). 

The login problem I experience is this: When I sign-in with the "Save your
ID and Password" option checked, it doesn't "take"; attempts to access
articles, during that session, or later sessions, get redirected back to the
sign-in page, <http://www.nytimes.com/auth/login>.  If, however, I sign-in
with the "Save your ID and Password" option *unchecked*, I can then
successfully access articles during the current session; unsurprisingly, I
must then sign-in again to access articles during a separate later session.
(I will note that I run Lynx with persistent cookies enabled.)

Here's why, I think:

On initial access to <http://www.nytimes.com/>, the site sets a cookie (if
it is not already present) called RMID via a request such as this:

  Set-cookie: RMID=<RMID>; expires=Friday, 22-Nov-2002 18:33:54 GMT; path=/; 
domain=.nytimes.com

(For brevity and security, I'll used <Cookie-Name> to represent a value for
Cookie-Name.)  So far, so good: Lynx sets this cookie as requested.

When I sign-in with the "Save your ID and Password" option checked, the
server attempts to set two cookies, NYT-S and RDB, via requests such as
these:

  Set-cookie: NYT-S=<NYT-S>; expires=Friday, 11-Nov-2002 13:31:54; path=/; 
domain=.nytimes.com
  Set-cookie: RDB=<RDB>; expires=Tuesday, 05-May-2002 14:31:54; path=/; 
domain=.nytimes.com

Lynx evidently refuses to set either of these cookies.  Why?  Well, note
what is missing in the "expires=" argument: there is no trailing " GMT"!
Because of this, Lynx apparently views the Set-cookie requests as malformed
and discards them.  (FWIW, it also does this when ".nytimes.com" is listed
in .lynxrc among the "cookie_loose_invalid_domains".)

As readers of LYNX-DEV are undoubtedly aware, there never was a IETF RFC
specification for "Netscape-style" ("version 1") cookies.  The closest thing
to an "official" specification is that found in the Netscape document
"Persistent Client State HTTP Cookies,"
<http://www.netscape.com/newsref/std/cookie_spec.html>, which states,

   The date string [the argument for "expires="] is formatted as:
   
     Wdy, DD-Mon-YYYY HH:MM:SS GMT

   This is based on RFC 822, RFC 850, RFC 1036, and RFC 1123, with
   the variations that the only legal time zone is GMT and the
   separators between the elements of the date must be dashes.

   expires is an optional attribute. If not specified, the cookie
   will expire when the user's session ends.


According to this, the trailing " GMT" is mandatory, and Lynx is justified
(if perhaps a bit persnickety) in rejecting Netscape-style cookies which
omit it.


Alternatively, when I sign-in with the "Save your ID and Password" option
unchecked, the NYT server sends Set-cookie requests like these:

  Set-cookie: NYT-S=<NYT-S>; path=/; domain=.nytimes.com
  Set-cookie: RDB=<RDB>; expires=Tuesday, 05-May-2002 14:33:56; path=/; 
domain=.nytimes.com

Lynx still rejects the RDB cookie, but, in this case, it accepts the NYT-S
cookie, which is apparently the important one for logging in, as a
transient, current-session-only cookie; hence I am able to access articles
during that session.


I've notified the NYT site, <mailto:address@hidden>, about the missing
"GMT".  Perhaps they will fix it.  In the meantime, here are two questions:

  1) Since "GMT" is the "only legal time zone" for Netscape-style cookies,
     should otherwise well-formed Set-cookie requests be rejected simply
     for omitting it?
     
  2) Lynx's trace log doesn't seem to flag the rejected cookies.  Their
     rejection, so far as I can tell, can only be inferred by their omission
     from subsequent GET requests.  For example, in the first scenario
     above, the log shows this:

-------------------------------
Initial access of the main page
-------------------------------

Set-cookie: RMID=<RMID>; expires=Friday, 22-Nov-2002 18:33:54 GMT; path=/; 
domain=.nytimes.com
[...]
HTMIME: PICKED UP Set-Cookie: 'RMID=<RMID>; expires=Friday, 22-Nov-2002 
18:33:54 GMT; path=/; domain=.nytimes.com'
[...]
LYSetCookie called with host 'www.nytimes.com', path 
'/auth/login?URI=http://www.nytimes.com/aponline/national',
    and Set-Cookie: 'RMID=<RMID>; expires=Friday, 22-Nov-2002 18:33:54 GMT; 
path=/; domain=.nytimes.com'
LYmktime: Parsing 'Friday, 22-Nov-2002 18:33:54 GMT'
LYmktime: clock=1037990034, ctime=Fri Nov 22 13:33:54 2002
LYProcessSetCookie: attr=value pair: 'RMID=<RMID>'
                    expires: 1037990034, Fri Nov 22 13:33:54 2002


---------------------------------------------------------------------------
Later in same session, after submitting username & password to sign-in page
---------------------------------------------------------------------------

Set-cookie: NYT-S=<NYT-S>; expires=Friday, 11-Nov-2002 13:31:54; path=/; 
domain=.nytimes.com
Set-cookie: RDB=<RDB>; expires=Tuesday, 05-May-2002 14:31:54; path=/; 
domain=.nytimes.com
[...]
HTMIME: PICKED UP Set-Cookie: 'NYT-S=<NYT-S>; expires=Friday, 11-Nov-2002 
13:31:54; path=/; domain=.nytimes.com'
[...]
HTMIME: PICKED UP Set-Cookie: 'RDB=<RDB>; expires=Tuesday, 05-May-2002 
14:31:54; path=/; domain=.nytimes.com'
[...]
LYSetCookie called with host 'www.nytimes.com', path '/auth',
    and Set-Cookie: 'NYT-S=<NYT-S>; expires=Friday, 11-Nov-2002 13:31:54; 
path=/; domain=.nytimes.com, RDB=<RDB>; expires=Tuesday, 
05-May-2002 14:31:54; path=/; domain=.nytimes.com'
LYmktime: Parsing 'Friday, 11-Nov-2002 13:31:54'
LYmktime: Parsing 'Tuesday, 05-May-2002 14:31:54'
LYProcessSetCookie: attr=value pair: 'NYT-S=<NYT-S>'
LYProcessSetCookie: attr=value pair: 'RDB=<RDB>'

---------------------------------------------------------------------------
Only later still, when attempting to access an article, is it now clear, to
me anyway, that the NYT-S and RDB cookies weren't actually set. 
---------------------------------------------------------------------------

LYCookie: Searching for 'www.nytimes.com:80', '/auth/chk_login'.
Checking cookie 1bdd38 RMID=<RMID>
        www.nytimes.com .nytimes.com 1 /auth/chk_login / 0
HTTP: Sending Cookie2: $Version ="1"
HTTP: Sending Cookie: RMID=<RMID>
[...]
Writing:
[...]
User-Agent: Lynx/2.8.3rel.1 libwww-FM/2.14 SSL-MM/1.4.1
Cookie2: $Version="1"
Cookie: RMID=<RMID>

----------------------------------
Sending HTTP request.
HTTP: WRITE delivered OK


  2) (cont.) Might it be possible to have the trace log flag rejected
     cookies in a more obvious manner? 
     

-- 
David Mosher  <address@hidden>

; To UNSUBSCRIBE: Send "unsubscribe lynx-dev" to address@hidden