[Bug-wget] Cookies and WWW-Authenticate

[Martin Carpenter]

Hello,

I believe I've found a bug relating to cookies and basic authentication
with wget 1.11.4. I have looked at the bug tracker and couldn't see it
there, perhaps someone would be so good as to confirm my finding? I'm
surprised that I'm apparently the first person to hit this.

We would like to use wget against a SiteMinder (Computer Associates)
protected website which uses basic authentication. SiteMinder sends a
cookie in the 401 response to the initial request, but wget steadfastly
ignores this (even with --keep-session-cookies and --save-cookies), does
not save it and does not send it back in the next request (with the
authentication credentials). SiteMinder consequently denies the logon.
This makes me sad.

I can work around this issue by specifying the initial cookie (which
happens to be static in my case) on the command line like this:

  --header 'Cookie: ...'

I checked the source, here are the relevant excerpts from src/http.c
(wget v1.11.4):

> static uerr_t
> gethttp (struct url *u, struct http_stat *hs, int *dt, struct url
*proxy)
> {
> ...
>   retry_with_auth:
>   /* We need to come back here when the initial attempt to retrieve
>      without authorization header fails.  (Expected to happen at least
>      for the Digest authorization scheme.)  */
> ...
>               else if (!u->user && BEGINS_WITH (www_authenticate,
"Basic"))
>                 {
>                   /* Need to register this host as using basic auth,
>                    * so we automatically send creds next time. */
>                   register_basic_auth_host (u->host);
>                 }
>               xfree (pth);
>               goto retry_with_auth;
> ...
>   /* Handle (possibly multiple instances of) the Set-Cookie header. */
>   if (opt.cookies)
> ...

The stanza at the end that sets the cookie does not get executed on the
first GET because we jump back to retry_with_auth. I imagine that it's
sufficient to move the "if(opt.cookies) {...}" stanza further up
gethttp() but this is a significant change and will need careful
testing. 

As a side comment, this function could use a little help: it's very long
(~1000 lines) and would be easier to navigate if the contents were
factored out. The multiple return points and gotos are interesting
too ;-)

I'm not likely to be online again much over the next week so apologies
for any slow response here. If nobody else picks this up in the interim,
I can have a bash at producing a patch in early September.

Martin.