Re: [Bug-wget] Inconsistent cookie handling between different machines

[Darshit Shah]

Hi,

Thanks for the report and the analysis. However, could you please share the
entire debug output from both runs? Please don't cut out anything, very often
the contextual information around the problem area is about as important. You
may redact the actual Cookie data if you want.

Also, please do share the results of `wget -V` from both the machines. I know
you said, it's the same version, but the compilation options may be different.

* Casey, Sean <address@hidden> [180818 09:06]:
> Hello All,
> 
> I'm not sure if this is the correct place to post this question but I've dug 
> as deep as I can from my end and this seems like the best place to solicit 
> help from the wget gurus.
> 
> I'm running into some weird behavior where the same request sent from the 
> same version of wget from different machines is handling cookie processing 
> differently.
> 
> This is the version that I'm running (on both machines):
> 
> $ wget -V
> GNU Wget 1.17.1 built on linux-gnu.
> 
> +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls +ntlm
> +opie -psl +ssl/openssl
> 
> Wgetrc:
>     /etc/wgetrc (system)
> Locale:
>     /usr/share/locale
> Compile:
>     gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/etc/wgetrc"
>     -DLOCALEDIR="/usr/share/locale" -I. -I../../src -I../lib
>     -I../../lib -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include
>     -DHAVE_LIBSSL -DNDEBUG -g -O2 -fPIE -fstack-protector-strong
>     -Wformat -Werror=format-security -DNO_SSLv2 -D_FILE_OFFSET_BITS=64
>     -g -Wall
> Link:
>     gcc -DHAVE_LIBSSL -DNDEBUG -g -O2 -fPIE -fstack-protector-strong
>     -Wformat -Werror=format-security -DNO_SSLv2 -D_FILE_OFFSET_BITS=64
>     -g -Wall -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro
>     -Wl,-z,now -L/usr/lib -lpcre -luuid -lssl -lcrypto -lz -lidn
>     ftp-opie.o openssl.o http-ntlm.o ../lib/libgnu.a
> 
> 
> This is the command I'm running:
> 
> $ wget --quiet --save-cookies /tmp/cookies.txt --keep-session-cookies 
> --no-check-certificate https://docs.google.com/uc?export=download&id=$DOC_ID
> 
> This is the response I'm getting on both machines (replaced cookie value with 
> sample text for brevity). Note the three cookies being set:
> 
> HTTP/1.1 200 OK
> Content-Type: text/html; charset=utf-8
> Cache-Control: no-cache, no-store, max-age=0, must-revalidate
> Pragma: no-cache
> Expires: Mon, 01 Jan 1990 00:00:00 GMT
> Date: Fri, 17 Aug 2018 19:45:05 GMT
> P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
> P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
> X-Content-Type-Options: nosniff
> X-Frame-Options: SAMEORIGIN
> X-XSS-Protection: 1; mode=block
> Server: GSE
> Set-Cookie: download_warning_random_text=SOME_COOKIE_VALUE; 
> Domain=.docs.google.com; Expires=Fri, 17-Aug-2018 19:50:05 GMT; Path=/uc; 
> Secure; HttpOnly
> Set-Cookie: NID=SOME_COOKIE_VALUE;Domain=.google.com;Path=/;Expires=Sat, 
> 16-Feb-2019 19:45:05 GMT;HttpOnly
> Set-Cookie: NID= SOME_COOKIE_VALUE;Domain=.google.com;Path=/;Expires=Sat, 
> 16-Feb-2019 19:45:05 GMT;HttpOnly
> Alt-Svc: quic=":443"; ma=2592000; v="44,43,39,35"
> Accept-Ranges: none
> Vary: Accept-Encoding
> Transfer-Encoding: chunked
> 
> When running from machine 1 (Ubuntu - 16.04 LTS) I see the following debug 
> statements (note all three cookies are stored):
> 
> cdm: 2
> Stored cookie docs.google.com -1 (ANY) /uc <permanent> <secure> [expiry 
> 2018-08-16 16:23:03] download_warning_random_text SOME_COOKIE_VALUE
> cdm: 2 3 4 5 6 7 8
> Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-02-15 
> 15:18:03] NID SOME_COOKIE_VALUE
> cdm: 2 3 4 5 6 7 8Deleted old cookie (to be replaced.)
> 
> Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-02-15 
> 15:18:03] NID SOME_COOKIE_VALUE
> Registered socket 3 for persistent reuse.
> URI content encoding = 'utf-8'
> Saving cookies to /tmp/cookies.txt.
> Done saving cookies.
> 
> When running from machine 2 (Ubuntu - 16.04.4 LTS) I see the following debug 
> statements (note only two of the three cookies are stored):
> 
> cdm: 2cdm: 2 3 4 5 6 7 8
> Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-02-16 
> 19:45:05] NID SOME_COOKIE_VALUE
> cdm: 2 3 4 5 6 7 8Deleted old cookie (to be replaced.)
> 
> Stored cookie google.com -1 (ANY) / <permanent> <insecure> [expiry 2019-02-16 
> 19:45:05] NID SOME_COOKIE_VALUE
> Registered socket 3 for persistent reuse.
> URI content encoding = 'utf-8'
> Saving cookies to /tmp/cookies.txt.
> Done saving cookies.
> 
> For some reason the "download_warning" cookie is not being stored when 
> running from machine 2.
> 
> I'm not using a custom ~/.wgetrc file from either machine and both machines 
> are using the stock /etc/wgetrc file (no difference between files).
> 
> For the life of me, I can't figure out why that third cookie isn't being 
> stored from machine 2?  The only thing I noticed that is different about that 
> cookie is that it's marked as "secure" while the other two are not.
> 
> I looked through the wget man pages and didn't see any other options that 
> impact cookie processing aside from the ones I've used.  Any help would be 
> greatly appreciated.
> 
> Thanks
> Sean
> -This e-mail and any attachments may contain CONFIDENTIAL information, 
> including PROTECTED HEALTH INFORMATION. If you are not the intended 
> recipient, any use or disclosure of this information is STRICTLY PROHIBITED; 
> you are requested to delete this e-mail and any attachments, notify the 
> sender immediately, and notify the LabCorp Privacy Officer at address@hidden 
> or call (877) 23-HIPAA / (877) 234-4722. 
> 

-- 
Thanking You,
Darshit Shah
PGP Fingerprint: 7845 120B 07CB D8D6 ECE5 FF2B 2A17 43ED A91A 35B6

signature.asc
Description: PGP signature