bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Bug-wget] recursive retrieval of webpage with session cookies


From: Tim Ruehsen
Subject: Re: [Bug-wget] recursive retrieval of webpage with session cookies
Date: Wed, 28 Nov 2012 11:50:57 +0100
User-agent: KMail/1.13.7 (Linux/3.2.0-4-amd64; KDE/4.8.4; x86_64; ; )

Am Wednesday 28 November 2012 schrieb Wolfgang Hennerbichler:
> Hello dear wget-fellows,
> 
> I'm currently crawling a webpage with wget. this page (stupid enough, I
> know) has different language settings, and although the URLs of the
> languages are different, there are specific buttons on the page that
> display data dependent on a session cookie.
> I may be wrong, but It seems that wget doesn't support this kind of
> recursive retrieval, because I just don't see a cookie-file hanging around
> when executing the following command:
> 
> COOKIE=/tmp/cookie
> 
> wget -q -N -r -l 10 -k -p \
> --header='Accept-Language: de-de, de, en-US, en;q=0.5' \
> --save-cookies $COOKIE \
>  --keep-session-cookies \
> --load-cookies $COOKIE \
> --wait=0.1 -D $DOMAIN,http://$DOMAIN http://$DOMAIN

Hi Wolfgang,

I just tested here with Wget 1.14 with my homepage and the file 'cookie' has 
been created with one (as expected) session cookie.

First, make shure that you have a current Wget version running.

If it didn't help:
Replace -q with -d and append some like '>trace.txt 2>&1' to your command 
line. After your command finished, look into trace.txt and search for Set-
Cookie (case insensitive). If you find a HTTP response containing such a line, 
than the cookie should be saved to the cookie file. If not, you likely will 
find some error/info message in trace.txt, saying why the cookie has not been 
accepted.

If you still can't help yourself, you can give one of us the $DOMAIN, so we 
can reproduce the issue. Preferable together with your trace.txt.
If your dare to post your $DOMAIN on the list, send it directly to me or 
someone else who cares.

Regards, Tim



reply via email to

[Prev in Thread] Current Thread [Next in Thread]