[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] recursive retrieval of webpage with session cookies
From: |
Tim Ruehsen |
Subject: |
Re: [Bug-wget] recursive retrieval of webpage with session cookies |
Date: |
Wed, 28 Nov 2012 11:50:57 +0100 |
User-agent: |
KMail/1.13.7 (Linux/3.2.0-4-amd64; KDE/4.8.4; x86_64; ; ) |
Am Wednesday 28 November 2012 schrieb Wolfgang Hennerbichler:
> Hello dear wget-fellows,
>
> I'm currently crawling a webpage with wget. this page (stupid enough, I
> know) has different language settings, and although the URLs of the
> languages are different, there are specific buttons on the page that
> display data dependent on a session cookie.
> I may be wrong, but It seems that wget doesn't support this kind of
> recursive retrieval, because I just don't see a cookie-file hanging around
> when executing the following command:
>
> COOKIE=/tmp/cookie
>
> wget -q -N -r -l 10 -k -p \
> --header='Accept-Language: de-de, de, en-US, en;q=0.5' \
> --save-cookies $COOKIE \
> --keep-session-cookies \
> --load-cookies $COOKIE \
> --wait=0.1 -D $DOMAIN,http://$DOMAIN http://$DOMAIN
Hi Wolfgang,
I just tested here with Wget 1.14 with my homepage and the file 'cookie' has
been created with one (as expected) session cookie.
First, make shure that you have a current Wget version running.
If it didn't help:
Replace -q with -d and append some like '>trace.txt 2>&1' to your command
line. After your command finished, look into trace.txt and search for Set-
Cookie (case insensitive). If you find a HTTP response containing such a line,
than the cookie should be saved to the cookie file. If not, you likely will
find some error/info message in trace.txt, saying why the cookie has not been
accepted.
If you still can't help yourself, you can give one of us the $DOMAIN, so we
can reproduce the issue. Preferable together with your trace.txt.
If your dare to post your $DOMAIN on the list, send it directly to me or
someone else who cares.
Regards, Tim