Re: [Bug-wget] wget ignores --mirror option on openWrt's Luci interface

[Tim Rühsen]

Hi Olivier,

Am Donnerstag, 11. April 2013 schrieb Olivier Diotte:
> On Thu, Apr 11, 2013 at 5:19 AM, Tim Ruehsen <address@hidden> wrote:
> > Hi Olivier,
> >
> > I got openWRT running.
> >
> > And I can reproduce the problem.
> >
> > Wget -r seems to miss some <a href URLs.
> >
> > I have a conference right now which might take the whole day...
> >
> > But maybe someone else could have a quick look at the html file.
> >
> > It is (attached)
> > 192.168.1.1/cgi-
> > bin/luci/;stok=5ddf8fa64dbdd8ffb5c878e8d2339567/admin/network/index.html
> >
> > and it contains e.g.
> > <a href="/cgi-
> > bin/luci/;stok=5ddf8fa64dbdd8ffb5c878e8d2339567/admin/network/routes/">S
> > tatic Routes</a>
> >
> > but wget doesn't recognize/download that URL.
> >
> > Regards, Tim
> 
> Hi Tim,
> 
> Thanks for your interest in my report.

;-) Well it's a bit like an interactive game...

> 
> I am not sure whether my original report was correct though: I
> originally thought wget missed href URLs, but I now think my problem
> is with the authentication.
> Attached is the simple script I use to do my tests. Based on those
> tests (and having not had a look at wget's code yet) here is what I
> gather:
> -The --save-cookies invocation creates the cookies.txt file (which, as
> far as I can tell, contains the correct cookie information) and the
> ./index.html file which is a logged-in /cgi-bin/luci/index.html file
> with the href tags and all
> -The --load-cookies invocation doesn't use the ./index.html file (it
> can be deleted prior without changing the behaviour) and it creates a
> hierarchy in the "172.16.1.1" folder (or whatever your router's
> hostname is)
> -All files downloaded by the --load-cookies invocation seem to be the
> login page and the requisite files of that page
> -It seems not to be possible to use any combination of '-l', '-r' or
> '-m' 'with the '--post-data' option (with or without the
> --save-cookies option)
> 
> Now, that is the behaviour I get on
> address@hidden:~/Downloads/wget-1.14/foo/usr/local/bin,0$ ./wget --version
> GNU Wget 1.14 built on linux-gnu.
> 
> +digest +https +ipv6 -iri +large-file +nls -ntlm +opie +ssl/gnutls
> 
> Wgetrc:
>     /usr/local/etc/wgetrc (system)
> Locale: /usr/local/share/locale
> Compile: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc"
>     -DLOCALEDIR="/usr/local/share/locale" -I. -I../lib -I../lib -O2
>     -Wall
> Link: gcc -O2 -Wall -lgnutls -lgcrypt -lgpg-error -lz -lz -lrt ftp-opie.o
>     gnutls.o ../lib/libgnu.a
> 
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later
> <http://www.gnu.org/licenses/gpl.html>.
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law.
> 
> Originally written by Hrvoje Niksic <address@hidden>.
> Please send bug reports and questions to <address@hidden>.
> 
> 
> Do you get the same behaviour on your end? Or do you have a
> combination of options which allow you to get the target/main page of
> a --load-cookies invocation to download correctly?

Sorry, I was in a hurry and atached the wrong index.html.
Now, I'm at home without the openWRT stuff.

But i am shure your problem has nothing to do with authentication.
wget -r saved some index.html files, one of them contained a '<a href 
...admin/routes/', but it hasn't been parsed by wget... I am pretty shure 
about that. I take a look tomorrow...

Regards, Tim