Re: LYNX-DEV Downloading a whole web site for local offline browsing

[Ari Moisio]

On Mon, 30 Mar 1998, Dave Eaton wrote:

> On Mon, 30 Mar 1998, David Combs wrote:
> 
> > Rather than telling everyone to go download that three-lettered
> > program (no, four: wget), and LEARN the darn thing, why not
> > simply add some of these features to LYNX?
> 
> Proceed with caution here ... you not only need a way to download
> recursively, you also need a way to define how to limit that download so
> you don't try to get the whole web by asking for one file "and all its
> links". That's a new set of parameters and logic lynx doesn't have already
> .... and ones which will make it just that much larger.

  Howabout changin the names used in lynx -crawl -traversal to  reflect
actual directory structure and file names.  That would be easier than
making script like this:

  For example only. Use at your own risk. Sorry finnis comments.

#!/bin/bash
lynx -traversal $1

# kaydaan lapi mita loydettiin
for URL in $(cat  traverse.dat)
        do
        echo -n "$URL " 
        # luodaan kohdetiedoston nimi, muutetaan madot ala viivoiksi ettei 
sehlli sekoilisi
        DEST=$(echo $URL | sed 's/\~/_/g' | cut -c 8-)
        # oletettu tiedosto jos mitaan ei ole annettu
        echo $DEST | grep "/$" && DEST=${DEST}index.html
        if ! test -f $DEST
        then
                DESTDIR=$(dirname $DEST)        
                test -d $DESTDIR || mkdir --parent $DESTDIR
                if  lynx -source $URL > $DEST
                then
                        echo OK
                fi
        fi
done


 
> Although it means "learning something new", using an existing tool that
> does just what is wanted may be better than inventing all over again.
> 
>  Dave Eaton
> --
>  Artronic Development  - Your Web Site Solution
>  12629 N. Tatum Blvd., Suite 224; Phoenix, AZ 85032
>  e-mail: address@hidden - voice: (602) 953-0336  -  http://www.arde.com
> 


--
Ari Moisio, Niittykatu 7, 41160 Tikkakoski, +358-40-5055239
<address@hidden><address@hidden><address@hidden>
PGP 0x3FAF0F05, fp 8A91 96E2 9864 D04D  575E 3EEE 72E9 DDD2
WWW: http://www.sci.fi/~armoi,http://www.arlainst.fi/~arimo