lynx-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

LYNX-DEV Re: Automatic dowload of web pages


From: Al Gilman
Subject: LYNX-DEV Re: Automatic dowload of web pages
Date: Sun, 1 Mar 1998 13:42:42 -0500 (EST)

Gary Keck wrote to me instead of to the list:

> I saw a reply from you to someone wanting to "mail" a web page to
> themselves using Lynx.  You suggested using "Expect".
> 
> I want to do the something similar...automatically...to set up a way to
> download futures market data each market day.
> 
> Problem is...while I can use a Unix shell on my ISPs machine, I can't
> load up executables (TCL, Expect).
> 
> Can you think of any other way to automatically get information from a
> web page into a file on a local unix box, without having to resort to
> external programs (using the Unix operating system & shell scripts
> only).
> 
> If I am unable to do so, I'll probably have to set up a Procomm script
> to my PC to do it -- which depends on my PC being on every night!  Not a
> good solution.
> 
> Any info you can provide would be much appreciated.
> 

I am not sure what method would be best for you, or for the Web
community at large.

From my ISP login, I have access to lots of bandwidth so that
there is no motivation to pre-fetch pages before I log in.
And if you get the isp to mount a cacheing proxy instead of
you cacheing pages in your personal account, the pooling of
demand makes the net more efficient.

You can explore whether your ISP will let you file kron jobs to
run overnight, but I am not sure that is the benefit that you
think it would be.

On hack that one could do with my account would be to have
procmail installed to filter your incoming mail -- see if they
will let you do that -- and run your mirroring script every time
that you get a mail message (if kron is not an option).  That's a
pretty kludgey way to do anything, but it could be the only thing
open to you.

HTH

Al


reply via email to

[Prev in Thread] Current Thread [Next in Thread]