bug-wget
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Bug-wget] FW: Wget export URL list


From: The PowerTool
Subject: [Bug-wget] FW: Wget export URL list
Date: Wed, 3 Sep 2014 16:13:36 -0400


From: address@hidden
To: address@hidden
Subject: RE: [Bug-wget] Wget export URL list
Date: Wed, 3 Sep 2014 16:12:07 -0400






> Can anyone tell me how to do this with wget ?
> I want it to spider a given website and return the list of full urls in
> that website.

You haven't provided any details of your requirements beyond obtaining the 
result and using wget.

wget alone will not do what you want.  If it were me, I would:

1. wget the html for the site (only download the html)
2. run the html through a simple bash script to rip out all of the URLs
  - and grep -in and -out, as appropriate

The simple script can be found at 
http://www.comp.eonworks.com/scripts/isolate_url_link-20020716.html

This assumes a real OS (not win*) and have a small working knowledge of wget, 
CL (bash), and can dl and extract the script.
                                                                                
  

reply via email to

[Prev in Thread] Current Thread [Next in Thread]