Re: [Bug-wget] wget2 Feature Suggestion - Triston Line

[Tim Rühsen]

On 07/12/2018 08:12 PM, Triston Line wrote:
> If that's possible that would help immensely. I "review" sites for my
> friends at UBC and we look at geographic performance on their apache and
> nginx servers, the only problem is they encounter minor errors from time to
> time while recursively downloading (server-side errors nothing to do with
> wget) so the session ends.

Just forgot: Check out Wget2's --stats-site option. It gives you
statistical information about all pages downloaded, including parent
(linked from), status, size, compression, timing, encoding and a few
more. You can visualize with graphviz or put the data into a database
for easy analysis.

Example:
$ wget2 --stats-site=csv:site.csv -r -p https://www.google.com
$ cat site.csv
ID,ParentID,URL,Status,Link,Method,Size,SizeDecompressed,TransferTime,ResponseTime,Encoding,Verification
1,0,https://www.google.com/robots.txt,200,1,1,1842,6955,33,33,1,0
2,0,https://www.google.com,200,1,1,4637,10661,83,83,1,0
4,2,https://www.google.com/images/branding/product/ico/googleg_lodp.ico,200,1,1,1494,5430,32,31,1,0
5,2,https://www.google.com/images/branding/googlelogo/1x/googlelogo_white_background_color_272x92dp.png,200,1,1,5482,5482,36,36,0,0
3,2,https://www.google.com/images/nav_logo229.png,200,1,1,12263,12263,59,58,0,0

Regards, Tim

signature.asc
Description: OpenPGP digital signature