Re: [Bug-wget] wget --spider doesn't get file size

[Tim Ruehsen]

Hi.

Am Monday 24 June 2013 schrieb Yiwei Yang:
> I want to get some file size. Some people recommend wget --spider. However,
> when I run it on some links, like http://autos.cn.yahoo.com/,
> http://www.cbsa.gc.ca/menu-eng.html,http://oprofile.sourceforge.net/mail/an
> d so on,  then it said Length: unspecified [text/html]. Is there way to
> solve this or I could use another way to get file size? Thank you!

With many dynamically composed pages the web server doesn't know about the 
page length when starting the transmission. That means you have to download 
the whole page to determine the length. --spider only works when the server 
sends a Content-Length: header (regularly used with static pages).

Keep in mind that dynamically composed pages (as the name says) do not have a 
fixed size. It might change from download to download or from time to time.

Regards

     Tim