Re: [Bug-wget] Using wget to measure web response times?

[Micah Cowan]

On 06/22/2010 08:01 AM, Keisial wrote:
> Giuseppe Scrivano writes:
>> <address@hidden> writes
>>> I have been using wget, together with 'time', with the following command 
>>> line parameters:
>>>
>>> time wget --page-requisites --secure-protocol=SSLV3 --load-cookies 
>>> cookies.txt --keep-session-cookies 
>>> https://portal.foo.com/test/appmanager/portal/desktop
>>>
>>> However, when I do this, on the same environment where we see > 30 seconds 
>>> response to a browser, I get 'real' response times from the 'time' command 
>>> of 3-5 seconds.  That would SEEM to indicate that the browser is taking 28+ 
>>> seconds to render the portal page.
>>>
>>> So I was wondering:  Is this a valid test and use of wget?  And, am I 
>>> interpreting the test results correctly?
>>>     
>> Using wget do you get _exactly_ the same information that you get using
>> the browser?
>>
>> Giuseppe
>>   
> 
> Maybe the slow time is in getting some css/javascript/image file
> included from the web page?

It's always possible that the server responds differently based on
something like the User-Agent or even Accept-Encodings header; for
instance if it sees Accept-Encodings: gzip, it might gzip the content
before sending or something (though if that adds 27 seconds, it might
not have been a good idea...); or it might dynamically generate some
(CSS?) file based on the User-Agent string.

JavaScript does seem a real possibility, since wget will never interpret
JavaScript code.

I'd probably try running tcpdump (Unix systems) to capture web packets,
to see if it's really the traffic that's taking that long, or if there's
a significant pause in there somewhere. Might be good to save a log from
that, and try to scan for GET URLs to see if the browser asks for pages
that wget does not.

-- 
HTH,
Micah J. Cowan
http://micah.cowan.name/