Re: [Bug-wget] [Bug-Wget] Issues with Perl-based test suite

[Tim Rühsen]

Hi Darshit,

I am answering inline...

Am Sonntag, 28. September 2014, 01:23:08 schrieb Darshit Shah:
> There are a few issues that I've been facing with the old perl based test
> suite that I'd like to highlight and discuss here.
> 
> 1. The way the test suite has been written, I was unable to hack together a
>    patch that will allow the various tests to be run under valgrind. I'd
> like a similar functionality like the new Python based test suite where the
> environment variable, VALGRIND_TESTS causes the Wget executable to be
> invoked under valgrind for memrory leak checks.
>    If anyone could help by contributing a patch / sharing their wisdom on
> how this can be achieved, I'd be very grateful.

This is pretty easy once we have the parallel test suite up and running.
I already did this in configure.ac for the Mget project, so there is not 
secrecy about it. I'll make a patch when the time comes...


> 2. Race Conditions: The tets suite seems to have some races somewhere in it.
> Over the last year or so, I've often seen Test-proxied-https.px fail and
> then pass in the second invokation. This seemed like some race, but
> occurred infrequently enopugh to be a pain point. However, Tim's recent
> patch for using the parallel tets harness seems to be causing more tests to
> fail for me. Now I have all the Test-iri* tests also failing very randomly
> and erratically. A second/third/nth invokation of make check will generally
> see them pass successfully. Without Tim's patch, these tests always passed
> without issues. I'm loath to believe that the patch itself is the cause of
> failure. My understanding is that, it is only triggering the issue more
> often leading to a very high rate of false positives.

Believe me, I made many test runs with the parallel test suite with and 
without -jn. The Test-iri* tests *always* succeeded with LC_ALL=C. They 
*never* succeeded when using TESTS_ENVIRONMENT to set a turkish locale.
I started investigating on friday and will continue next week.

But I never saw spurious failures which might indicate races (But I'll have an 
eye on that). How much work is it for you to install a Debian unstable into a 
virtual machine ? Just for comparison with my machine. If these races persist 
in the virtual machine, something would likely be wrong with your hardware 
(RAM ?) If not, something is wrong with your environment (I remember you have 
a cutting edge Arch Linux box).
Shouldn't be too hard to find out...

>    Again, I urge everyone reading this to share their insights on what /
> where the issue lies and how we can fix it.
> 
> In general, I'd like to see the Perl based test suite deprecated in the near
> future. The Python based test suite suffers from none of the above
> mentioned issues and is highly flexible and extendable. The HTTP Server in
> that test suite is feature complete and all tests can now be ported to it.
> The FTP module however does not yet exist and hence, we must keep the perl
> based tests around till that requirement is fulfilled.

I agree in general, but still have some problems with the python test suite 
(e.g. I am not talking python so far, python test suite seems slower than the 
perl test suite, it seems to be more complex to set up a new test). But I 
promise to work into it in the near future to give you a more detailed 
feedback and/or a helping hand.

Tim

signature.asc
Description: This is a digitally signed message part.