[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnurl] 02/282: runtests: make random seed fixed for a month
From: |
gnunet |
Subject: |
[gnurl] 02/282: runtests: make random seed fixed for a month |
Date: |
Wed, 01 Apr 2020 14:27:47 +0200 |
This is an automated email from the git hooks/post-receive script.
ng0 pushed a commit to branch master
in repository gnurl.
commit a7d4693a4813a13d6b2e6f223f119e13b3b58102
Author: Daniel Stenberg <address@hidden>
AuthorDate: Wed Dec 18 15:37:20 2019 +0100
runtests: make random seed fixed for a month
When using randomized features of runtests (-R and --shallow) it is
useful to have a fixed random seed to make sure for example extra
commits in a branch or a rebase won't change the seed that would make
repeated runs work differently.
As it is also useful to change seed sometimes, the default seed is now
determined based on the current month (and first line curl -V
output). When the month changes, so will the random seed.
The specific seed is also shown in the standard test suite top header
and it can be set explictly with the new --seed=[num] option so that the
exact order of a previous run can be achieved.
Closes #4734
---
tests/runtests.1 | 16 +++++++++++++---
tests/runtests.pl | 33 ++++++++++++++++++++++++++-------
2 files changed, 39 insertions(+), 10 deletions(-)
diff --git a/tests/runtests.1 b/tests/runtests.1
index 69aafe579..ffda8d066 100644
--- a/tests/runtests.1
+++ b/tests/runtests.1
@@ -78,6 +78,9 @@ people checking the failures and the reasons for them might
not have physical
access to the machine and logs.
.IP "-R"
Run the tests in a scrambled, or randomized, order instead of sequentially.
+
+The random seed initially set for this is fixed per month and can be set with
+\fI--seed\fP.
.IP "-r"
Display run time statistics. (Requires Perl Time::HiRes module)
.IP "-rf"
@@ -91,11 +94,18 @@ If \fB-R\fP is also used, the scrambling is done after the
repeats have
extended the test sequence.
.IP "-s"
Shorter output. Speaks less than default.
-.IP "--shallow=[num](,seed)"
+.IP "--seed=[num]"
+When using \fI--shallow\fP or \fI-R\rP that random certain aspects of the
+behavior, this option can set the initial seed. If not set, the random seed
+will be set based on the currently set local year and month and the first line
+of the "curl -V" output.
+.IP "--shallow=[num]"
Used together with \fB-t\fP. This limits the number of tests to fail in
torture mode to no more than 'num' per test case. If this reduces the amount,
-the given 'seed' will be used to randomly discard entries to fail until the
-amount is 'num'.
+the script will randomly discard entries to fail until the amount is 'num'.
+
+The random seed initially set for this is fixed per month and can be set with
+\fI--seed\fP.
.IP "-t[num]"
Selects a \fBtorture\fP test for the given tests. This makes runtests.pl first
run the tests once and count the number of memory allocations made. It then
diff --git a/tests/runtests.pl b/tests/runtests.pl
index 27f62edd5..8a8a6e02a 100755
--- a/tests/runtests.pl
+++ b/tests/runtests.pl
@@ -69,6 +69,7 @@ BEGIN {
use strict;
use warnings;
use Cwd;
+use Digest::MD5 qw(md5);
# Subs imported from serverhelp module
use serverhelp qw(
@@ -323,7 +324,7 @@ my $torture;
my $tortnum;
my $tortalloc;
my $shallow;
-my $shallowseed;
+my $randseed = 0;
#######################################################################
# logmsg is our general message logging subroutine.
@@ -3008,6 +3009,7 @@ sub checksystem {
logmsg sprintf("* Env: %s%s", $valgrind?"Valgrind ":"",
$run_event_based?"event-based ":"");
logmsg sprintf("%s\n", $libtool?"Libtool ":"");
+ logmsg ("* Seed: $randseed\n");
if($verbose) {
logmsg "* Ports:\n";
@@ -5046,18 +5048,20 @@ while(@ARGV) {
$tortalloc = $1;
}
}
- elsif($ARGV[0] =~ /--shallow=(\d+)(,|)(\d*)/) {
+ elsif($ARGV[0] =~ /--shallow=(\d+)/) {
# Fail no more than this amount per tests when running
# torture.
- my ($num, $seed)=($1,$3);
+ my ($num)=($1);
$shallow=$num;
- $shallowseed=$seed?$seed:1234; # get a real seed later
- srand($shallowseed); # make it predictable
}
elsif($ARGV[0] =~ /--repeat=(\d+)/) {
# Repeat-run the given tests this many times
$repeat = $1;
}
+ elsif($ARGV[0] =~ /--seed=(\d+)/) {
+ # Set a fixed random seed (used for -R and --shallow)
+ $randseed = $1;
+ }
elsif($ARGV[0] eq "-a") {
# continue anyway, even if a test fail
$anyway=1;
@@ -5118,11 +5122,12 @@ Usage: runtests.pl [options] [test selection(s)]
-l list all test case names/descriptions
-n no valgrind
-p print log file contents when a test fails
- -R scrambled order
+ -R scrambled order (uses the random seed, see --seed)
-r run time statistics
-rf full run time statistics
-s short output
- --shallow=[num](,seed) make the torture tests thinner
+ --seed=[num] set the random seed to a fixed number
+ --shallow=[num] randomly makes the torture tests "thinner"
-t[N] torture (simulate function failures); N means fail Nth function
-v verbose output
-vc path use this curl only to verify the existing servers
@@ -5175,6 +5180,20 @@ EOHELP
shift @ARGV;
}
+if(!$randseed) {
+ my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) =
+ localtime(time);
+ # seed of the month. December 2019 becomes 201912
+ $randseed = ($year+1900)*100 + $mon+1;
+ open(C, "$CURL --version 2>/dev/null|");
+ my @c = <C>;
+ close(C);
+ # use the first line of output and get the md5 out of it
+ my $str = md5($c[0]);
+ $randseed += unpack('S', $str); # unsigned 16 bit value
+}
+srand $randseed;
+
if(@testthis && ($testthis[0] ne "")) {
$TESTCASES=join(" ", @testthis);
}
--
To stop receiving notification emails like this one, please contact
address@hidden.
- [gnurl] branch master updated (697584161 -> 10e1f0a79), gnunet, 2020/04/01
- [gnurl] 08/282: curl.h: remove WIN32 define, gnunet, 2020/04/01
- [gnurl] 02/282: runtests: make random seed fixed for a month,
gnunet <=
- [gnurl] 03/282: docs/RELEASE-PROCEDURE.md: pushed some release dates, gnunet, 2020/04/01
- [gnurl] 07/282: tool_dirhie.c: fix the copyright year range, gnunet, 2020/04/01
- [gnurl] 05/282: tool_dirhie: Allow directory traversal during creation, gnunet, 2020/04/01
- [gnurl] 04/282: KNOWN_BUGS: AUTH PLAIN for SMTP is not working on all servers, gnunet, 2020/04/01
- [gnurl] 01/282: RELEASE-PROCEDURE.md: fix next release date (Feb 26), gnunet, 2020/04/01
- [gnurl] 10/282: libtest/mk-lib1521: adapt to new public header layout, gnunet, 2020/04/01
- [gnurl] 06/282: bump: work towards 7.69.0 is started, gnunet, 2020/04/01
- [gnurl] 12/282: scripts/delta: adapt to new public header layout, gnunet, 2020/04/01
- [gnurl] 15/282: vtls: Refactor Curl_multissl_version to make the code clearer, gnunet, 2020/04/01
- [gnurl] 20/282: cmake: Improve libssh2 check on Windows, gnunet, 2020/04/01