[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: svds test failure
|
From: |
Daniel J Sebald |
|
Subject: |
Re: svds test failure |
|
Date: |
Tue, 31 Jul 2012 14:00:58 -0500 |
|
User-agent: |
Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.24) Gecko/20111108 Fedora/3.1.16-1.fc14 Thunderbird/3.1.16 |
On 07/31/2012 01:25 PM, Daniel J Sebald wrote:
On 07/31/2012 12:47 PM, Rik wrote:
On 07/31/2012 07:02 AM, address@hidden wrote:
No, you're not. There issue is that svds has a test that is
nondeterministic and sometimes the error is higher than tolerance. One
the machine that failed, try running "test svds" a few times and see
how often that fails.
I don't know if we can make this test nondeterministic because the
randomness comes from ARPACK, not from Octave, so I don't know if we
can seed it. We could just increase the tolerance.
7/31/12
Melvin,
This issue comes up again and again. The issue is that ARPACK is an
iterative solution solver and the success or failure can depend on the
initial starting point relative to the solution. In order to stop
variability in tests, we initialize Octave's random number generators and
then create an initial seed location (which is now always the same) to
pass
to ARPACK. However, even this hasn't fully stopped all failures. It is
possible that this is due to small machine-dependent round-off errors
which
render even the initial seed location slightly different between
machines.
I agree with your assessment. However, there are six tests in svds.m and
only two of them use the 'opts' input options. The tests' fail/pass
shouldn't be dependent upon having a specific initial state for the code.
There are some other tests here that look somewhat suspect as well. For
example:
%!testif HAVE_ARPACK
%! s = svds (speye (10));
%! assert (s, ones (6, 1), 2*eps);
To expect a numerical routine to be accurate within two times the
machine precision is asking for quite a bit. The tests which have 'opts'
passed in are asking for accuracy within 1e-10, which seems more
reasonable.
I realize one would expect the result to be deterministic and not fail
in one case and pass in the other, but who knows how ARPACK is set up?
Melvin, is there a way you could make some simple mod to the svds.m
tests to indicate exactly which test(s) is failing.
I think it is the test I noted above that is causing the problem. Look
at these results:
octave:4> s = svds (speye (10))
s =
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
octave:5> s - 1
ans =
8.8818e-16
6.6613e-16
4.4409e-16
4.4409e-16
2.2204e-16
0.0000e+00
octave:6> eps
ans = 2.2204e-16
octave:7> assert (s, ones (6, 1), 2*eps)
error: assert (s,ones (6, 1),2 * eps) expected
1
1
1
1
1
1
but got
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
maximum absolute error 8.88178e-16 exceeds tolerance 4.44089e-16
error: called from:
error:
/usr/local/src/octave/octave-builtin_bin2dec/octave-build1/../octave/scripts/testfun/assert.m
at line 235, column 5
octave:7> s = svds (speye (10))
s =
1.0000
1.0000
1.0000
1.0000
1.0000
1.0000
octave:8> s - 1
ans =
6.6613e-16
4.4409e-16
2.2204e-16
0.0000e+00
0.0000e+00
0.0000e+00
octave:9>
Two things to note above. One is that the result isn't accurate to
within 2*eps and the test fails. The second is that the results are
*not* deterministic. Therefore, it is random that this test could pass
or fail. I suggest bumping the tolerance up to 10*eps or maybe
100*eps... or 1e-10 if you like, same as the rest. It is simply trying
to test the results are fairly accurate, right?
There is one test here that has no tolerance, expecting exact accuracy.
%!testif HAVE_ARPACK
%! [u2,s2,v2,flag] = svds (zeros (10), k);
%! assert (u2, eye (10, k));
%! assert (s2, zeros (k));
%! assert (v2, eye (10, 7));
octave:10> [u2,s2,v2,flag] = svds (zeros (10), k);
octave:12> u2
u2 =
Diagonal Matrix
1 0 0 0 0 0 0
0 1 0 0 0 0 0
0 0 1 0 0 0 0
0 0 0 1 0 0 0
0 0 0 0 1 0 0
0 0 0 0 0 1 0
0 0 0 0 0 0 1
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
octave:13> u2 - eye (10,k)
ans =
Diagonal Matrix
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
octave:14> s2
s2 =
Diagonal Matrix
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
octave:15> s2 - zeros (k)
ans =
0 -0 -0 -0 -0 -0 -0
-0 0 -0 -0 -0 -0 -0
-0 -0 0 -0 -0 -0 -0
-0 -0 -0 0 -0 -0 -0
-0 -0 -0 -0 0 -0 -0
-0 -0 -0 -0 -0 0 -0
-0 -0 -0 -0 -0 -0 0
octave:16> v2
v2 =
Diagonal Matrix
1 0 0 0 0 0 0
0 1 0 0 0 0 0
0 0 1 0 0 0 0
0 0 0 1 0 0 0
0 0 0 0 1 0 0
0 0 0 0 0 1 0
0 0 0 0 0 0 1
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
octave:18> v2 - eye (10, 7)
ans =
Diagonal Matrix
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
0 0 0 0 0 0 0
The reason this works out I suspect is because ARPACK's internal routine
does no iterations for this case, seeing as everything is zeros. That,
or minimal computations because for some odd reason the 's2' check above
shows all negative zero except for positive zero on the diagonal.
Dan