[Classpathx-crypto] on testing (long)

[Raif S. Naffah]

hello there,

test.cipher.TestOfSerpent is a model for testing a cipher's correctness, 
and i would like us to think on how we can use this pattern for testing 
all our symmetric key block ciphers.

things to consider:

* we should also test for the cloneability of our ciphers.  this can be 
as as easy as:

   a. initialise a cipher instance,
   b. encrypt one block,
   c. clone the cipher,
   d. encrypt a second block with both the cipher and its clone,
   e. verify that the result from both is the same.
   f. repeat the exercise with decryption.

* the TestOfSerpent uses test vector values that are hard-coded in the 
class.  should we (a) continue doing that, (b) use something similar to 
TestOfNistVectors where the test vector values are read from files in a 
pre-designated location, or (c) combine both approaches?

having hard-coded values that must be processed correctly for the test 
to pass, works around the fact that TestOfNistVectors does not fail if 
it cannot locate the files it is expecting.

besides, having hard-coded values inside the test-case, allows 
application of the (more or less) same pattern to non-NIST ciphers.

i'd go with (c).

* if we go with (b) or (c), how many hard-coded test vectors should we 
use?  --all of them is out of the question because of size and speed. 
theoretically one value should be enough, but 3 is more "re-assuring."

* should there be any special criteria for selecting the agreed number 
of test-vectors? or throwing a dice would suffice?

* once we have coded those TestOfxxx for our ciphers.  should we still 
keep the selfTest() method in the API and implementations?

in its actual form the setlfTest() for the ciphers only test for 
symmetry, which may be true even for incorrectly implemented ciphers. 
in other words it's almost useless!

we should change it to either (a) something similar to the selfTest() of 
hash algorithms, (b) something similar to the TestOfxxx discussed above 
with some hard-coded test vectors, or (c) get rid of it altogether!

* if we get rid of the selfTest() method in the API, how can we 
guarrantee that noone/nothing would replace our "tested" implementations 
with their (probably) insecure/leaking ones?  would a signed jar be enough?

* the test vectors generated by, at least, the NistKat and NistMCT tools 
should be included in the distribution, since they are used by the 
TestOfNistVectors, which (a) if amended to be more rigourous, and (b) 
passes all its tests for the test vector files, would constitute a 
comprehensive "bill-of-health" for the correctness of a cipher's 
implementation.

relying on the NistKat and NistMCT tools to generate them in-situ is not 
a valid option IMO.  a conscientious user would download these test 
vectors from a different source (NIST itself, the home page of the 
algorithm designers, or a trustworthy site/distribution), and only then 
run the TestOfNistVectors over them.

so am i contradicting myself? no.  am i putting more stringent 
conditions on what to distribute and how to distribute it? may be.

let me explain.

+ the NistKat and NistMCT tools are used only for (a) generating test 
vectors for new cipher implementation, and (b) for bootstraping the process.

+ once the test vector files are generated, they are "certified" (by a 
process which i still dont know how).  the end result would be a 
signed-jar for each of the NIST-compatible algorithms; eg. aes-tv.jar 
for the AES test vectors.

+ the gnu-crypto.jar is signed.

+ the TestOfNistVectors, or similar class, would use the cipher 
implementation from the gnu-crypto signed jar with the test vectors from 
the cipher's test vectors signed jar to assert the correctness of that 
cipher.

does it make sense?

* whatever we come with from the discussion of the previous point, a 
third jar, grouping the test vectors, and separate from the 
gnu-crypto.jar and gnu-crypto-test.jar should be considered.


comments and suggestions are welcome + cheers;
rsn