[Swarm-Modelling] comparing models

[Scott Christley]

Anybody familiar with Robert Axelrod's cultural dissemination model?  
And the follow up paper by Axtell, Axelrod, Epstein, and Cohen which 
"docks" Sugarscape with Axelrod's model?  Even if you don't, that's 
fine, my questions are general enough.

In the docking paper, they discuss ways on how to compare the two 
models; how to determine whether they were equivalent or not.  They 
proposed two forms of equivalence, one strong and the other weak.  The 
strong equivalence being that one model could "produce several 
distributions of measurements that were statistically indistinguishable 
from distributions produced" by the other model.  The weak form of 
equivalence being that similar patterns are found in the distributions; 
broad descriptions like increasing, increasing then decreasing, 
periodic, etc.

I've implemented Axelrod's model in Swarm, now I am going to implement 
the model again but with a fundamentally different algorithm(cultural 
dissemination rule) underneath.  Then I want to compare them to see if 
they are equivalent.

Now statistics is not my strong suit, so I hope that somebody can give 
me some pointers or suggest some reading material.

* If I know the size of the state space, which is really large, can I 
use that to determine how many samples (i.e. how many test runs of the 
simulation with a different starting seed) I need in order to get 
confidence that the distribution is the real distribution of the space? 
 Can I characterize that level of confidence?

In Axelrod's paper, he mentions that with the base parameter set it 
describes a state space of 10^500, but he didn't perform many runs.  40 
runs for some and only 10 runs for his larger parameter sets.

* In the docking paper, they mention two statistical tests: two-sided 
Mann-Whitney U statistic and the Kolmogorov-Smirnov (K-S) test.  Are 
there any others?  Any good books or papers that talk about these types 
of tests, pros-cons, underlying assumptions, etc?

Lastly, do you know of any recent papers which try to more clearly 
define the notion of equivalence for agent-based models?

thanks
Scott

ps; the two papers I mentioned are in this book:

Robert Axelrod, "The Complexity of Cooperation, Agent-Based Models of 
Competition and Collaboration", Princeton University Press 1997, 
Chapter 7 and Appendix A.