Slow performance on Linux

[Francesco Potorti`]

Jim Van Zandt writes:

   My results on a 90 MHz Pentium:

   test #1          0.23125 +/- 3.1% (10 runs)
   test #2           1.2576 +/- 5.0% (102 runs)
   test #3         0.062327 +/- 13.6% (1000 runs)

   ...about twice as long for the first two tests, and over 100 times as
   long for the third test.

No, it's just the opposite!  It is about 100 times quicker for the
third test!  How can it be?

   The parse error was generated by the formfeed.

I sometimes add comments to source files just before posting them,
this time I added a form feed too, not remembering that octave does
not take it as a blank ...

   I think it makes more sense to limit the total *time* (to perhaps 100
   sec) rather than the number of repetitions.

I added a check in order to limit the length of each test to
maxseconds (default 60).

   We might also think about scaling up the size of the problem rather
   than the number of iterations.  

Yes, this is reasonable but more difficult, because you say:

                 Of course, the scaling has to be understood.  If the
   benchmark were "the biggest matrix that can be inverted in 60 sec",
   then doubling the size of the matrix takes 8 times the processing
   speed.

and it is not always obvious how the scaling works and how precise it
is (ordinarily one discards lowest order factors).  

However, here is another bm.m.  This time I use the example in the
octave manual for the lsode function, which is very short.  It would
be interesting to know if we obtain yet such different results for
different machines with this change.

On Alpha:

bm
test #1         0.0939749 +/- 4.8% (18 runs)
test #2         0.453003 +/- 3.6% (10 runs)
test #3          1.40105 +/- 1.6% (10 runs)

------------------- bm.m ---------------------
1;  # This is not a function file
  
function xdot = xdot (x, t)
  r = 0.25; k = 1.4; a = 1.5; b = 0.16; c = 0.9; d = 0.8;
  xdot(1) = r*x(1)*(1 - x(1)/k) - a*x(1)*x(2)/(1 + b*x(1));
  xdot(2) = c*a*x(1)*x(2)/(1 + b*x(1)) - d*x(2);
endfunction

#
# Do benchmark
#
nf = 3;

function time = test(f)
  global s0, t;
  oldtime = cputime();
  if (f == 1)     inv(hadamard(7));
  elseif(f==2) inv(hadamard(8));
  elseif(f==3) lsode ("xdot",[1;2],(t=linspace(0,50,200)'));
  endif
  time = cputime() - oldtime;
endfunction

targetaccuracy = 0.025;
minrepetitions = 10;
maxrepetitions = 10000;
maxseconds = 60;

for f = 1:nf
  res = [];
  test(f);  # run a first test to increase the RSS, load things and so on
  tic();
  for rep = 1:maxrepetitions
    res(rep) = test(f);
    if (rep < minrepetitions) 
      continue 
    endif
    # purged results: remove min and max elements
    pres = res((res != max(res)) & (res != min(res)));
    if (std(pres)/mean(pres) < targetaccuracy || toc() > maxseconds)
      break 
    endif
  endfor
  # print 95% confidence interval
  printf("test #%d\t\t%8g +/- %.1f%% (%d runs)\n", ...
          f, mean(pres), 200*std(pres)/mean(pres), rep);
endfor