Re: Feedback from JRES in Dijon

[Bengt Richter]

Hi zimoun,

On +2019-12-05 16:52:47 +0100, zimoun wrote:
> Hi,
> 
> On Thu, 5 Dec 2019 at 16:45, Konrad Hinsen <address@hidden> wrote:
> 
> > > So they are doing physical simulation (fluid dynamics), so they don't
> > > (can't) get the same result when running the same experiment
> > > twice. They wart replicability, that is, even if the results are
> > > different, they are close enough to each other that you have to draw
> > > the same scientific conclusion, independent of your compiler or other
> > > package inputs.
> >
> > That's a common point of view in the numerical simulation community.
> > What the people defending it don't realize is that both reproducibility
> > and replicability matter, but in different situations and for different
> > reasons. Reproducibility matters for verification ("was the computation
> > done correctly?"), replicability matters for validation ("was the
> > computation the right one for the scientific question?").
> 
> If I might, one the best presentation [1] -- that I am aware of -- on
> this. Sorry in French.
> 
> [1] 
> https://webcast.in2p3.fr/video/les-enjeux-et-defis-de-la-recherche-reproductible
> https://aramis.resinfo.org/wiki/lib/exe/fetch.php?media=pleniaires:aramis_keynote_enjeux-et-defis-recherche-reproductible_konrad_hinsen.pdf
>

Is [1] available as a libre video download?
My current youtube-dl could not find the video of [1], though I watched it with 
firefox.
I think it would be good for my French, such as it is, to watch that a few 
times ;-)

IMO if you can't reproduce bit-identical results, you should find out _exactly_ 
why.
And if you do get exactly the same result, you should also know exactly why ;-)

IME, in the process of figuring out the explanations you will always learn 
something.

You may be reproducing perfectly a biased result which could be improved as
a numerical result by counter-intuitively adding noise to the inputs
(to spread roundoff across the lsb roundoff boundary so that a mean will show
where in the interval the best estimate fell).

And if you are on the hairy edge of having a non-singular matrix to invert,
results can change because your thread got interrupted at a point where fp regs
got stored as 64 bits and thus restored with different precision than if the
interrupt didn't happen. Just because someone logged in remotely for unrelated 
reasons.

Such an infinitesimal value change can change the search order for pivot point 
in a matrix
and change the order of computation, or even make zero elements that otherwise 
differed
enough to make the inversion possible. And maybe you shouldn't actually store 
an inverse
from a library routine and then multiply, but rewrite your algorith to avoid 
truncations-
Of course your results may still be unusably sensitive ;-)

OTOH, you may notice that your algorithm would do much better if you 
pre-translated your
inputs to a better coordinate system orgin (e.g. to make lots of off-diagonal 
exact zeroes).

You may want to analyze how roundoff errors propagate through a Kalman filter 
vs a direct MLE version,
to see at what bit region in your numerical results input effects turn to 
computational artifacts,
to judge how to express your idea of the scientific significance of your 
results.

The video touches on IEEE 754 (and I do believe the lecturer understands these 
issues, but
he says (IIU.fr :) no programming language gives control over the FPU -- is 
that true?? I mean,
including on-the-metal x86_64 assembly language?? I thought you could set all 
those bits
that the firmware pays attention to, no?)

BTW, if you are interested in floating point, links from here should be fun:

--8<---------------cut here---------------start------------->8---
https://en.wikipedia.org/wiki/William_Kahan
--8<---------------cut here---------------end--------------->8---

Hey, in the '60s we wrote floating point in assembler, (after having written 
the assembler
for the custom system :) and were happy to get division timed under a 
millisecond ;-) Time flies :)

> Maybe we could convert it to an entry for the HPC blog. What do you think?
> 
> 
> Cheers,
> simon
> 

-- 
Regards,
Bengt Richter