Re: [h5md-user] File format versioning

[Felix Höfling]

Am 03.09.2014, 14:51 Uhr, schrieb Konrad Hinsen  
<address@hidden>:

> Felix Höfling writes:
>
>  > One may also take a different point of view and come to another  
> conclusion
>  > about what backwards-compatibility in the context of a file format  
> means.
>  > I have detailed this earlier, but the discussion was only between  
> Pierre
>  > and me, see eg:
>  > http://article.gmane.org/gmane.science.simulation.h5md.user/657
>
> That's certainly a valid point of view as well. Before deciding on
> version numbers, we need to decide on what "backwards-compatible"
> means, and say so clearly in the documentation.
>
> Fundamentally, my point of view is "change what you want, but document
> everything clearly". That's more important for me than semantic  
> versioning,
> which, as you say, does not have a clear meaning for file formats.
>
> Konrad.

How is this handled by other file formats? Does a Python 2.7 code run with  
a Python 2.4 interpreter? Or can Acrobat Reader 4 (=very old) process an  
encrypted 1.4 file?

At the end of the day, we need to find (and document, I agree) a rule  
which fits our purposes. For me, a change in the major version number also  
signals new features or a major redesign. Compare HDF4 with HDF5, both  
have not much in common AFAIK. In particular, one aim of having "modules"  
in H5MD was to have a stable core specification.

I see the following options:

i) a very strict interpetation of "backwards-compatibility": any 1.x file  
can be processed by any 1.y software.

ii) a somewhat relaxed interpretation such that previously mandatory  
fields can be removed (what else should be allowed ...?)

If we go for i) I suggest that small/unimportant breaking changes are  
simply not applied and other solutions are needed. Specifically, making  
"time" optional does not warrant an new major version, so the change is  
not possible. A compatibility-preserving alternative is to link "time" to  
"step", indicating that "time" contains no additional information.

Having no "step" does not make sense for me, in this case a  
time-independent H5MD element may be used. (We may say that "step" can  
also mean "frame" or something else.)

Replacing the full list of steps/times by a regular grid is a bit more  
difficult. In case i) the only solution I see is to keep the full list and  
add an attribute with the increment, showing that a regular grid is used.  
Compression will hopefully do the rest.

Felix