Re: [h5md-user] Particle tracking

[Peter Colberg]

On Sun, Sep 01, 2013 at 04:23:35PM -0400, Pierre de Buyl wrote:
> Peter Colberg <address@hidden> a écrit :
> >On Sun, Sep 01, 2013 at 02:15:41PM -0400, Pierre de Buyl wrote:
> >>Did you consider variable length datatypes? It would allow to keep
> >>the first dimension of datasets the time.
> >
> >The same applies as for Array data types, it does not allow slicing.
> >
> >http://article.gmane.org/gmane.science.simulation.h5md.user/269
> 
> Is it worth losing slicing?

The use of a variable-length data type has negative impacts far beyond
the loss of slicing. The data elements of one time sample are no
longer stored in a continuous (chunked) layout, but in a dynamically
growing heap. This is presumably most inefficient for particle data.

> As time-depedent data requires chunking anyway, what about using
> H5S_UNLIMITED for the N (where N is the number of particles)
> dimension of the datasets in the particles group. This would allow
> for the storage of a (potentially) growing number of particles,
> without having to (i) having to know N_max and (ii) storing unused
> values.

But then the time dimension is fixed, which is undesirable. Not all
sampling schemes use a fixed interval. In my code, I use a logarithmic
sampling scheme of blocks with linearly-spaced samples, where the
total number of trajectory samples is hard (and unnecessary) to
predict.

Peter