Re: [h5md-user] Particle tracking

[Pierre de Buyl]

Hi Mathieu, and welcome!

Peter Colberg <address@hidden> a écrit :
> On Sun, Sep 01, 2013 at 12:43:18AM +0200, Mathieu Leocmach wrote:
> > I like very much the H5MD initiative and I wonder if I could use this
> > format to output and share my experimental data.
> >
> > I track particles, usually colloids via confocal microscopy. Contrary to
> > simulations, I observe only a small portion of the system and particles
> > always come in and out of the field of view. Therefore the number of
> > particles in a frame is not constant in time and the number of particles
> > seen at least once is increasing with time.
> >
> > Do you think your file structure can accommodate this type of data?

As of now, it is possible but implies to pick a solution such as using
the "id" data (see Olaf Lenz's post - btw, the thread has split) and
tagging "empty" particle slots with an id of -1, for instance.

As Olaf and Peter mentioned, you should still set a maximum number Nmax of
particles to be considered but if you enable chunking on your dataset you
do not need to store Nmax. Empty chunks do not need storage.

Example: say Nmax=10^6, chunk size=1000.

For 1 to 1000 particles, you only store one chunk (1000 points), for
i*1000 to i+1*1000 particles, you store only i+1 chunks.

This allows to set Nmax to large values without wasting disk space.

> I think the H5MD file format would be well suited for this data. As
> Olaf mentioned, however, the specification currently specifies data
> groups for fixed particle number only.
>
> A while ago I proposed [1] an extension to extend the step/time/value
> scheme with a range dataset to accommodate variable particle numbers.
> Now would be the right time to finalize this proposal.
>
> [1] http://thread.gmane.org/gmane.science.simulation.h5md.user/76

Did you consider variable length datatypes? It would allow to keep
the first dimension of datasets the time.

Pierre