[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[h5md-user] Dataset layouts
From: |
Peter Colberg |
Subject: |
[h5md-user] Dataset layouts |
Date: |
Tue, 9 Jun 2015 16:15:06 -0400 |
User-agent: |
Mutt/1.5.23 (2014-03-12) |
Hi all,
While working on a truly large-scale scientific application using
the HDF5 MPI I/O driver, I noticed it would be useful to mandate that
“meta-datasets” are always stored with compact layout. This would
apply to proposal 100 (scalar step/time), and `box/edges` for the
fixed-geometry case.
When using parallel HDF5, most C API calls are collective, which means
that a function must be called by all processes in the communicator
used to access a file. Compact datasets are metadata and therefore
must be written/read collectively. This goes nicely with the other
metadata stored in attributes.
In similar spirit, the specification states that time-dependent
datasets have an infinite first dimension. I suggest we specify
explicitly that these datasets must have a chunked layout.
Regards,
Peter
- [h5md-user] Dataset layouts,
Peter Colberg <=
- Re: [h5md-user] Dataset layouts, Pierre de Buyl, 2015/06/09
- Re: [h5md-user] Dataset layouts, Peter Colberg, 2015/06/09
- Re: [h5md-user] Dataset layouts, Peter Colberg, 2015/06/09
- Re: [h5md-user] Dataset layouts, Pierre de Buyl, 2015/06/10
- Re: [h5md-user] Dataset layouts, Pierre de Buyl, 2015/06/15
- Re: [h5md-user] Dataset layouts, Felix Höfling, 2015/06/16
- Re: [h5md-user] Dataset layouts, Peter Colberg, 2015/06/24