[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [h5md-user] Dataset layouts
From: |
Pierre de Buyl |
Subject: |
Re: [h5md-user] Dataset layouts |
Date: |
Wed, 10 Jun 2015 15:31:34 +0200 |
User-agent: |
Mutt/1.5.23 (2014-03-12) |
On Tue, Jun 09, 2015 at 04:41:46PM -0400, Peter Colberg wrote:
> Hi Pierre,
>
> On Tue, Jun 09, 2015 at 10:30:31PM +0200, Pierre de Buyl wrote:
> > I understand the motivation. Did you notice a difference in performance or
> > is
> > there some other motivation?
>
> It is not related to performance at all.
>
> Non-compact and compact datasets have to be handled differently for
> writing and reading. A non-compact dataset is only written/read by one
> process, while a compact dataset is written/read by all processes.
>
> When writing/reading a compact dataset from only one process, the
> program will hang. When writing a non-compact dataset from all
> processes, the program will block for a finite period due to
> write conflicts.
>
> The compact layout is preferable over the non-compact layout for
> efficiency reasons (and being equivalent to attributes), hence I
> propose to mandate the compact layout.
I see no problem with that. I'll need to convince Andrew Colette for the
convenience, though :-)
https://groups.google.com/d/topic/h5py/kbhrO-XhFnE/discussion
P