Re: [Gnumed-devel] wells score - how to store it ?

[Tim Churches]

Karsten Hilbert wrote:
>>In fact, OIO uses a separate table for each version of each form you
>>define. OIO uses a generic table to hold metadata about each field on
>>each form, but the actual data for each form is held in its own table.
>>NetEpi Case Manager does the same - it uses one table per form version,
>>but we store the form and form field metadata in Python objects (which
>>are stored in .py files currently, but in XML files in the version after
>>next). We also plan to decouple forms from tables somewhat, and to
>>roll-forward and consolidate data stored in previous versions of forms
>>in the latest version.
> 
> We mix both approaches.
> 
> We have tables for
> 
> a) versioned form templates + template metadata
> b) field definitions linked to templates
> c) form instances linked to form templates
> d) form field data instances linked to field defs
> 
> All audited, of course.
> 
> We then have Python classes that operate on types of form
> templates. I guess we could write a class to generically
> (albeit quite logically not efficiently) display some sort of
> GUI for *any* form definition.

Something like Traits? See
http://www.sauria.com/~twl/conferences/pycon2005/20050323/Traits%20-%20The%20Next%20Generation.html

> We can have specialized Python child classes implementing
> logic inherent to one particular form template version.
> 
> I do like Ian's suggestion of having form template (version)
> specific child classes of the generic GUI generation class in
> order to efficiently support forms where neede.
> 
> Anyone see any holes in this arrangement ?
> 
> 
>>The process of versioning and deploying
>>user-defined forms in a multi-user system is the hardest thing to get right.
> 
> Agree. The one horror that must be avoided by the very first
> implementation of our forms layer is that form templates may
> get overwritten and old data made inaccessible therefore. I so
> detest that and it happens in every single commercial EMR I
> have to use ...
> 
>>For storing small numbers of records, as in GNUmed,
> 
> Small numbers of records ? Where does large begin ?

For population health work, I arbitrary regard small as up to 500,000
records, medium as up to 5 million records, large as up to 25 million
records, very large as up to several hundred million records and
ginormous as more than that. We found that an EAV approach ran into
performance problems after a few hundred thousand case records - which
equates to perhaps 10 or 20 time that many records in the EAV data
table(s). Single user performance (with PostgreSQl as teh back end) was
OK, but simultaneous multi-user performance was woeful. But it was fine
in the 10 to 20 thousand (case) record range.

>>the more flexible
>>EAV (entity-attribute-value) model would be worth considering.
> 
> Given the above don't we do that although fleshed out quite
> a bit ?

So all the data is stored in one table, with one entity-attribute-value
triplet per row? That's the EAV model - although for storage efficinecy
reasons, separate tables are often used for each data type.

Tim C