Re: [Gnumed-devel] optimization for progress notes retrieval?

[Karsten Hilbert]

On Tue, Sep 12, 2006 at 06:20:22AM +0800, Syan Tan wrote:

> a while back , a threaded draft solution was offered for a slow tree browser ?
The main reason it is not yet in the tree is that I couldn't
quite get my head around it in time for the release.

A solution for the slowness of the tree browser initial load
is needed, though.

My plan has two steps:

1) make the emr tree browser load on demand,
   IOW when opening a branch

This may already be fast enough. A cache can be used for
repeat access.

2) eventually do use async SQL fetching (in general, not
   just for the tree)

Recently there was a thread on wxPython-users about
DelayedResult which does just that. It seemed simple and
straightforward to use. I think we should try that one.

> - I was thinking some sort of callback interface might work,
This is how DelayedResult works. It can post results to a
callback function from a thread via wx.CallAfter().

> like the python twisted framework.
I have read people saying Twisted and wxPython don't go
together too well. While good-looking I'd think it's too big
for our current purpose.

> e.g. the browser and the journal manage their
> own threaded callback functions, which update their respective gui,
> and gmClinicalRecord has a callback register function, and has an incremental
> fetch, which fetches by health issue/episode , and returns e.g. an iterator
> whose next() function fetches the next episode.
That's surely partially how it should be. I particularly
like the iterator/generator approach as we just might be
able to make it fit in with a database cursor.

The larger plan for the EMR tree browser is this, however:

The current exporter used by the EMR tree is probably too
complicated.

The episode node level must display the aggregated narrative
for this episode on the right hand side below the statistics
part.

The right hand side needs to be displayed in HTML. Richard
worked on this a bit and has it fully working in his
previous program. I tend to favour a (Cheetah) template for
this. There need to be templates for the various node levels
of the tree. The templates will be stored in the database. I
am still undecided as to whether to have the database itself
fill in the templates or whether to do it client-side.
Probably the latter. Either way it may make sense to
pre-process the data server-side into dicts of strings which
can then easily be used for filling in templates
client-side. This approach would probably bypass the cache
inside cClinicalRecord but that may be OK for read-only
narrative data which is rarely if ever changed.

I mean, after all, per-patient how much narrative data can
it really be ?  80 characters per line, 60 lines per page,
100 pages makes 480 000 characters, what, less than 512 KB ?

We should probably transfer the entire narrative structure
in one go slightly pre-processed. The emr journal view
already goes in that direction and its fast, too (apart from
the parent-child table scan issue). I will add an
appropriate view for that.

Karsten
-- 
GPG key ID E4071346 @ wwwkeys.pgp.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346