Re: [Gnumed-devel] Specs (was: Gnumed brochures)

[Ian Haywood]

I've been quietly watching this flamewar with a mixture of amusement and horror.
I see good points on all sides, let me throw in my $0.02 (which hopefully has a 
anti-pryetic effect)

Before specification & design comes an even eariler phase: requirements.
There is in fact a large literature around this, much time and money has been 
wasted in
software engineering because of poor "requirements engineering" We may console 
ourselves
that are not the first, nor the last project to suffer this. The advantage of 
FOSS
is our timelines and labour-budgets are basically infinite ;-))

And it's here problems arise. Obviously in Australia we have different, much 
broader requirements.
It's simply the case that a prospective program does (almost) everything -- or 
it's useless.
>From our perspective as FOSS developers that's very bad. Linux, for example, 
>started as a very
crappy basic system, that was only useful to kernel hackers. However, such a 
community existed, there were able
to use it, play with it, and help with development.

Do we have such a community?: yes, in potential, but the core "kernel" must 
exist, being much bigger than Linus' first release of
his kernel, local teams can then add on modules to suit local requirements, 
which is of course what we've spent six years trying to build.
It's just really hard.

Karsten Hilbert wrote:
> The way I see the current state of GNUmed is that we have a
> (trying to be) strategic backend with a tactical (in many
> parts) frontend thereto.
Glad to read this.

On the whole, I'm happy with the backend too. SQL is so high-level it's 
practically like UML anyway.
It is sometimes a struggle to get Karsten to accept our required changes, but he
does eventually and I can hardly argue that his contemplations acts as some 
sort of bottleneck. He is a good guardian of the
backend, and that's an important role.

The frontend is harder. I don't think anyone disagrees that it hasn't been 
managed as well. In AU we need "the kitchen sink": a
very complex user interface where users need to enter and do intricate
manipluation on highly structured data at high speed. It's easy to get it 
wrong, as the various proprietary players have. The
only way it can work is if the frontend adheres to a unifying 
philosophy/paradigm/whatever where there is a common system across all modules.
To some extent, it doesn't matter what the paradigm is, so long you choose it, 
specify it and actually stick to it.

So, Richard and I need to write our own frontend, (which we are quietly doing 
already) Gnumed already supports completely different pluggable
UI "personalities", no-one has ever had a problem with that.

So where's the problem? I have to say it's the business layer. Personally I 
find coding it causes serious
posterior pain. I find it overly complex, I'm (still) really frustrated and 
confused by the prohibition
on python exceptions, (that's the only thing where I can claim "confusion": I 
was around for most of the other design decisions
and either participiated or raised no dissent) However, now I find it slows my 
development speed almost to a halt.
It's also slows the client down to an unusable speed. I am aware of the CS 
dictum that optimisation comes last, but in our
situation client speed is often make-or-break for usability. (and I've often 
speculated about secret German CS tech,
given you guys don't seem to notice!)

Plus, it's so complex it puts a steep learning curve on new developers, so far 
(AFAIK) only Carlos has ascended
that incline, much kudos to him.

This is something where I would vote for a putting a bomb under it and starting 
afresh, problem is that means
blowing up the GUI layer too, (although some large fragments could be re-used) 
so I can't, in fairness, ask this of people
who can use the current codebase on Real Patients today. Nevertheless, others 
must understand our position:
gnumed is a paperweight unless we can massively increase features, that means 
the internal coding process needs to be more highlevel, and
less 'manual'

Ok, maybe more like $2-worth now, please bear with me.

I think we can have a better middleware. This is largely a 'technical' matter 
and in some ways orthogonal to the problems
of requirements/specs/etc. But historically I think it's where we went wrong, 
back in 2000-1 we were running along merrily
with coding and designing basically in parallel, then things came unstuck as 
development basically halted for 18 months
as we had no agreement on how to actually shift data from backend to widget and 
back. On unfortunately is wasn't something we
can say "each to his own": we needed to agree or we might as well have been 
writing separate clients.

Our current middleware comes unstuck because it does everything twice: 
middleware objects need to be manually defined
even though 90% they correspond to stuff in the backend. They don't do 
error-handling well (I want to be able to
write error-handling *once* and never worry about it again, I don't want to 
check for errors *every* *single* *time*
I call the middleware layer!), and inserting new rows is harder then it should 
be. Working with 'advanced' postgres
features like stored procedures is harder than it should be, and so 
under-utilised.

I'm sure you've guessed where this is leading. I have written a basic extension 
to my python-postgres adapter to allow
access to the database 'pythonically', so long as certain naming conventions 
are adhered to (mainly ones that we
already adhere to)

example.

l = sql.find_patient ("Smith") # runs the backend prepared query find_patient, 
returning row instances
p = l[0]
a = p.allergies # runs select * from allergies where fk_patient = <pk of 
patient>, which is cached

del a[0] # runs delete from allergies where pk = <the pk of that row>

a.new (reaction="rash", drug="penicillin") # insert into allergies (reaction, 
drug, fk_patient) values ("rash", "penicillin", <pk of patient>)
a.do_something (foo) # calls the prepared query do_something (<pk of allergy>, 
foo)
p.update () # commits the above changes in a single transaction by traversing 
the local object tree.

all this without defined specific "allergy" and "patient" classes, however such 
classes can be defined and will be used automatically if you
want some client-side functionality.

class allergy (PgRow):

        def do_something (self, ....):

but we don't have do do any "bureaucracy" when defining a new business layer 
object.

In many ways this is the culmination of what the current business layer has 
been evolving towards for some time,
the biggest difference this that it throws error exceptions, which simplifies 
things a lot.

the code is in gnumed/test-area/ian/purepg.py

What am I proposing? To use this or something like it instead? Of course that's 
what I'd like
but I'm realistic, that the other coders seem happy with the status quo and 
aren't facing the
same tasks Richard and I are.

Ian