[Gnumed-devel] Re: how to do intended_reviewer / requestor? (was: HL7 test results import.)

[Jim Busser]

onto list...

On 2010-08-07, at 1:55 PM, Luke Kenneth Casson Leighton wrote:

(at the end of your post)

> * encounters are marked "lab data import"
> * episodes are marked "lab data"
> * lab_requests are created or merged
> * test_data is created or merged (based on the unit type)
> 
> _everything_ succeeds, _all_ data goes in... except... err not if you
> can't find a patient or a doctor record.
> 
> actually, the last few patches, they do succeed :)
> 
> so, now, everything is identifiable by the words "lab data" in one or
> more tables.
> 
> thoughts?

(near the beginning of your post)

> i'm trying to parse 2) ...
> what's occurring to you is...
> an HL7 message-presentation and message-editing widget...
> you'd need a list of segment-identifiers (50 or so) etc...
> would need field identifiers as well (close to 1000...
> ...
> do i have that right?

Not quite... I was thinking only a few fields.

We *still* need to define the hinted-at import_log table:

- primary key
- importer_name (name of the script)
- imported_source (filename)
- imported_when (datetime)
- error_info (string, default NULL)

error_info is contemplated to do nothing more than to make a record of any 
attempted-to-be-imported files which proved to be invalid in terms of the 
expected file or message *structure* e.g. in this case HL7. (A mismatch in the 
expected *content* would not be flagged at this level but rather at the level 
of any affected messages).

error_info would simply record the first instance {line, message, position} of 
of parser failure

"Structurally invalid" could be identified during a "read only" script phase, 
or in the course of "live processing"/ It is just that if an invalid structure 
gets encountered *after* one or more message-results had already been imported, 
all of these message-results are going to have to be "reclaimed" based on their 
origin from the found-to-be-defective file. Therefore the file should be 
read-written into a staging table and determined to be non-defective (else the 
just-imported messages purged) before the results would get written into the 
patient clinical tables.

An editing widget would need only a few fields (not 50-150) and these could 
simply added to the unmatched table:

fk_import_id (foreign key to the source file, as referenced in import_log)

message_content (the raw, unparsed message NOMODIFY)

automatch_result (suggest the script populate this with one among:)
        matched <-- to patient and provider; invisible to user; proceed to 
auto-create
        patient? <-- await manual matching *or* approval of creation
        provider? <-- infrequent, unusual routing *or* a praxis SNAFU
        ?? <-- rare (but possible) major source SNAFU

patient_identifiers (parser populates using HL7 PID 001 … 013)
        --> e.g. ||||SMIFF^ARFUR||19450928|M|||^^SURREY^^BC||585-1510

provider_identifiers (parser populates using HL7 OBR 016 | OBR 028)
        --> e.g. 22333^MEDIC^IAN^TEST|00000^NEWDR^JAMES~H09999^DISNEYLAND^CLINIC
        (note: ORC 012 could be used in place of OBR 016 as these are the same)

gm_patient (for widget-assisted, user guided matching)
        --> e.g. dem.identity.pk

gm_provider (for widget-assisted, user guided matching)
        --> e.g. dem.staff.pk

gm_flag_unmatchable (boolean, default NULL)
        --> user sets to TRUE when appropriate

gm_user_comment (string, default NULL)
        --> user may input a comment

> if so - then basically, this would be a mini-project on its own, and
> i'm not convinced of its value... and if you're going to present "lab
> data", why not spend the time creating a gnumed lab_request and gnumed
> test_results viewer, instead of an HL7-specific viewer?

I think the above (with or without refactoring) is sufficiently general to 
achieve what you suggest (?)

> also, by simply "auto-creating" a tag-identifiable patient and
> "auto-creating" a tag-identifiable doctor (as staff) in cases where
> automated disambiguation is not possible, the import _will_ succeed.
> 
> everything else is on auto-create, and goes down as "lab data", so
> why not patients and doctors too?

Good question. Short answers:

1) when patients did not *seem* to exist,
        - they may already exist
        --> in this case, the auto-created persons will have to be merged
        - or, even when they did not yet exist, they should sometimes not be 
created
        --> therefore better that creation be user-approved
                --> it can then be done on the next iteration

2) when providers did not seem to exist
        - something is usually wrong

Doctors are cautious (hopefully you would prefer that they be this way <grin>) 
so let me offer some prototypical, conservative doctor-responses to some 
scenarios, and see to what extent programmer's rebuttals can "win over" the 
arguments:

******************************
1) Praxis downloads a file containing messages on multiple patients NOT 
EXISTING in the praxis, ordered by or copied-to NO PROVIDERS in the praxis. On 
surface, this sounds like it should have to be some colossal mistake, but it 
could be *possible* where some file got written by the path lab into the wrong 
inbox, say maybe as a result of a corrupted index. In such a scenario, it would 
be *far* more sensible to reject the import file (i.e. batch-tag all its 
imported but unmatched messages for deletion) instead of having to find inside 
GNUmed multiple auto-created patients and provided who really never ought to 
have been created.

--> Therefore, for messages in which no provider can be matched, do not 
auto-create patients.
--> Praxis "IT support" may need to delete all messages that were imported into 
the staging table from this source file

******************************

2) Praxis downloads a file containing messages on multiple patients who are 
either auto-matched or manually-matchable to patients in the praxis, BUT no 
matching provider exists. This can happen in two situations:

a) the praxis imported the results for this provider *without* the praxis 
administrator having created the staff (via GNUmed GUI). This should be a 
RARE-IF-EVER occurrence but, even if it did happen,

--> it's better that the administrator simply create the missing provider. 
Simply export any unmatched records, re-run through the importer, and these 
records will then auto-resolve.

b) the ordering provider directed (or the patient requested that) a copy go to 
the praxis, but not to a specific provider within the praxis. This assumes that 
the praxis may have its own identifier, which may or may not be true for any 
path lab system. Supposing that this is possible,

--> the staff must decide which provider should inherit responsibility

******************************

3) Patient is not matchable, but a provider is matchable.

Two main possibilities here:

a) a praxis provider was incorrectly inputted at the lab
        å(the message should have gone to someone else; I have gotten these!)
        … we do not want to create this patient
        … we want to reject them, with a comment in the unmatched record

b) praxis provider *was* correctly identified; in this case, the patient was:
        … existing already in praxis, but was a failed match, *or*
        … was cared for outside the praxis (but still reasonable to create) 
and/or
        … is pending to join and be cared for in the praxis (still reasonable 
to create)

So 2a really only needs the affected messages to be reprocessed,
Whereas 2b and 3b need the user-inputted data to "improve" the processing of 
the messages on the next iteration.

So what we need to decide is:

i) which messages to "reprocess"?
        anything that was modified after creation?
        extend to include all messages from any one source file?

ii) how to incorporate user-inputted data into an "improved" 
(no-longer-original) message?

-- Jim