Re: [Gnumed-devel] how to do intended_reviewer / requestor? (was: HL7 test results import.)

[lkcl]

allo jim, let me deal with each part separately, in different posts.  this
one the proposed "import_log".

Jim Busser wrote:
> 
>> i'm trying to parse 2) ...
>> what's occurring to you is...
>> an HL7 message-presentation and message-editing widget...
>> you'd need a list of segment-identifiers (50 or so) etc...
>> would need field identifiers as well (close to 1000...
>> ...
>> do i have that right?
> 
> Not quite... I was thinking only a few fields.
> 
> We *still* need to define the hinted-at import_log table:
> 
> - primary key
> - importer_name (name of the script)
> - imported_source (filename)
> - imported_when (datetime)
> - error_info (string, default NULL)
> 
> error_info is contemplated to do nothing more than to make a record of any
> attempted-to-be-imported files which proved to be invalid in terms of the
> expected file or message *structure* e.g. in this case HL7. (A mismatch in
> the expected *content* would not be flagged at this level but rather at
> the level of any affected messages).
> 

ok - that's sensible.  that i can agree with.



> 
> error_info would simply record the first instance {line, message,
> position} of of parser failure
> 
> "Structurally invalid" could be identified during a "read only" script
> phase, or in the course of "live processing"/ It is just that if an
> invalid structure gets encountered *after* one or more message-results had
> already been imported, all of these message-results are going to have to
> be "reclaimed" based on their origin from the found-to-be-defective file.
> Therefore the file should be read-written into a staging table and
> determined to be non-defective (else the just-imported messages purged)
> before the results would get written into the patient clinical tables.
> 

these are "parser errors" - syntax and grammar errors.  yes, absolutely, i
agree that there needs to be a "reject due to corrupted data" log.



> 
> 
> An editing widget would need only a few fields
> 

to do with _parser_ errors, yes.

I think the above (with or without refactoring) is sufficiently general to
achieve what you suggest (?)



if i was talking specifically about parser/grammar errors, yes, absolutely
it would.  it hadn't explicitly occurred to me to mention that: this is
still all very new to me :)  so, yes: one line of HL7 data gets corrupted /
truncated, or it's cut/paste incorrectly from an email and newline
characters are inserted, that's the sort of thing that yes, you need to know
the line number of the error, you *don't* need to know anything about the
*content* of the line, just "this line was wrong, wark wark".  an
experienced HL7 operator can then be called in, slap the wrist of the
less-experienced staff member, tell them not to use cut/paste again but to
make sure that they use email attachments, to preserve the data integrity,
everything is hunky-dory.

what i was actually referring to was "meaning" errors.  "informational"
errors.  _that's_ the point at which you need to start understanding the HL7
fields themselves.  _that's_ the point at which you need to be able to
present the HL7 data itself, warts and all.  _that's_ the point at which you
need to start "understanding" 100s to 1000s of fields, and _this_ situation
is my primary concern.

more in the next message.

l.

-- 
View this message in context: 
http://old.nabble.com/Re%3A-how-to-do-intended_reviewer---requestor--%28was%3A-HL7-test-results-import.%29-tp29385122p29386937.html
Sent from the GnuMed - Dev mailing list archive at Nabble.com.