[Gnue-dev] Re: DataObjects.txt

[Jason Cater]

On Sunday 23 December 2001 08:24 am, Neil Tiffin wrote:

<snip>
> I have a list of mail prospects say 2 million.  I want all of the
> ones with a last name of "smith", so I fire off a selection from the
> gui.  Now the GUI is designed to display the name and address of all
> records returned.  Since this list contains the names of the
> worldwide Smith Family Registry the query returns 1 million Smiths.
>
> Up to this point the ResultSet is still on the server (whatever that
> is, SQL Server or GEAS lets say).
>
> Now I have to get the name and address (which are in the ResultSet)
> back to the GUI (the space allocated on the GUI screen is for 15
> records).
>
> I see two ways:
> 1) use the get single record API calls already defined.  To display
> the first 15 names the GUI made 15 calls to nextRecord() and put them
> in a table on the screen.  That works, except (in GEAS/CORBA) I have
> to marshal at least 15 CORBA objects (slow).  However, If I want to
> skip and go to the middle of the list or page down in increments of
> 15 records it seems a little kludgy.  Because I can only iterate the
> list by calling nextRecord(). So to bypass 500,000 records I have to
> marshal 500,000 CORBA objects using nextRecord() and throw them away.
> Hopefully I have missed something and this is not the case.
>
> 2) define a way to pass subsets of the ResultSet back to the GUI.
> The getNRecords(startAt=0, number=1) API was to solve the limitations
> in (1). Since it returns a list, only one CORBA object would be
> marshaled and the GUI could skip all over the place and return as
> many records as the GUI/client wants.  Also Reports could get a page
> of data at one time instead of having to iterate through the
> nextRecord() logic each time.
>

Neil, 

I think I understand your point.  Let me delve into the ResultSet internals 
for a moment to clarify a few things.

Each database driver defines its own ResultSet... so we have an 
OracleResultSet, GEASResultSet, PgResultSet, etc.  These are transparent to 
the client as they all define the same set of external methods.  I bring this 
point up because it is important to understand that the underlying ResultSet 
knows how the underlying database or middleware backend works. 

The nextRecord()-type methods are designed for the clients to use against 
their internally maintained ResultSet - they do not necessarily correspond to 
the method used by the ResultSet to obtain data from the backend (in your 
case, the CORBA calls to GEAS.)

With the relational drivers (and I'm sure this would not be different with 
the GEAS driver), the ResultSet supports a "pre-fetch cache size" that the 
client can set. The ResultSet internals signal to the backend (when the 
backend supports such a mechanism) what the pre-fetch size is, and each time 
rows are retrieved from the backend, a chunk of rows of this size is 
returned.  For your example, we would set the prefetch size to 15 (since 15 
rows are displayed at a time). The client's ResultSet would tell the backend 
to return 15 rows at a time.  

The client would still make 15 calls to nextRecord() to fill its screen, but 
its underlying ResultSet only make 1 call to the backend to get those 15 
rows.  The 15 nextRecord() calls all happen locally. Now, when the 16th row 
needs to be displayed, another chunk of 15 rows is retrieved from the 
backend; the 17-30th rows are cached before they are needed.  

You also mention jumping to the middle of a ResultSet or using Page-Down to 
skip records.  This is an area we need to look at.  We do have a gotoRecord 
or jumpRecord function (I forget the name) that lets you jump ahead to record 
xxx; also, the ResultSet API does not state that if you are looking at 
records 30-45, records 1-29 must be loaded. 

It does, however, usually happen this way with other backends simply because 
their cursors (or whatever mechanism they use) do not support jumping around. 
To look at record 30, we must first iterate through records 1-29.  It would 
be nice if GEAS supported a jump-ahead cursor -- this would be yet another 
advantage of using GEAS. 

I *think* this is the functionality you are referencing.  So, I'm not sure of 
the significance of a getNRecords function in the ResultSet API; however, the 
GEAS CORBA interface certainly needs a similar mechanism for the ResultSets 
to call. Otherwise, you are correct that 15 CORBA calls would be needed.

Does this help?  By the way, Merry Christmas. 

-- Jason