Re: sqlite3

[Lars Ingebrigtsen]

Eli Zaretskii <eliz@gnu.org> writes:

>>From what I see there, you completely ignore the text encoding issue.
> SQLite needs UTF-8, but our internal representation is a superset of
> UTF-8, so we need to decide what to do with the stuff that's beyond
> UTF-8.  Does SQLite check this and return an error indication if the
> byte sequence we feed it is not a valid UTF-8?  If it does check that,
> we can rely on it, but we need to signal an error with a clear
> explanation when it detects something like that.  Alternatively, we
> should check this ourselves before calling SQLite, and signal an error
> on our own.

Good catch.  😀  I was scratching my head about that for about ten
minutes -- did I forget to put a "POSSIBLY FIXME" in there?

SQLite does expect UTF-8, but as far as I can tell, it doesn't really
check anything.  So the current code puts our private binary
utf-8-private-plane-encoded things into SQLite -- and gets it back as
is.  (See the sqlite-char test for this stuff.)

That is, 

"insert into test2 (col1, col2) values ('fó\x82o', 3)"

and

"insert into test2 (col1, col2) values ('f\x82o', 3)"

both round-trip without any problems as is.  Perhaps SQlite just sees it
as a byte string?

So I wasn't quite sure what to do here.

> One other comment is the use of fixnums for integer values.  Since
> SQLite can use the full 64-bit width, I guess we need to use bignums
> when the value exceeds a fixnum?

Yup.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no