[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Let BIO also handle already allocated buffers
From: |
Christian Grothoff |
Subject: |
Re: Let BIO also handle already allocated buffers |
Date: |
Sun, 19 Apr 2020 00:57:56 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 |
Hi Alessio,
I'm a bit torn about your proposal. It makes sense for BIO to support
to-memory serialization instead of just to-file.
However, there are two issues. First, your patch introduces a new type
'struct GNUNET_BIO_MemoryHandle'. I think to avoid having to duplicate
all of the individual BIO-read/write functions, the 'in memory' vs 'in
file' should just be a different function to create the same 'struct
GNUNET_BIO_ReadHandle' or 'GNUNET_BIO_WriteHande' we already have, just
this time backed by just buffer instead of a file. [This is what for me
makes the patch not really ready to merge at an API-level.]
Second, as I've mentioned to some people before but certainly not to
you, I've considerd the BIO API to be an abomination (for at least 5
years now) that we should likely re-do completely, and so I'm VERY
hesitant to see more code use that horrible style ;-).
Instead of (to take an example from hostlist):
((GNUNET_OK == GNUNET_BIO_read_string (rh, "url", &uri,
MAX_URL_LEN)) &&
(NULL != uri) &&
(GNUNET_OK == GNUNET_BIO_read_int32 (rh, ×_used)) &&
(GNUNET_OK == GNUNET_BIO_read_int64 (rh, &quality)) &&
(GNUNET_OK == GNUNET_BIO_read_int64 (rh, &last_used)) &&
(GNUNET_OK == GNUNET_BIO_read_int64 (rh, &created)) &&
(GNUNET_OK == GNUNET_BIO_read_int32 (rh, &hellos_returned)))
I would prefer to see something like:
struct GNUNET_BIO_ReadSpec spec[] = {
GNUNET_BIO_read_string (&uri, MAX_URL_LEN),
GNUNET_BIO_read_uint32 (×_used),
GNUNET_BIO_read_uint64 (&quality),
GNUNET_BIO_read_abs_time (&last_used)
GNUNET_BIO_read_end()
};
if (GNUNET_OK !=
GNUNET_BIO_run (bio, spec))
...
more similar to the gnunet{pq,sq,my,json} APIs we have that use similar
'spec' arrays to serialize from/to database and or JSON.
bio = GNUNET_BIO_open_file(fd); // and
bio = GNUNET_BIO_open_buffer(buf, buf_size);
would then both of course make sense to have.
Note that it the above construction could likely even be made to support
arrays in a reasonably sane way:
GNUNET_BIO_write_array (array_len,
array_base,
sizeof (*array_base),
GNUNET_BIO_write_uint32 (NULL+offset),
GNUNET_BIO_write_uint64 (NULL+offset),
GNUNET_BIO_read_end())
where the 'write_array' would first write the
'array_len', and then iterate over the 'inner'
specification (given as varargs!), adding to
each 'offset' the array_base + i*sizeof(*array_base).
With a bit of macros the syntax could even be made pretty nice ...
OTOH, I'm also wondering if we're still not re-inventing ASN.1 badly here...
My 2 cents
Happy hacking!
Christian
On 4/18/20 4:34 PM, Alessio Vanni wrote:
> Hello,
>
> aside from working on files by buffering I/O, the BIO library also does
> *formatted* I/O, like serializing strings or integers.
>
> I found myself in a situation where my client and my service are
> communicating using variable-length messages (containing a sequence of
> variable-length strings, mostly), so I'd have to parse the messages "by
> hand" using some kind of serialization. Since BIO has this formatted
> I/O, I thought I could "offload" this de/serialization to the library,
> but it works only on files and writing the message to a file and reading
> it back again is suboptimal.
>
> As such, I hacked together a set of functions in BIO to work with
> already allocated buffers, such as when dealing with messages received
> from a message queue. The patch is attached with this mail. Right now
> the API is written to cover my needs, so it might be lacking in some
> parts.
>
> Thanks,
> A.V.
>
signature.asc
Description: OpenPGP digital signature