Re: Re: A more general question about curl

[Martin Schanzenbach]

Hi,

I already talked to Tim a year ago about this.
Unfortunately since GNU was not participating in GSoC this did not get
any traction. Neither me or Tim have (had?) the bandwidth to do this.

But in terms of requirements what we need first is what curl multi API
offers _including_ the curl_multi_fdset functionality that exposes the
file descriptors so we can use our own scheduler.

This would allow us to replace curl everywhere except the GNS proxy.

The _secondary_ requirement is that we can access the SSL backend and
metadata on the TLS handshake: 
https://git.gnunet.org/gnunet.git/tree/src/gns/gnunet-gns-proxy.c#n957

BR
Martin

Excerpts from hyazinthe's message of 2022-09-11 18:36:11 +0200:
> Hello everyone,
> 
> 
> wget2 community is hearing our conversation and is reaching out to us for 
> collaboration:
> 
> story - short:
> > On 10. Sep 2022, at 20:53, Tim Rühsen <tim.ruehsen@gmx.de> wrote:
> >
> > just found this issue: https://gitlab.com/gnuwget/wget2/-/issues/550
> >
> > Feel free to add the details needed for GNUnet.
> 
> story - medium:
> > On 10. Sep 2022, at 20:47, Tim Rühsen <tim.ruehsen@gmx.de> wrote:
> >
> > TL;DR the wget2 project is willing to help and a multi API may be easy
> > to implement. But someone has to write down the exact needs.
> 
> Full story, its rest - 
> https://lists.gnu.org/archive/html/wget-dev/2022-09/msg00008.html :
> > On 10. Sep 2022, at 20:53, Tim Rühsen <tim.ruehsen@gmx.de> wrote:
> > I try to explain what libwget does today and that it seems very straight 
> > forward to implement an API - and by the way - everybody is invited to do 
> > that :-)
> >
> > Libwget has several layers of abstraction of accessing the network stack(s).
> >
> > So you have indeed the synchronously, super simplified
> >   response = wget_http_get(...) like in example/http_get.c
> >
> > Then there is async HTTP API layer with more control over the details, see 
> > example/http_get2.c.
> >
> >   err = wget_http_open(&conn, url); // comes back immediately
> >   err = wget_http_send_request(conn, req); // comes back immediately
> >
> > resp = wget_http_get_response(conn); // waits until error/timeout or 
> > response
> >
> >   wget_http_close(&conn);
> >
> > While you could send several requests over a single connection, HTTP/1.1 
> > has issues with it. But it works fine with HTTP/2. In this case 
> > wget_http_get_response(conn) can be called in a loop, returning the 
> > finished response in the order they came in. You can also have as many open 
> > connections as you like - but what is missing, if I understood correctly, 
> > is an API that fetches the responses from more than one connection at once, 
> > like
> >
> >   resp = wget_http_get_response(conn1, conn1, ..., NULL);
> >
> > Now there is also a TCP+SSL API (used by the above mentioned high level 
> > functions). This API is works asynchronously. It is like
> >
> >   tcp = wget_tcp_init()
> >   ... set all kind of configurations to the 'tcp' handle ...
> >   err = int wget_tcp_connect(tcp, host, port) // returns immediately
> >   wget_tcp_write() // returns after timeout or immediately if no timeout
> >   wget_tcp_read() // returns after timeout or immediately if no timeout
> >   wget_tcp_close()
> >   wget_tcp_deinit()
> >
> > Internally, this API uses select/poll, but just uses a single 'tcp' handle.
> >
> > Now, what a "multi"  API basically would look like is e.g.
> >
> > a function wget_tcp_select(array of tcp handles, timeout) which can be 
> > called in a loop and which returns an array of "ready" tcp handles (ready 
> > for write or read, configurable per tcp handle).
> >
> >
> > For me it looks like this is straight forward to implement (depends on the 
> > details / requirements).
> >
> >
> > Internally, wget_ready_2_transfer(int fd, int timeout, int mode) in
> > io.c just needs a companion function that takes a list/array of fds.
> >
> > If there really is interest from the GNUnet community, why not open an 
> > issue at https://gitlab.com/gnuwget/wget2/issues to discuss the details and 
> > the needs. Once we agree upon the details, the implementation can be done 
> > by anyone - whoever likes to pick it up.
> 
> 
> Greetings,
> Bastian Schmidt

signature.asc
Description: PGP signature