[Chicken-hackers] Problem with OpenSSL and file descriptors (sendfile)

[Peter Bex]

Hello Chicken hackers,

I've traced an issue with Spiffy in HTTPS mode to a nasty interaction
between OpenSSL and sendfile.

For those who don't know sendfile: it is a Chicken egg that attempts
to push data from an input port to an output port as fast as possible.
It can use the Linux-specific sendfile() call if available, or an
mmap()-based implementation, and if those aren't possible it can fall
back to a simple buffered read/write loop in pure Scheme.

In order to get the fastest performance, sendfile extracts the file
descriptors from the given ports if available and uses them in the
call to those underlying mmap/sendfile POSIX functions.

Now, it turns out that openssl's ssl-accept stores the raw file
descriptors it receives from the underlying call to tcp-accept in
the input and output ports it creates.  This means that when you use
the nice Scheme port abstraction everything will be fine; it uses
SSL_write() when you write to the port, causing the data to be wrapped
in SSL protocol stuff.  However, when you extract the descriptor using
port->fileno, you get the raw TCP descriptor and when you write
something to that, you screw up the SSL protocol conversation.

Now, I commented out the setslot lines in openssl.scm and sure enough,
sendfile on a SSL port works fine again.  It's of course not able to
use the fastest implementation because the output needs to go through
openssl, but at least the abstraction doesn't fail and Spiffy can use
HTTPS transparently without a lot of port checking cruft.

However, this breaks many other things; for example, when a request
comes in Spiffy logs the incoming request with the IP address of the
remote side.  This uses tcp-addresses, which expects a TCP port with
a file descriptor slot.

I think the clean way to solve this is not to use the file descriptor
slot as this isn't supposed to be written to with SSL, but some other
slot (?), and then provide ssl equivalents to all TCP operations that
would no longer work when we do that.  This has already been done
for the operations on listeners.

I don't know if that's possible; how to know what slot is valid and
available for these ports, and that no other code uses this slot?

Cheers,
Peter
-- 
http://sjamaan.ath.cx
--
"The process of preparing programs for a digital computer
 is especially attractive, not only because it can be economically
 and scientifically rewarding, but also because it can be an aesthetic
 experience much like composing poetry or music."
                                                        -- Donald Knuth