Re: [Sks-devel] HTTP POST reverse proxy setup

[Brian Minton]

On Thu, Oct 06, 2016 at 10:14:47AM +0000, Kim Minh Kaplan wrote:
> Brian Minton wrote:
> > Kristian, is there a good way to easily check, e.g. with curl, the POST
> > capabilities of a reverse proxy in front of SKS?
> 
> Not that easy. But you can try something like this:
> 
> printf '\000\000\000\000\r\n' >/tmp/data.$$
> curl -so /tmp/hashquery.$$ --data-binary @/tmp/data.$$
> http://keyserver.example:11371/pks/hashquery
> cmp /tmp/data.$$ /tmp/hashquery.$$

Here's the result from a machine on my local net

laptop:~$  printf '\000\000\000\000\r\n' >/tmp/data.$$
laptop:~$  curl -so /tmp/hashquery.$$ --data-binary @/tmp/data.$$ 
http://192.168.1.9:11371/pks/hashquery
laptop:~$  cmp /tmp/data.$$ /tmp/hashquery.$$
laptop:~$ echo $?
0
laptop:~$  printf 
'\000\000\000\001\000\000\000\020\146\363\324\122\357\006\345\037\341\123\164\353\101\036\202\110'
 > /tmp/data.$$
laptop:~$  curl -so /tmp/hashquery.$$ --data-binary @/tmp/data.$$ 
http://192.168.1.9:11371/pks/hashquery
laptop:~$  echo "a83e9c29252cc63c56331270caa66e3af44251fe  /tmp/hashquery.$$" | 
sha1sum -c
/tmp/hashquery.10110: OK

And here's the result from a remote machine using ipv6:

remote:~$ printf '\000\000\000\000\r\n' >/tmp/data.$$
remote:~$ curl -so /tmp/hashquery.$$ --data-binary @/tmp/data.$$ 
http://keyserver.brian.minton.name:11371/pks/hashquery 
remote:~$ cmp /tmp/data.$$ /tmp/hashquery.$$
remote:~$ echo $?
0
remote:~$ printf 
'\000\000\000\001\000\000\000\020\146\363\324\122\357\006\345\037\341\123\164\353\101\036\202\110'
 > /tmp/data.$$
remote:~$ curl -so /tmp/hashquery.$$ --data-binary @/tmp/data.$$ 
http://keyserver.brian.minton.name:11371/pks/hashquery
remote:~$ echo "a83e9c29252cc63c56331270caa66e3af44251fe  /tmp/hashquery.$$" | 
sha1sum -c
/tmp/hashquery.6865: OK

And the result from the same machine using ipv4:

remote:~$ printf '\000\000\000\000\r\n' >/tmp/data.$$
remote:~$ curl -so /tmp/hashquery.$$ --data-binary @/tmp/data.$$ -4 
http://keyserver.brian.minton.name:11371/pks/hashquery 
remote:~$ cmp /tmp/data.$$ /tmp/hashquery.$$
remote:~$ echo $?
0
remote:~$ printf 
'\000\000\000\001\000\000\000\020\146\363\324\122\357\006\345\037\341\123\164\353\101\036\202\110'
 > /tmp/data.$$
remote:~$ curl -so /tmp/hashquery.$$ --data-binary @/tmp/data.$$ -4 
http://keyserver.brian.minton.name:11371/pks/hashquery 
remote:~$ echo "a83e9c29252cc63c56331270caa66e3af44251fe  /tmp/hashquery.$$" | 
sha1sum -c
/tmp/hashquery.6865: OK


I'm not seeing any errors.  However, I'm guessing curl isn't sending an
Expect: 100-continue header.  Let me see if I can force curl to do that:


remote:~$ curl -v -H 'Expect: 100-continue' -so /tmp/hashquery.$$ --data-binary 
@/tmp/data.$$ -4 http://keyserver.brian.minton.name:11371/pks/hashquery 
* Hostname was NOT found in DNS cache
*   Trying 174.59.255.142...
* Connected to keyserver.brian.minton.name (174.59.255.142) port 11371 (#0)
> POST /pks/hashquery HTTP/1.1
> User-Agent: curl/7.38.0
> Host: keyserver.brian.minton.name:11371
> Accept: */*
> Expect: 100-continue
> Content-Length: 24
> Content-Type: application/x-www-form-urlencoded
> 
< HTTP/1.1 417 Expectation Failed
< Content-Type: text/html
< Content-Length: 363
< Connection: close
< Date: Thu, 06 Oct 2016 12:51:48 GMT
* Server lighttpd/1.4.39 is not blacklisted
< Server: lighttpd/1.4.39
< 
{ [data not shown]
* Closing connection 0


So, there, I'm seeing the error.  I'm guessing that the script Kristian is
using is sending the Expect: 100-continue header, while the actual sks client
is not.  Otherwise, I don't know how I'd be able to gossip successfully with
other peers, which seems to be happening.

Let's see what happens when we try talking directly to sks instead of the
reverse proxy:

bminton.is-a-geek.net:~$ curl -v -H 'Expect: 100-continue' -so 
/tmp/hashquery.$$ --data-binary @/tmp/data.$$ -4 
http://127.0.0.1:11371/pks/hashquery
*   Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 11371 (#0)
> POST /pks/hashquery HTTP/1.1
> Host: 127.0.0.1:11371
> User-Agent: curl/7.50.1
> Accept: */*
> Expect: 100-continue
> Content-Length: 24
> Content-Type: application/x-www-form-urlencoded
>
* Done waiting for 100-continue
} [24 bytes data]
* We are completely uploaded and fine
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Server: sks_www/1.1.6
< Cache-Control: no-cache
< Pragma: no-cache
< Expires: 0
< Content-length: 405
< X-HKP-Results-Count: 1
< Content-type: pgp/keys
< Access-Control-Allow-Origin: *
<
{ [405 bytes data]
* Closing connection 0
bminton.is-a-geek.net:~$ echo "a83e9c29252cc63c56331270caa66e3af44251fe  
/tmp/hashquery.$$" | sha1sum -c
/tmp/hashquery.26533: OK

So, my reverse proxy (lighttpd) is definitely not standards compliant.
However, I'm hesitant to switch back to apache as my reverse proxy, since my
setup seems to be gossiping with peers.  


thanks,
-- 
Brian Minton
brian at minton dot name http://brian.minton.name
Live long, and prosper longer!
OpenPGP fingerprint = 8213 71DD 4665 CF4F AE20  2206 0424 DC19 B678 A1A9

signature.asc
Description: PGP signature