[GNUnet-developers] GNUnet VPN/EXIT performance over wifi and loopback

[Daniel Golle]

Hi Christian,

I'm hoping to really reach everyone interested this time in this
email ;)

> Hi!
> 
> The cipher is a variant of Axolotl, so repeated ECDHE on Curve25519,
> SHA-512 key ratcheting for each message, and Twofish+AES for symmetric
> encryption.  This (kind of) encryption is done TWICE, once at the link
> layer, and then also end-to-end.

Thanks for the info, that's the precise answer to the question I was
hoping for.

> 
> Comparing loopback performance of an encrypted system with cleartext is
> IMO totally useless -- you're just measuring the CPU speed for the
> ciphers, and in our case they're rather expensive.  Not to mention on a
> real network, I'd imagine bandwidth/latency to be the critical factor,
> not CPU speed.

Well, it helped to get a general impression of the performance to be
expected, especially when comparing with the results below.
(the results on an actual MIPS SoC look very similar to what I sent
before)

So these are the results when running iperf3 between two routers
connected via WiFi (IBSS mode).

address@hidden:~# iperf3 -c 10.82.1.2
Connecting to host 10.82.1.2, port 5201
[  4] local 10.82.2.2 port 53015 connected to 10.82.1.2 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.01   sec  2.30 MBytes  19.2 Mbits/sec    0   49.5 KBytes       
[  4]   1.01-2.00   sec  2.68 MBytes  22.6 Mbits/sec    0   72.1 KBytes       
[  4]   2.00-3.00   sec  2.46 MBytes  20.6 Mbits/sec    0   77.8 KBytes       
[  4]   3.00-4.00   sec  4.42 MBytes  37.1 Mbits/sec    0    112 KBytes       
[  4]   4.00-5.01   sec  3.88 MBytes  32.4 Mbits/sec    0    124 KBytes       
[  4]   5.01-6.00   sec  4.53 MBytes  38.2 Mbits/sec    0    139 KBytes       
[  4]   6.00-7.00   sec  5.12 MBytes  43.0 Mbits/sec    0    214 KBytes       
[  4]   7.00-8.00   sec  6.67 MBytes  56.0 Mbits/sec    0    277 KBytes       
[  4]   8.00-9.02   sec  6.88 MBytes  56.3 Mbits/sec    0    277 KBytes       
[  4]   9.02-10.00  sec  5.88 MBytes  50.6 Mbits/sec    0    277 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  44.8 MBytes  37.6 Mbits/sec    0             sender
[  4]   0.00-10.00  sec  44.5 MBytes  37.3 Mbits/sec                  receiver

iperf Done.

Now with gnunet-vpn in between the two (connected over the same single
wireless hop as above, using UDP transport):

address@hidden:~# iperf3 -c 10.11.155.173
Connecting to host 10.11.155.173, port 5201
[  4] local 10.11.10.1 port 42761 connected to 10.11.155.173 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  42.4 KBytes   347 Kbits/sec    0   14.1 KBytes       
[  4]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    6   9.90 KBytes       
[  4]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   9.90 KBytes       
[  4]   3.00-4.00   sec  22.6 KBytes   185 Kbits/sec    0   12.7 KBytes       
[  4]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   12.7 KBytes       
[  4]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   14.1 KBytes       
[  4]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    0   17.0 KBytes       
[  4]   7.00-8.00   sec  55.1 KBytes   452 Kbits/sec    0   21.2 KBytes       
[  4]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   25.5 KBytes       
[  4]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   29.7 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec   120 KBytes  98.5 Kbits/sec    6             sender
[  4]   0.00-10.00  sec  66.5 KBytes  54.4 Kbits/sec                  receiver

iperf Done.

Given the loopback results, I was expecting something better here as
well :(

The only hint of anything potentially going fundamentally wrong are
repeating error messages on the log:
Aug 08 17:35:51-003724 cadet-p2p-5380 ERROR  core wait time 1133613 µs > 1 
second

As the throughput is very bursty, my first assumption was that buffer
bloat is also one of the problems we are hitting here.
Looking at the results, Dave Tath suggested to simultanously measure
both, bandwidth and latency in order to detect bufferbloat.
Hence it would be nice if gnunet-vpn could carry at least basic
ICMP (echo-request, echo-reply) in addition to the setup UDP and TCP
redirects.


Cheers


Daniel



> 
> -Christian
> 
> On 08/04/2015 11:18 PM, demos wrote:
> > Hello from BattleMesh!
> > 
> > Here are first results, testing iperf on localhost (MIPS Malta):
> > 
> > direct:
> > [ ID] Interval           Transfer     Bandwidth       Retr
> > [  4]   0.00-10.02  sec   776 MBytes   650 Mbits/sec    0             sender
> > [  4]   0.00-10.02  sec   775 MBytes   649 Mbits/sec
> > receiver
> > 
> > over vpn-gnunet/exit-gnunet:
> > [ ID] Interval           Transfer     Bandwidth       Retr
> > [  4]   0.00-10.00  sec  1.36 MBytes  1.14 Mbits/sec   16             sender
> > [  4]   0.00-10.00  sec  1.18 MBytes   990 Kbits/sec
> > receiver
> > 
> > 
> > I expected performance to be bad... But 1:600 is worse than I thought.
> > 
> > Probably some profiling can improve this, maybe stuff like alignment
> > problems also eats more performance.
> > gnunet-developers: Which stream chipher is used between VPN/EXIT
> > endpoints (people asked, I don't know...)?
> > 
> > 
> > Cheers
> > 
> > 
> > Daniel
> 
> 
>