Re: [GNUnet-developers] commit 5d021c879 ("implement MTU calculation and adjustments")

[Christian Grothoff]

Hi Daniel,

Thanks for the review, comments below.

On 2/1/19 11:30 AM, Daniel Golle wrote:
> Hi Christian,
> 
> I just reviewed 5d021c879 ("implement MTU calculation and adjustments")
> and want to make some comments on it:
> 
> +  case AF_INET:
> +    base_mtu
> +      = 1480 /* Ethernet MTU, 1500 - Ethernet header - VLAN tag */
> +      - sizeof (struct GNUNET_TUN_IPv4Header) /* 20 */
> +      - sizeof (struct GNUNET_TUN_UdpHeader) /* 8 */;
> +    break;
> +  case AF_INET6:
> +    base_mtu
> +      = 1280 /* Minimum MTU required by IPv6 */
> +      - sizeof (struct GNUNET_TUN_IPv6Header) /* 40 */
> +      - sizeof (struct GNUNET_TUN_UdpHeader) /* 8 */;
> 
> This seems wrong to me, for several reasons:
> Ethernet MTU applies for both, IPv4 and IPv6 the same way.
> 
> Assuming MTU 1500 for IPv4 should then be assumed for IPv6 as well.

Nope. 1500 is indeed the Ethernet MTU, but it's not guaranteed for the
entire IPv4 Internet to be available.  But it is very common, hence it
makes sense as a default target size.  With IPv4, if we are wrong and
shoot too high, routers will fragment the packets for us, so all is well.

With IPv6, routers never fragment packets. So if we picked 1500 and are
wrong, we get _no_ communication at all. OTOH, IPv6 *requires* an MTU of
1280 by the layer 2 to be provided. So we are guaranteed to be fine
without fragmentation at an MTU of 1280.  Hence these starting values
are (likely) good choices (in the absence of measurement data showing
something else).

> Assuming the minimum of 1280 for IPv6 and not equally for IPv4 will
> obviously break IPv4 in situations where MTU (a property of Ethernet)
> is less than 1480 (eg. in case of using a VPN for WAN connectivity
> which can easily reduce MTU below 1480).

I indeed based my calculations that if the MTU is really << 1500, I'm OK
with fragmentation.

> VLAN tag doesn't affect MTU, it remains 1500 (unless you have a very
> broken dump switch sitting in between. Those do exist, but are rare
> and then people usually just don't use VLANs over them anyway).

Right, here I'm just extra conservative. I don't think the 4 bytes hurt
us badly, but that is debatable.

> In short: those assumptions don't hold true in reality. You will need
> to discover MTU for the path and there is no way around it. It's a
> very common mistake, and I saw it causing all sorts of weird problems
> in the past. Please don't hard-code any value there, we will regret
> that and it may work in 85% of typical setups and screw up people who
> are in less typical network environments.

Do you have a proposal how to do reliable MTU discovery even in
scenarios where we have to assume that ICMPv4 messages and UDP packets
are dropped in one direction? One of my design goals was that this
should work even if UDP traffic works one-way-only, and ICMP traffic is
blocked.  If we _require_ MTU discovery and/or bi-directional
connectivity to be available, that'll hurt us in some situations as well.

That said, I'm _not_ against extending the code to _attempt_ MTU
discovery and use a different MTU if we found it applies.  But this is a
_first_ (still somewhat incomplete) implementation that isn't even
supposed to have all the features. In particular, I would like this one
to be tested first in _simple_ networks, then develop integration tests
for _complex_ networks (like you describe) and _then_ add features to do
better in those.

In any case, if you worry that I wanted to specify that this is the MTU
calculation that everybody must always use, that's not the case. It's
just the current first calculation, we can certainly try to do better in
the future.

> Obviously there is a privacy aspect of it as well, which may justify
> hardcoding it to 1280 for **both** IPv4 and IPv6 -- as it will be only
> GNUnet doing that, this will then identify GNUnet users which is also
> not great.

Given that an adversary can easily tell apart IPv4 vs IPv6, I see no
privacy gain here in having the same MTU for both. As for identifying
GNUnet users, my design goal here was this: design the protocol in such
a way that there is no bit on the wire that is easily distinguished from
uniformly random. As a result, if other protocols adopted the same
standard, they would all be virtually indistinguishable.  But of course,
as long as only GNUnet does this, it'll always stand out a bit.

> I guess doing it the way it is done in popular protocols
> used over UDP is the best thing we can do, so please take a look at
> WebRTC and RTP implementations around, BitTorrent UTP may also be a
> good reference.

We should, in fact I also wanted to take another look at QUIC, which
seems closer to us in terms of purpose than WebRTC/RTP.  But as I said,
this is a _first_ draft, I've never even just run the code, I wanted to
get the overall rough setup right (most of the crypto, make sure the
APIs can work out, etc.), the performance fine-tuning (and more NAT
traversal support, etc.) is all expected to come later.

Help is of course always welcome, both in terms of concrete design
proposals, test scenarios or code ;-)