Re: [Bug-wget] the libidn problem

[Daniel Stenberg]

On Tue, 30 Jun 2015, Ander Juaristi wrote:

> the library user (me, us, in this case) doesn't have to know anything about 
> UTF-8, so we should rely on the library for everything UTF-8-related.

I fully agree with this and I will stand by this rule. That's why I sent the 
"security notice" pointing out this problem without actually providing any 
fix, since the fix is for the libidn team (and really, the distros) to apply 
and ship.

> To start rolling the discussion, I've decided to dust off my confidence and 
> propose a simple algorithm (based on 
> https://en.wikipedia.org/wiki/UTF-8#Description) that should detect invalid 
> UTF-8 sequences based on the input length.

I would assume that you first need to check that the input is claimed to be a 
UTF8 locale/encoding since I take it a user can use others and then your check 
shouldn't discard the input on the same premises. To do that, you need to use 
the same hueristics and logic libidn uses to find out if it is. And then you 
risk getting out of synch with libidn as it develops. Or you just get some 
detail wrong and the problem is back.

I'm not saying wget couldn't do something like this, as "security in depth" 
and all that and it might be better with this check even if there's a risk 
that it lets some badness through than to not have the check at all.

But really, the effort should instead be put on the libidn side once and for 
all. There are MANY programs using libidn that otherwise would need the same 
check getting implemented.

I have not yet seen any single good reason for why libidn can't do this check 
itself. That's where it belongs.

--

 / daniel.haxx.se