bug#53400: base64: decode Base64URL

[Sergey Ponomarev]

Thank you, Pádraig,

Your are absolutely right that for a better correctness it must be a
way to catch padding errors. And I just decided to check how it
behaves currently:

$ echo "Aladin" | base64
QWxhZGluCg==

Now let's remove the padding:
$ echo "QWxhZGluCg" | base64 -d
Aladin
base64: invalid input
$ echo $?
1

So base64 decoded as much as it can but wrote a message into stderr
and returned an error code.
I think this is a good compromise: for those who are really want to
know about the padding they can check an error code.
We may add an additional error code to show that the base64 was
decoded as an Base64URL.

Also we may introduce an additional flag `-u` to say that we do expect
a Base64URL input and don't set the error code. The same flag may be
also used for base64 encoding.

Speaking about an additional flag the same idea was proposed in the
ticket for OpenSSL and I answered:

> Having it as a separate flag is a good thing to do for a better correctness. 
> Still many unexpirienced users will fail just because they copied a first 
> answer on SO. This makes the feature much less useful. Especially the big 
> problem is that some base64 data may be decoded successfully during a 
> development and testing and the code with bugs may be delivered to end users. 
> Again, this is a very widely used tool.

So maybe the idea with decode both but set the exit code + additional
flag will be sufficient?

On Fri, 21 Jan 2022 at 17:19, Pádraig Brady <P@draigbrady.com> wrote:
>
> On 20/01/2022 21:31, Sergey Ponomarev wrote:
> > Package: coreutils
> >
> > Currently the base64 util can't  parse base64 url safe.
> > RFC 3548 Base 64 Encoding with URL and Filename Safe Alphabet
> > https://datatracker.ietf.org/doc/html/rfc3548#page-6
> >
> > But basically this is the same encoding: just two symbols differ and
> > no padding. So it's easy and safe just to parse both forms.
> > The coreutils have a separate command `basenc --base64url` to parse
> > the b64u but actually there is no any sense to make it as a separate
> > command.
> > Having this as a separate command makes it unusable for small Linux
> > distros like OpenWrt where the device's disk is very small.
>
> Note coreutils can be built like busybox to include multiple commands in a 
> single binary
>
> >
> > Currently **many** shell scripts uses additional transformation like:
> >
> > ```sh
> > base64_padding()
> > {
> >    local len=$(( ${#1} % 4 ))
> >    local padded_b64=''
> >    if [ ${len} = 2 ]; then
> >      padded_b64="${1}=="
> >    elif [ ${len} = 3 ]; then
> >      padded_b64="${1}="
> >    else
> >      padded_b64="${1}"
> >    fi
> >    echo -n "$padded_b64"
> > }
> >
> > base64url_to_b64()
> > {
> >    base64_padding "${1}" | tr -- '-_' '+/'
> > }
> > ```
>
> I'm not sure about base64 accepting _either_ encoding without options.
> There would be robustness issues with that.
>
> Also blinding assuming padding may introduce robustness issues with truncated 
> data.
>
> It might be useful to have a "--decode-relaxed" option or something
> to support decoding any b64 variant and assume no padding,
> though preprocessing isn't that onerous, so this is marginal.
>
> > But this over complicates everything, reduces performance and is error 
> > prone.
> >
> > I created a similar ticked for OpenSSL base64
> > https://github.com/openssl/openssl/issues/17559
> >
> >
> > Also as an additional request: can we add the new param `-D` in upper
> > case which will be an alias to `-d`.
> > The issue is that on MacOS the small -d was used for debug and they
> > used the -D in upper case. This creates a lot of problems for scripts
> > that should work on both Linux and MacOS. So most people use the long
> > --decode. But many users just don't know about this interop problem
> > and once I faced a problem when my college gave me a script that
> > didn't work for me.
> > This will prevent unexpected compatibility problems.
>
> macos using -D is unfortunate.
> It's surprising they diverged from FreeBSD here.
> The fact there is a cross platform --decode option,
> suggests we don't need to add -D support,
> as it doesn't help folks using -d on other systems.
>
> cheers,
> Pádraig



-- 
Sergey Ponomarev, skype:stokito