Re: [Chicken-hackers] CHICKEN in production

[Florian Zumbiehl]

Hi,

> > Take JSON as an example: JSON character strings can encode NULs, so if
> > CHICKEN were to reject NULs in character strings, you could not write a
> > JSON parser in CHICKEN.
> 
> That's technically true, but where will you find a JSON document in the
> wild that contains "\u0000"?  JSON parsers written in C won't be able to
> handle it either, since AFAIK they all return C strings.  You probably
> won't be able to store it in a database either, since they use C APIs
> somewhere in the process.

If some code can't handle that, it's by definition not a JSON parser.

Also, I don't have a clue where I would find such JSON documents, but I
don't have any representative knowledge about all JSON documents, so I
don't think that that's a useful datapoint, especially so given that JSON
was just an example. What we do know is that NULs are allowed, so anyone
who faces a problem that could be solved by storing NULs in JSON (for
example) would be in perfect accordance with the standard when doing so,
and yet would not be able to process those documents with a CHICKEN
program.

The fact that there are some interfaces that can't represent certain values
is not exactly an argument for not being able to represent those values in
a general purpose programming language. Unix filenames? Can't represent
slashes. Therefore, reject slashes! DNS hostnames? Can't represent colons.
Therefore, reject colons! Well, you get the idea ...

> I am frankly sick of tools bending over backwards to support NUL.

I am frankly sick of people making up their own variants of standards,
creating all kinds of interoperability and security problems, and even more
of environments that make it unnecessarily difficult to implement
conforming implementations.

Similar-to-$standard-but-different-in-more-or-less-well-documented-ways
implementations of protocols and formats are the source of so many
headaches, it's not funny anymore. Not only does it create interoperability
problems, it's also a common security problem, so common in fact that the
langsec community has invented the term "computationally equivalent
parsers" for what you should do to avoid it.

To continue with the JSON example: Suppose you are writing some backend
module that gets fed JSON documents from some outside-facing system that as
part of its pre-processing checks syntactic validity. As you are guaranteed
to receive syntactically valid JSON documents, you obviously don't need to
worry about parsing failures. Well, except the parser egg that you were
sold as being a JSON parser is in fact not a JSON parser, but a
JSON-but-without-NULs parser, so some creative person submits a JSON
document with NULs to your frontend system, which validates it, passes it
to your JSON-but-without-NULs parser, and voilà, you have a DoS, congrats!

Florian