Re: [Help-smalltalk] performance of the json parser

[Robin Redeker]

On Sat, Nov 03, 2007 at 09:05:20PM -0500, Stephen Compall wrote:
> On Sun, 2007-11-04 at 01:07 +0100, Robin Redeker wrote:
> > Well, those only operate on unicode characters as the json
> > implementation does. But they indeed don't do any encoding stuff.
> > So that might explain a bit of the overhead, and also the GC stuff might
> > be explain the bad performance. Is GC known to cause multiple ms long
> > breaks?
> 
> It explains a *lot* of the overhead.  In 604 (615 was "have JSON work on
> Unicode"), I got 0/14/1, with only a few outliers (presumably due to GC)
> and 0/1 dominating the set.  In 619 I get 5/34/14, though of course
> using I18N also loads the GC so it can't account for all of that.

Thats very interesting, thanks for testing that. It surprises me a bit
that GC is indeed such a huge overhead.

I guess the GC doesn't leave much space for optimization?

> I should add that I would find a JSON parser/formatter that supports
> I18N to be far more useful than one without, regardless of the load.
> Furthermore, I would rather have UnicodeStrings than Strings, which by
> my count would eliminate 17 of the 18 Iconvs that your test produces on
> each iteration, though this doesn't appear to be an option.

A JSON parser _MUST_ support Unicode or else it's not a JSON parser :-)
Maybe I can change the parser to parse utf-8 encoded byte-buffers
instead of real Unicode strings. But I don't think that there is much
to be saved here anyways. (The best way to save performance is probably
to implement a C parser for utf-8 encoded byte-buffers with the same
ST API)

And indeed, skipping the iconv step would just test the JSON parser
itself. In my application that uses the JSON parser I actually can't
skip that step as I get the data from the network and have to run it
through iconv anyways.

Maybe I could move GC into some process that only runs when the system
is idle and have a timer that ensures that the GC runs once in while.
That would probably optimize the latency of my server application a bit.


Robin