new json parser

[Willem de Jong-2]

Hi all,

I have uploaded a json parser to trapexit. (http://forum.trapexit.org/viewtopic.php?p=44029#44029)

More than anything else, the purpose of this parser is to convince Richard O'Keefe that having a json parser with a SAX-like API is a good idea. But fortunately it can also serve more practical purposes.

As mentioned, the parser has a SAX-like API. It also has a mechanism to parse input data in blocks - if it reaches the end of the input data, it will call a function to provide the next block. The combination of these properties results in a parser that can process input of
arbitrary size, or a stream of data.

Included is a handler function that translates the SAX events into the same output format produced by mochijson2. Looks like this: {struct, [{<<"key1">>, <<"value1">>}, {<<"key2">>, 123}]}. Interestingly, the new parser (sax parser combined with handler) is roughly 2x as fast as mochijson2 (On my PC, at least).

The parser consists of just 1 file, with some edoc documentation and a couple of very simple examples included. It only works on UTF-8 encoded input (which implies that it can also parse ASCII, of course).

Regards,

Willem

_______________________________________________
erlang-questions mailing list
erlang-questions@...
http://www.erlang.org/mailman/listinfo/erlang-questions

[Bob Ippolito]

2008/8/8 Willem de Jong <w.a.de.jong@...>:

> Hi all,
>
> I have uploaded a json parser to trapexit.
> (http://forum.trapexit.org/viewtopic.php?p=44029#44029)
>
> More than anything else, the purpose of this parser is to convince Richard
> O'Keefe that having a json parser with a SAX-like API is a good idea. But
> fortunately it can also serve more practical purposes.
>
> As mentioned, the parser has a SAX-like API. It also has a mechanism to
> parse input data in blocks - if it reaches the end of the input data, it
> will call a function to provide the next block. The combination of these
> properties results in a parser that can process input of
> arbitrary size, or a stream of data.
>
> Included is a handler function that translates the SAX events into the same
> output format produced by mochijson2. Looks like this: {struct,
> [{<<"key1">>, <<"value1">>}, {<<"key2">>, 123}]}. Interestingly, the new
> parser (sax parser combined with handler) is roughly 2x as fast as
> mochijson2 (On my PC, at least).
>
> The parser consists of just 1 file, with some edoc documentation and a
> couple of very simple examples included. It only works on UTF-8 encoded
> input (which implies that it can also parse ASCII, of course).

I haven't profiled it but my guess is that yours is faster because it
doesn't keep track of the current character/line position in the doc.
mochijson2 does it because I like good error messages... :)

The second part is that mochijson2 is written such that it is
compatible with and performs well in R11B because that's what I was
using at the time. I'd have written it differently for R12B which
behaves very different with binaries (as far as performance goes).

-bob
_______________________________________________
erlang-questions mailing list
erlang-questions@...
http://www.erlang.org/mailman/listinfo/erlang-questions

[Chris Anderson-11]

On Fri, Aug 8, 2008 at 9:22 AM, Bob Ippolito <bob@...> wrote:

> 2008/8/8 Willem de Jong <w.a.de.jong@...>:
>> Hi all,
>>
>> I have uploaded a json parser to trapexit.
>> (http://forum.trapexit.org/viewtopic.php?p=44029#44029)
>>
>> Included is a handler function that translates the SAX events into the same
>> output format produced by mochijson2. Looks like this: {struct,
>> [{<<"key1">>, <<"value1">>}, {<<"key2">>, 123}]}. Interestingly, the new
>> parser (sax parser combined with handler) is roughly 2x as fast as
>> mochijson2 (On my PC, at least).

I added another handler that output the B. term format ( JSON {} =>
Erlang {[]} ) which turned out to be a 6 character change from
Willem's handler. I also added a variation on mochijson's test suite,
and profile() function that runs the test suite under fprof.

You can profile by json_parser:profile().

The paste is available here: http://gist.github.com/4651

Chris

--
Chris Anderson
http://jchris.mfdz.com
_______________________________________________
erlang-questions mailing list
erlang-questions@...
http://www.erlang.org/mailman/listinfo/erlang-questions