JSON parser grammar

[Oliver Hunt-2]

On Jul 3, 2009, at 3:41 PM, Hallvord R. M. Steen wrote:

> On Wed, 03 Jun 2009 21:42:46 +0200, Allen Wirfs-Brock <Allen.Wirfs-Brock@...
> > wrote:
>
>> a) Allow strings, numbers, Booleans, and null in addition to  
>> objects and
>> arrays as top level JSON text.
>> The ES5 spec. already has this although it isn't in the RFC.  I  
>> haven't
>> heard any suggestions that we remove it.
>
>
> This may be a stupid question, but..
>
> How can you allow "strings" as top level JSON text and keep the  
> 15.12.2 step 2 requirement that JSON.parse() must throw if the input  
> isn't valid JSON source text? In other words, how is an  
> implementation supposed to know if I'm passing in a string of random  
> content that should be "parsed" into a string or a malformed piece  
> of JSON that should cause an exception?

A piece of text is either a string literal or it is not -- i suspect  
you're confusing JSON.parse("foo") where you are passing a string  
containing the characters f,o and o with JSON.parse("\"foo\"") in  
which the string contains the characters ",f,o,o and " -- eg. a  
string literal.

--Oliver

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Hallvord R. M. Steen]

>>> a) Allow strings, numbers, Booleans, and null in addition to objects  
>>> and
>>> arrays as top level JSON text.
>>> The ES5 spec. already has this although it isn't in the RFC.  I haven't
>>> heard any suggestions that we remove it.

>> How can you allow "strings" as top level JSON text?
>
> A piece of text is either a string literal or it is not -- i suspect  
> you're confusing JSON.parse("foo") where you are passing a string  
> containing the characters f,o and o with JSON.parse("\"foo\"") in which  
> the string contains the characters ",f,o,o and " -- eg. a  string  
> literal.

Indeed I was, particularly since IE8's implementation doesn't seem to understand this string-inside-string feature yet so when I tried this earlier I remained confused :-p. Thanks for clarifying.

Another question: The JSON grammar says

JSONNumber ::
-opt  DecimalIntegerLiteral JSONFraction opt  ExponentPart opt


JSONFraction ::
. DecimalDigits

This apparently makes numbers like "1." illegal? Should this really throw:

JSON.parse('[1.]') ?

And what about
JSON.parse('[1.e10]') ?

Both are of course allowed in normal JavaScript source text.

--
Hallvord R. M. Steen, Core Tester, Opera Software
http://www.opera.com http://my.opera.com/hallvors/
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Douglas Crockford]

Hallvord R. M. Steen wrote:

> Another question: The JSON grammar says
>
> JSONNumber ::
> -opt  DecimalIntegerLiteral JSONFraction opt  ExponentPart opt
>
>
> JSONFraction ::
> . DecimalDigits
>
> This apparently makes numbers like "1." illegal? Should this really throw:
>
> JSON.parse('[1.]') ?
>
> And what about
> JSON.parse('[1.e10]') ?
>
> Both are of course allowed in normal JavaScript source text.

There are lots of things accepted by JavaScript that are not part of JSON.
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Allen Wirfs-Brock-2]

IE8 does correctly process:
    JSON.parse("\"foo\"")
I just tried it and it worked fine.  There are a few discrepancies between the IE8 JSON implementation and the current ES5 draft. See http://blogs.msdn.com/jscript/archive/2009/06/23/native-json-support-in-ie8-and-tracking-the-ecmascript-fifth-edition-draft-specification.aspx for details.

Allen

>-----Original Message-----
>From: Hallvord R. M. Steen [mailto:hallvord@...]
>Sent: Saturday, July 04, 2009 4:49 AM
>To: Oliver Hunt
>Cc: Allen Wirfs-Brock; Rob Sayre; Mark S.Miller; es-discuss@...;
>Douglas Crockford; Robert Sayre
>Subject: Re: JSON parser grammar
>
>>>> a) Allow strings, numbers, Booleans, and null in addition to objects
>>>> and
>>>> arrays as top level JSON text.
>>>> The ES5 spec. already has this although it isn't in the RFC.  I
>haven't
>>>> heard any suggestions that we remove it.
>
>>> How can you allow "strings" as top level JSON text?
>>
>> A piece of text is either a string literal or it is not -- i suspect
>> you're confusing JSON.parse("foo") where you are passing a string
>> containing the characters f,o and o with JSON.parse("\"foo\"") in
>which
>> the string contains the characters ",f,o,o and " -- eg. a  string
>> literal.
>
>Indeed I was, particularly since IE8's implementation doesn't seem to
>understand this string-inside-string feature yet so when I tried this
>earlier I remained confused :-p. Thanks for clarifying.
>
>Another question: The JSON grammar says
>
>JSONNumber ::
>-opt  DecimalIntegerLiteral JSONFraction opt  ExponentPart opt
>
>
>JSONFraction ::
>. DecimalDigits
>
>This apparently makes numbers like "1." illegal? Should this really
>throw:
>
>JSON.parse('[1.]') ?
>
>And what about
>JSON.parse('[1.e10]') ?
>
>Both are of course allowed in normal JavaScript source text.
>
>--
>Hallvord R. M. Steen, Core Tester, Opera Software
>http://www.opera.com http://my.opera.com/hallvors/

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Hallvord R. M. Steen]

On Sat, 04 Jul 2009 13:49:01 +0200, Hallvord R. M. Steen
<hallvord@...> wrote:

> Another question: The JSON grammar says
> JSONNumber ::
> -opt  DecimalIntegerLiteral JSONFraction opt  ExponentPart opt
> JSONFraction ::
> . DecimalDigits
> This apparently makes numbers like "1." illegal?

I suggest that the spec is changed. In 15.12.1.1 under "Syntax", add
"JSONFraction" on a new line after

JSONNumber ::
        -opt  DecimalIntegerLiteral JSONFractionopt  ExponentPartopt

in other words, define JSONNumber as follows:


JSONNumber ::
        -opt  DecimalIntegerLiteral JSONFractionopt  ExponentPartopt
        JSONFraction

I suggest this change because the browser implementations seem to align on
allowing this.

--
Hallvord R. M. Steen, Core Tester, Opera Software
http://www.opera.com http://my.opera.com/hallvors/
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Mark S. Miller-2]

On Tue, Aug 25, 2009 at 9:32 AM, Hallvord R. M. Steen <hallvord@...> wrote:

This apparently makes numbers like "1." illegal?

[...]

I suggest that the spec is changed. [...]

I suggest this change because the browser implementations seem to align on
allowing this.

While the JSON spec itself allows parsers to accept strings outside the standard JSON grammar, the ES5 spec for the JSON object was carefully constructed to give us a validating parser: one that rejects non-JSON text. I would like to preserve this property, as it is what justified the need for adding a new JSON primitive to JavaScript.

Without a new primitive, the best that JS code has been able to do efficiently is json2.js and json_sans_eval.js. The first being too dangerous as it guards an eval with delicate complex regular expressions no one understands. The second is about as fast but is non-validating. If a safe efficient non-validating parser were adequate we wouldn't need a new primitive.

--
   Cheers,
   --MarkM

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Oliver Hunt-2]

On Aug 25, 2009, at 9:32 AM, Hallvord R. M. Steen wrote:

> On Sat, 04 Jul 2009 13:49:01 +0200, Hallvord R. M. Steen
> <hallvord@...> wrote:
>
>> Another question: The JSON grammar says
>> JSONNumber ::
>> -opt  DecimalIntegerLiteral JSONFraction opt  ExponentPart opt
>> JSONFraction ::
>> . DecimalDigits
>> This apparently makes numbers like "1." illegal?
>
> I suggest that the spec is changed. In 15.12.1.1 under "Syntax", add
> "JSONFraction" on a new line after
>
> JSONNumber ::
> -opt  DecimalIntegerLiteral JSONFractionopt  ExponentPartopt
>
> in other words, define JSONNumber as follows:
>
>
> JSONNumber ::
> -opt  DecimalIntegerLiteral JSONFractionopt  ExponentPartopt
> JSONFraction
>
> I suggest this change because the browser implementations seem to  
> align on
> allowing this.

The WebKit JSON parser does not allow this (You can test in Safari  
4.0.3 or a nightly build) -- In general the JSON parser in WebKit is  
strictly conformant with the spec (i am unaware of anywhere the parser  
breaks conformance to the strict grammar and lexer of the spec -- if  
someone finds one i consider it a bug and encourage them to file at http://bugs.webkit.org 
  :D ).  We also haven't received any bugs relating to compatibility  
of a strict parser/lexer vs. eval-driven parsing.

--Oliver

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Hallvord R. M. Steen]

On Tue, 25 Aug 2009 21:57:11 +0200, Mark S. Miller <erights@...>  
wrote:

> While the JSON spec itself allows parsers to accept strings outside the
> standard JSON grammar, the ES5 spec for the JSON object was carefully
> constructed to give us a validating parser: one that rejects non-JSON  
> text.

Well, I may have suggested a change in the wrong place but the intention  
was to make this legal in JSON text, not to break validation generally.

On Tue, 25 Aug 2009 22:09:54 +0200, Oliver Hunt <oliver@...> wrote:

> The WebKit JSON parser does not allow this (You can test in Safari 4.0.3  
> or a nightly build)

Indeed. As you haven't seen any compat problems so far I'll withdraw the  
suggestion and make sure we (Opera) follows the spec as-is, hopefully the  
Gecko and IE teams will fix it too :)

--
Hallvord R. M. Steen, Core Tester, Opera Software
http://www.opera.com http://my.opera.com/hallvors/
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Mark S. Miller-2]

On Wed, Aug 26, 2009 at 1:18 PM, Hallvord R. M. Steen<hallvord@...> wrote:

> On Tue, 25 Aug 2009 21:57:11 +0200, Mark S. Miller <erights@...>
> wrote:
>
>> While the JSON spec itself allows parsers to accept strings outside the
>> standard JSON grammar, the ES5 spec for the JSON object was carefully
>> constructed to give us a validating parser: one that rejects non-JSON
>> text.
>
> Well, I may have suggested a change in the wrong place but the intention was
> to make this legal in JSON text, not to break validation generally.

As I've often said to Crock, one of the most wonderful things about
the JSON spec is that there is no possibility for versioning the spec,
so JSON is done. It is what it is. We're better off living with its
problems than we would be living with versioning problems. The world
would be a much better place if other standards groups had the wisdom
to disband when they were done. (And no, I'm not slyly suggesting that
EcmaScript is done. It's not.)


> On Tue, 25 Aug 2009 22:09:54 +0200, Oliver Hunt <oliver@...> wrote:
>
>> The WebKit JSON parser does not allow this (You can test in Safari 4.0.3
>> or a nightly build)
>
> Indeed. As you haven't seen any compat problems so far I'll withdraw the
> suggestion and make sure we (Opera) follows the spec as-is, hopefully the
> Gecko and IE teams will fix it too :)

Good, thanks. We should be sure to include such tests in the
to-be-official EcmaScript test suites we're accumulating. Might help
encourage conformance ;).

--
    Cheers,
    --MarkM
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Allen Wirfs-Brock-2]

>-----Original Message-----
>From: es-discuss-bounces@... [mailto:es-discuss-
>bounces@...] On Behalf Of Hallvord R. M. Steen
>
>Indeed. As you haven't seen any compat problems so far I'll withdraw the
>suggestion and make sure we (Opera) follows the spec as-is, hopefully
>the
>Gecko and IE teams will fix it too :)
>

One of the hazards of shipping early in the standards cycle is having some of the details changing out from under you.  We intent to fix IE8 JSON ES5 conformance issues (and a couple of bugs) as part of our normal maintenance process.  Most of the know issues are discussed here http://blogs.msdn.com/jscript/archive/2009/06/23/native-json-support-in-ie8-and-tracking-the-ecmascript-fifth-edition-draft-specification.aspx 

Allen
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss