JSON.stringify replacer array semantics

[Oliver Hunt-2]

The spec for JSON doesn't describe the exact behaviour of the replacer  
argument to stringify when the replacer is an array.  The relevant  
text (from 15.12.3) is

If Type(replacer) is object and the [[Class]] internal property of  
replacer is "Array", then
     Let K be an internal List consisting of the values of those  
properties of replacer that have array index names. The elements of  
the list are in ascending array index order.

As the replacer array may contain any sequence of values we need to  
specify the acceptance, conversion, and conversion timing for each  
element.  The spec doesn't really specify these details at all, saying  
only that replacer may be an array of strings.

Two issues that need to be clarified (that i can see):
    * What happens if the replacer array is modified during  
serialisation?
    * What happens to non-string primitives are in the replacer array?

json2.js basically seems to do something akin to the following if  
replacer is an array
    givenReplacer = replacer;
    replacer = [];
    var length = replacer.length;
    for (var i = 0; i < length; i++) {
        if (typeof givenReplacer[i] === "string")
            replacer.push(givenReplacer[i]);
    }

This resolves the above issues by making a copy of the array (thus  
isolating it from mutation of the replacer array), and resolves the  
non-string property issue by just ignoring any non-string values.  I  
believe it would be more consistent with ECMAScript to perform  
[[ToString]] conversion instead of filtering non-string values, but if  
we do that we need to specify when the [[ToString]] conversion should  
occur.  I believe that at the beginning of the stringify algorithm,  
probably just before what is currently step 7 (where the wrapper is  
created)

7. If Type(replacer) is object and the [[Class]] internal property of  
replacer is "Array", then
     a. Let oldReplacer be replacer
     b. Let replacer be a new array
     c. Let len be the result of calling the [[Get]] internal method  
of oldReplacer with argument "length"
     d. Let index be 0
     e. Repeat while index < len
         i. Let prop be the result of calling the [[Get]] internal  
method of oldReplacer with argument index
         ii. Let prop be the result of calling the method ToString(prop)
         iii. if prop is not undefined
             A. Call the [[Put]] internal method of replacer with  
arguments index and prop
         iv. Increment  index by 1

Remaining steps 7, 8, and 9 move to 8, 9,  and 10

--Oliver

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Allen Wirfs-Brock-2]

Thanks, these are good observations. It's useful to have fresh eyes looking at this stuff.

I believe the intent was that step 5.a of JO should have the same effect as the json2.js code you quoted. However, if that is the case it is missing the filtering to include only primitive string values.

I'm sympathetic to the argument that rather than filtering for string values it would be better to apply ToString to the replacer elements. However, that would be a departure from what json2.js does.  IE8 also currently matches the json2.js in this regard. FF?? The most likely use case I can think of for non string replacer array values would be passing an array like [2,4,6,8] to select only some elements of an array that is  being serialized.  However, since the replacer whitelist isn't used in JA this really isn't a real use case.  That argument pretty much convinced me that there isn't really much value in support non-string values in replacer arrays and it is probably best to stick with the json2.js behavior. Doug and Rob, do you have any thoughts on this?

The effect of a side-effect of the stringify algorithm mutating the replacer array is as well defined as any other side-effect sensitive aspect of the algorithm, if the entire stringify algorithm is strictly implemented as specified. A strict interpretation of the current specification requires that the List in JO step 5.a gets recreate on each use of JO. So side-effects that modify the replacer array would result in different List contents.  I think it probably is a good idea to move step 5.a into the top level stringify algorithm so that the List is only generated once before any possible side-effect producing operations.  This may be a variance from what the IE8 and FF implementations are currently doing (I haven't yet crafted a test for this that produces the necessary side-effects) but I don't think that in the short term the difference is likely to impact anyone and for the long term it is probably best to preclude the possibility of side-effecting replacer arrays.

(Note that we use Lists within the algorithms rather than "new array" because as an internal specification data type Lists are not sensitive to side-effects induced via modifications to Array.prototype)

Allen

>-----Original Message-----
>From: es-discuss-bounces@... [mailto:es-discuss-
>bounces@...] On Behalf Of Oliver Hunt
>Sent: Thursday, June 04, 2009 4:02 PM
>To: es-discuss@...
>Subject: JSON.stringify replacer array semantics
>
>The spec for JSON doesn't describe the exact behaviour of the replacer
>argument to stringify when the replacer is an array.  The relevant
>text (from 15.12.3) is
>
>If Type(replacer) is object and the [[Class]] internal property of
>replacer is "Array", then
>     Let K be an internal List consisting of the values of those
>properties of replacer that have array index names. The elements of
>the list are in ascending array index order.
>
>As the replacer array may contain any sequence of values we need to
>specify the acceptance, conversion, and conversion timing for each
>element.  The spec doesn't really specify these details at all, saying
>only that replacer may be an array of strings.
>
>Two issues that need to be clarified (that i can see):
>    * What happens if the replacer array is modified during
>serialisation?
>    * What happens to non-string primitives are in the replacer array?
>
>json2.js basically seems to do something akin to the following if
>replacer is an array
>    givenReplacer = replacer;
>    replacer = [];
>    var length = replacer.length;
>    for (var i = 0; i < length; i++) {
>        if (typeof givenReplacer[i] === "string")
>            replacer.push(givenReplacer[i]);
>    }
>
>This resolves the above issues by making a copy of the array (thus
>isolating it from mutation of the replacer array), and resolves the
>non-string property issue by just ignoring any non-string values.  I
>believe it would be more consistent with ECMAScript to perform
>[[ToString]] conversion instead of filtering non-string values, but if
>we do that we need to specify when the [[ToString]] conversion should
>occur.  I believe that at the beginning of the stringify algorithm,
>probably just before what is currently step 7 (where the wrapper is
>created)
>
>7. If Type(replacer) is object and the [[Class]] internal property of
>replacer is "Array", then
>     a. Let oldReplacer be replacer
>     b. Let replacer be a new array
>     c. Let len be the result of calling the [[Get]] internal method
>of oldReplacer with argument "length"
>     d. Let index be 0
>     e. Repeat while index < len
>         i. Let prop be the result of calling the [[Get]] internal
>method of oldReplacer with argument index
>         ii. Let prop be the result of calling the method ToString(prop)
>         iii. if prop is not undefined
>             A. Call the [[Put]] internal method of replacer with
>arguments index and prop
>         iv. Increment  index by 1
>
>Remaining steps 7, 8, and 9 move to 8, 9,  and 10
>
>--Oliver
>
>_______________________________________________
>es-discuss mailing list
>es-discuss@...
>https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Oliver Hunt-2]

On Jun 5, 2009, at 11:24 AM, Allen Wirfs-Brock wrote:

> The effect of a side-effect of the stringify algorithm mutating the  
> replacer array is as well defined as any other side-effect sensitive  
> aspect of the algorithm, if the entire stringify algorithm is  
> strictly implemented as specified. A strict interpretation of the  
> current specification requires that the List in JO step 5.a gets  
> recreate on each use of JO. So side-effects that modify the replacer  
> array would result in different List contents.  I think it probably  
> is a good idea to move step 5.a into the top level stringify  
> algorithm so that the List is only generated once before any  
> possible side-effect producing operations.

json2.js appears to perform JO.5.a at the top level of the stringify  
so this makes sense.  Firefox also does the copy to an internal list  
on the entry to stringify, but performs toString conversion on each  
item in replacement for each object it calls.

--Oliver

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss