false assumption in JSON Map deserialization

Consider the following Java collections Map:

Map<String, Integer> m = new HashMap<String,Integer>(); m.put( "stringVal", 1);

When serialized by default it looks like this:

{"map": {"entry": [{

  "string": "stringVal",

  "int": 1

}]}}

The problem is that the Map entry is serialized as a JSON object with “type”:value as the first field,value pair which is to represent the key of the map.  The second “type”:value pair is the *value* of the map entry.  The problem is that it is using ordering as an assumption.  JSON objects do not use field ordering to be equivalent so the object:

{

  "string": "stringVal",

  "int": 1

}

Is equivalent to:

{

  "int": 1,

  "string": "stringVal"

}

So, where this becomes an issue is in working with other JSON APIs.  In my case I use XStream for server, and flex for client.  When flex serializes the JSON object above the field order is indeterminate, which is fine for JSON.  But when XStream receives the JSON it either makes a Map<String,Integer> or a Map<Integer,String> and the latter will not match my objects.

The format really should require a signifier to denote key/value, perhaps something like:

{

  "key": { "string": "stringVal" },

  "val": { "int": 1 }

}

Now it would still be possible if the order were flipped to get the map entry right.

I can file a Jira for this.  But my question is, what options do I have to serialize Map other ways?

Thanks,

Tim

Right.  I did, and it no longer is a problem for me.  Since the keys of my map were Strings I implemented a local converter that utilizes the format that Jorg suggested.  And, since I had could work with the client developer, I had him change the format there as well.  But this was not the easiest of bugs top spot… in fact when I told my client developer counter part he didn’t really believe me at first.  It wasn’t until we were able to show the issue happen once in correct order and once in incorrect from the same running VM that the non-deterministic nature of the problem became evident.

But the fact remains that the default converter, as is, does not work with clients that serialize JSON per the JSON spec.  So, I contend, it is a bug.  Or, at least something that can trip up others…  I’m suggesting a fix that would solve the issue.

In fact, I’m wondering if this is a problem for XML in XStream as well.  Isn’t there not necessarily a guaranteed order of elements in XML as well, for instance, if serialized from a DOM?

Why don't you just change it yourself in your own converter and register that converter for the specified type? I've done this several times for things I needed.

Harsch, Timothy J. (ARC-SC)[PEROT SYSTEMS] wrote:

Therefore even your suggested change will not help, since the stream-based

model requires that the key is read first, but the JSON spec does not

guarantee this either.

Why does the key need to be read first?  Looking at the source for MapConverter it looks like they both are read into temp variables.  Implementing my suggestion would just require an little logic to assign key to key when found and a moveUp/moveDown or two.  I think the suggested fix is workable.  The real question is for XStream development team to decide if the change in message modeling is worth the break in backward compatibility versus the capability of staying compatible with clients that do not adhere to order in JSON fields.

Perhaps just adding it to the extended converters would do?

Tim

PS Since I was able to negotiate the message format change with the client, I was able to implement a converter that was in the format you suggested.  We were only using strings for keys in this case so it worked fine.

-----Original Message-----

From: news [news@...] On Behalf Of Jörg Schaible

Sent: Tuesday, September 15, 2009 11:13 AM

To: user@...

Subject: [xstream-user] Re: false assumption in JSON Map deserialization

Hi Tim,

Harsch, Timothy J. (ARC-SC)[PEROT SYSTEMS] wrote:

Consider the following Java collections Map:

Map<String, Integer> m = new HashMap<String,Integer>(); m.put(

"stringVal", 1);

When serialized by default it looks like this:

{"map": {"entry": [{

  "string": "stringVal",

  "int": 1

}]}}

The problem is that the Map entry is serialized as a JSON object with

"type":value as the first field,value pair which is to represent the key

of the map.  The second "type":value pair is the *value* of the map entry.

 The problem is that it is using ordering as an assumption.  JSON objects

do not use field ordering to be equivalent so the object:

{

  "string": "stringVal",

  "int": 1

}

Is equivalent to:

{

  "int": 1,

  "string": "stringVal"

}

So, where this becomes an issue is in working with other JSON APIs.  In my

case I use XStream for server, and flex for client.  When flex serializes

the JSON object above the field order is indeterminate, which is fine for

JSON.  But when XStream receives the JSON it either makes a

Map<String,Integer> or a Map<Integer,String> and the latter will not match

my objects.

The format really should require a signifier to denote key/value, perhaps

something like:

{

  "key": { "string": "stringVal" },

  "val": { "int": 1 }

}

Now it would still be possible if the order were flipped to get the map

entry right.

I can file a Jira for this.  But my question is, what options do I have to

serialize Map other ways?

actually it is worse. The JSON deserialisation is based on Jettison that

implements a StAX reader i.e. for XStream it looks like reading XML.

However, StAX (and XStream) is a stream-based API where the sequence of

elements *is* significant. This contradicts obviously the JSON spec. 

Therefore even your suggested change will not help, since the stream-based

model requires that the key is read first, but the JSON spec does not

guarantee this either.

The element sequence is guaranteed in JSON only for arrays, therefore all

you can do is writing an own MapConverter that writes the Map.Entry

instances as array. However, if your keys are all of same type and can be

uses as valid JSON labels, you may optimize your JSON by writing the key as

tag:

{

  "stringVal": 1

}

- Jörg

---------------------------------------------------------------------

To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

---------------------------------------------------------------------

To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email