Serialization issue with Jettison

[jlouvel]

Hi all,

First, let me thank you for developing such a nice project! In the
Restlet Framework (http://www.restlet.org), we recently added an XStream
extension per our community request:
http://wiki.restlet.org/docs_2.0/13-restlet/28-restlet/264-restlet.html

We are very happy with it but one of our users encountered an issue with
the JSON serialization that I was able to isolate into a very simple
Eclipse project (see attached file).

In the "test1" package, here is the output, with the XML version
followed by the JSON version:

<test1.A>
   <b>
     <name>child</name>
   </b>
   <a>
     <b>
       <name>parent</name>
     </b>
   </a>
</test1.A>

------------------

{"test1.A":{"b":[{"name":"child"},{"name":"parent"}],"a":{}}}


As you can see,  the "a" property is empty in the JSON case and the
"name" property is added as an array to the "b" property.

In the "test2" package, I've switched the declaration in the A.java of
the "a" and "b" fields and the output looks good:

<test2.A>
   <a>
     <b>
       <name>parent</name>
     </b>
   </a>
   <b>
     <name>child</name>
   </b>
</test2.A>

------------------

{"test2.A":{"a":{"b":{"name":"parent"}},"b":{"name":"child"}}}


Is it normal to observe such behavior? Should I enter a bug report?

Also, we are wondering when the next release of XStream is scheduled and
if it will include an upgrade to Jettison 1.1?

Best regards,
Jerome Louvel
--
Restlet ~ Founder and Lead developer ~ http://www.restlet.org
Noelios Technologies ~ Co-founder ~ http://www.noelios.com


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jörg Schaible-2]

Hi Jerome,

Jerome Louvel wrote:

>
> Hi all,
>
> First, let me thank you for developing such a nice project! In the
> Restlet Framework (http://www.restlet.org), we recently added an XStream
> extension per our community request:
> http://wiki.restlet.org/docs_2.0/13-restlet/28-restlet/264-restlet.html

Nice to hear, I'll add a link on the XStream site also.

> We are very happy with it but one of our users encountered an issue with
> the JSON serialization that I was able to isolate into a very simple
> Eclipse project (see attached file).
>
> In the "test1" package, here is the output, with the XML version
> followed by the JSON version:
>
> <test1.A>
>    <b>
>      <name>child</name>
>    </b>
>    <a>
>      <b>
>        <name>parent</name>
>      </b>
>    </a>
> </test1.A>
>
> ------------------
>
> {"test1.A":{"b":[{"name":"child"},{"name":"parent"}],"a":{}}}
>
>
> As you can see,  the "a" property is empty in the JSON case and the
> "name" property is added as an array to the "b" property.
>
> In the "test2" package, I've switched the declaration in the A.java of
> the "a" and "b" fields and the output looks good:
>
> <test2.A>
>    <a>
>      <b>
>        <name>parent</name>
>      </b>
>    </a>
>    <b>
>      <name>child</name>
>    </b>
> </test2.A>
>
> ------------------
>
> {"test2.A":{"a":{"b":{"name":"parent"}},"b":{"name":"child"}}}
>
>
> Is it normal to observe such behavior? Should I enter a bug report?

Jettison is a backend for StAX. Actually XStream core does not know that it
is using Jettison, it simply writes in this case XML using the StAX API.
Jettison has to guess from the incoming XML events what kind of objects it
will write for JSON. Therefore the problem is with Jettison as the XML is
correct.
 
> Also, we are wondering when the next release of XStream is scheduled

I *hope* to finish the next release this year ... ;-)

> and if it will include an upgrade to Jettison 1.1?

No. Unfortunately a lot of XStream's JSON tests fail with Jettison 1.1 and
the result is even worse. Please, advice your users to use the 1.0.x
version of Jettison (above looks like 1.1 already). Actually I already
started to look for alternate JSON support. One reason why the next release
did not happen yet.

- Jörg


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[jlouvel]

Hi Jorg,

Thanks for the prompt reply and for the link back to Restlet :) I added
an advice regarding the Jettison version in our wiki and look forward to
playing with next XStream version!

Another feature we are eager to have is the support for XStream on GAE.
We already ship a Restlet edition for GAE and would benefit from
shipping our XStream extension as well. Here is our related issue:
http://restlet.tigris.org/issues/show_bug.cgi?id=863

--

Something that is puzzling me is how can Jettison produce two different
JSON outputs if the Stax events are strictly identical?

Doesn't the fact that switching the declarations order of the fields in
the Java class solves the issue, imply that XStream might actually
produces the Stax events in a slightly different order, maybe confusing
Jettison?

In other words, could the XML output be similar even though Stax events
are sent in a slightly different order?

Best regards,
Jerome



Jörg Schaible a écrit :

> Hi Jerome,
>
> Jerome Louvel wrote:
>
>> Hi all,
>>
>> First, let me thank you for developing such a nice project! In the
>> Restlet Framework (http://www.restlet.org), we recently added an XStream
>> extension per our community request:
>> http://wiki.restlet.org/docs_2.0/13-restlet/28-restlet/264-restlet.html
>
> Nice to hear, I'll add a link on the XStream site also.
>
>> We are very happy with it but one of our users encountered an issue with
>> the JSON serialization that I was able to isolate into a very simple
>> Eclipse project (see attached file).
>>
>> In the "test1" package, here is the output, with the XML version
>> followed by the JSON version:
>>
>> <test1.A>
>>    <b>
>>      <name>child</name>
>>    </b>
>>    <a>
>>      <b>
>>        <name>parent</name>
>>      </b>
>>    </a>
>> </test1.A>
>>
>> ------------------
>>
>> {"test1.A":{"b":[{"name":"child"},{"name":"parent"}],"a":{}}}
>>
>>
>> As you can see,  the "a" property is empty in the JSON case and the
>> "name" property is added as an array to the "b" property.
>>
>> In the "test2" package, I've switched the declaration in the A.java of
>> the "a" and "b" fields and the output looks good:
>>
>> <test2.A>
>>    <a>
>>      <b>
>>        <name>parent</name>
>>      </b>
>>    </a>
>>    <b>
>>      <name>child</name>
>>    </b>
>> </test2.A>
>>
>> ------------------
>>
>> {"test2.A":{"a":{"b":{"name":"parent"}},"b":{"name":"child"}}}
>>
>>
>> Is it normal to observe such behavior? Should I enter a bug report?
>
> Jettison is a backend for StAX. Actually XStream core does not know that it
> is using Jettison, it simply writes in this case XML using the StAX API.
> Jettison has to guess from the incoming XML events what kind of objects it
> will write for JSON. Therefore the problem is with Jettison as the XML is
> correct.
>  
>> Also, we are wondering when the next release of XStream is scheduled
>
> I *hope* to finish the next release this year ... ;-)
>
>> and if it will include an upgrade to Jettison 1.1?
>
> No. Unfortunately a lot of XStream's JSON tests fail with Jettison 1.1 and
> the result is even worse. Please, advice your users to use the 1.0.x
> version of Jettison (above looks like 1.1 already). Actually I already
> started to look for alternate JSON support. One reason why the next release
> did not happen yet.
>
> - Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jörg Schaible-2]

Hi Jerome,

Jerome Louvel wrote:

>
> Hi Jorg,
>
> Thanks for the prompt reply and for the link back to Restlet :) I added
> an advice regarding the Jettison version in our wiki and look forward to
> playing with next XStream version!
>
> Another feature we are eager to have is the support for XStream on GAE.
> We already ship a Restlet edition for GAE and would benefit from
> shipping our XStream extension as well. Here is our related issue:
> http://restlet.tigris.org/issues/show_bug.cgi?id=863

this is also target for the next release.

> --
>
> Something that is puzzling me is how can Jettison produce two different
> JSON outputs if the Stax events are strictly identical?

? They are not. As you said, you switched the declaration. Sequence matters
for XML.
 
> Doesn't the fact that switching the declarations order of the fields in
> the Java class solves the issue, imply that XStream might actually
> produces the Stax events in a slightly different order, maybe confusing
> Jettison?

Replace the JSON driver with the one of StAX and you see what XStream
generates calling the XMLStreamWriter of StAX.

> In other words, could the XML output be similar even though Stax events
> are sent in a slightly different order?

Jettison has to have a heuristic to determin out of the calls of the
XMLStreamWriter when to write a JSONArray instead of a JSONObject. An array
is generated if two siblings (in sequence ?) start have the same name. For
whatever reason it forgets in the fist case that the second (nested) 'b'
element is actually not a neighbor.

- Jörg


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[jlouvel]

Hi Jorg,

I have done more investigation and confirmed that the issue lies in
Jettison. See related issues:
http://jira.codehaus.org/browse/JETTISON-53
http://jira.codehaus.org/browse/JETTISON-58

As suggested in issue 53, removing the hack for nested arrays did solve
our issue. For now, we are using a custom version of Jettison 1.0.1 (see
JAR attached).

Best regards,
Jerome


Jörg Schaible a écrit :

> Hi Jerome,
>
> Jerome Louvel wrote:
>
>> Hi Jorg,
>>
>> Thanks for the prompt reply and for the link back to Restlet :) I added
>> an advice regarding the Jettison version in our wiki and look forward to
>> playing with next XStream version!
>>
>> Another feature we are eager to have is the support for XStream on GAE.
>> We already ship a Restlet edition for GAE and would benefit from
>> shipping our XStream extension as well. Here is our related issue:
>> http://restlet.tigris.org/issues/show_bug.cgi?id=863
>
> this is also target for the next release.
>
>> --
>>
>> Something that is puzzling me is how can Jettison produce two different
>> JSON outputs if the Stax events are strictly identical?
>
> ? They are not. As you said, you switched the declaration. Sequence matters
> for XML.
>  
>> Doesn't the fact that switching the declarations order of the fields in
>> the Java class solves the issue, imply that XStream might actually
>> produces the Stax events in a slightly different order, maybe confusing
>> Jettison?
>
> Replace the JSON driver with the one of StAX and you see what XStream
> generates calling the XMLStreamWriter of StAX.
>
>> In other words, could the XML output be similar even though Stax events
>> are sent in a slightly different order?
>
> Jettison has to have a heuristic to determin out of the calls of the
> XMLStreamWriter when to write a JSONArray instead of a JSONObject. An array
> is generated if two siblings (in sequence ?) start have the same name. For
> whatever reason it forgets in the fist case that the second (nested) 'b'
> element is actually not a neighbor.
>
> - Jörg
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jörg Schaible-2]

Hi Jerome,

Jerome Louvel wrote at Dienstag, 24. November 2009 17:09:

> Hi Jorg,
>
> I have done more investigation and confirmed that the issue lies in
> Jettison. See related issues:
> http://jira.codehaus.org/browse/JETTISON-53
> http://jira.codehaus.org/browse/JETTISON-58
>
> As suggested in issue 53, removing the hack for nested arrays did solve
> our issue. For now, we are using a custom version of Jettison 1.0.1 (see
> JAR attached).

thanks for reporting back and confirming the problem.

- Jörg


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email