Surrogate pair output support (unit test)

[Olivier Jaquemet]

Hi all,

In December 2007, Dave Byrne submitted a patch to add support for
surrogate pair output :
http://www.jdom.org/pipermail/jdom-interest/2007-December/016016.html

Unfortunately this patch has never made into any JDOM release, nor in
the cvs (as far as I could find)...

A simple test case for this bug is attached and end of this mail.
As of version 1.1, the test will fail with the following error :
  junit.framework.ComparisonFailure:
    expected: <root>𐀀 𐀀</root>
    but was: <root>�� ��</root>
With the patch submitted by Dave, everything works fine.

Could you add his patch for the next release of JDOM ?
Thank your for you answer !

Regards,
Olivier Jaquemet

-----
The unit test :

  public void testHighSurrogatePair() throws JDOMException, IOException {
    SAXBuilder builder = new SAXBuilder();
    builder.setExpandEntities(true);
    Document doc = builder.build(new StringReader("<?xml
version=\"1.0\"?><root>𐀀 𐀀</root>"));
   
    Format format = Format.getCompactFormat().setEncoding("ISO-8859-1");
    XMLOutputter outputter = new XMLOutputter(format);
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    outputter.output(doc, baos);
    String xml = baos.toString();
   
    assertEquals("<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>" +
format.getLineSeparator() +
                 "<root>𐀀 𐀀</root>" +
format.getLineSeparator(), xml);
  }

_______________________________________________
To control your jdom-interest membership:
http://www.jdom.org/mailman/options/jdom-interest/youraddr@...

[Dave Byrne-2]

When I submitted the patch, Jason Hunter followed up and asked for test
cases before applying it to CVS which I never did.  Perhaps you could
whip up some junit tests?

-----Original Message-----
From: Olivier Jaquemet [mailto:olivier.jaquemet@...]
Sent: Thursday, May 07, 2009 6:36 AM
To: jdom-interest@...
Cc: Dave Byrne
Subject: Surrogate pair output support (unit test)

Hi all,

In December 2007, Dave Byrne submitted a patch to add support for
surrogate pair output :
http://www.jdom.org/pipermail/jdom-interest/2007-December/016016.html

Unfortunately this patch has never made into any JDOM release, nor in
the cvs (as far as I could find)...

A simple test case for this bug is attached and end of this mail.
As of version 1.1, the test will fail with the following error :
  junit.framework.ComparisonFailure:
    expected: <root>𐀀 𐀀</root>
    but was: <root>�� ��</root>
With the patch submitted by Dave, everything works fine.

Could you add his patch for the next release of JDOM ?
Thank your for you answer !

Regards,
Olivier Jaquemet

-----
The unit test :

  public void testHighSurrogatePair() throws JDOMException, IOException
{
    SAXBuilder builder = new SAXBuilder();
    builder.setExpandEntities(true);
    Document doc = builder.build(new StringReader("<?xml
version=\"1.0\"?><root>𐀀 𐀀</root>"));
   
    Format format = Format.getCompactFormat().setEncoding("ISO-8859-1");
    XMLOutputter outputter = new XMLOutputter(format);
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    outputter.output(doc, baos);
    String xml = baos.toString();
   
    assertEquals("<?xml version=\"1.0\" encoding=\"ISO-8859-1\"?>" +
format.getLineSeparator() +
                 "<root>𐀀 𐀀</root>" +
format.getLineSeparator(), xml);
  }


_______________________________________________
To control your jdom-interest membership:
http://www.jdom.org/mailman/options/jdom-interest/youraddr@...

[jhunter]

On May 7, 2009, at 6:35 AM, Olivier Jaquemet wrote:

> Hi all,
>
> In December 2007, Dave Byrne submitted a patch to add support for  
> surrogate pair output :
> http://www.jdom.org/pipermail/jdom-interest/2007-December/016016.html
>
> Unfortunately this patch has never made into any JDOM release, nor  
> in the cvs (as far as I could find)...

Correct.  I asked a few questions that didn't get a reply and I'm not  
expert enough on surrogate pair outputs to blaze ahead unless there's  
a test case to keep me honest.

> A simple test case for this bug is attached and end of this mail.
> As of version 1.1, the test will fail with the following error :
> junit.framework.ComparisonFailure:
>   expected: <root>𐀀 𐀀</root>
>   but was: <root>�� ��</root>
> With the patch submitted by Dave, everything works fine.

Excellent!  With that I'll feel safer checking it in.  It doesn't seem  
in big demand, but if we can do that's right w/ surrogate pairs then  
we should.

-jh-

_______________________________________________
To control your jdom-interest membership:
http://www.jdom.org/mailman/options/jdom-interest/youraddr@...

[Olivier Jaquemet]

Jason Hunter wrote:
>> A simple test case for this bug is attached and end of this mail.
>> ...
> Excellent!  With that I'll feel safer checking it in.  It doesn't seem
> in big demand, but if we can do that's right w/ surrogate pairs then
> we should.
I'm glad I could help a bit with this test... Some "warnings" though :
- it is quite a high level unit test as I don't know the low level JDOM
API to test....
- it does not test the boundary condition (invalid surrogate pair) which
Dave's properly checked in his code.
- it does not perform regression testing on other basic character set
such as ASCII, ISO-8859-1 or UTF-8, but I suppose you already have valid
unit tests on this..

Also note that the issue you were raising a year ago regarding the
method signature change introduced by Dave's patch is still there... You
may wan't to have a deeper look at this if you don't want to break
people's code in your next release.

Regards,
Olivier.

_______________________________________________
To control your jdom-interest membership:
http://www.jdom.org/mailman/options/jdom-interest/youraddr@...