Please make CharsetDecoder less strict in SessionInputBufferImpl

[Joost van de Wijgerd]

Hello HttpComponents Developers,

My company just went in production two days ago with a new server partly
based on your http-nio package. We have created
a custom http connector as well as a servlet based connector (tomcat).
We started using httpcore-nio because we expect to
do a lot more concurrents with it. We are running in the following
problem:

2009-06-30 14:14:11,234 ERROR [I/O dispatcher 4] -
java.nio.charset.MalformedInputException: Input length = 1
java.nio.charset.MalformedInputException: Input length = 1
        at
java.nio.charset.CoderResult.throwException(CoderResult.java:260)
        at
org.apache.http.impl.nio.reactor.SessionInputBufferImpl.readLine(Session
InputBufferImpl.java:190)
        at
org.apache.http.impl.nio.codecs.AbstractMessageParser.parse(AbstractMess
ageParser.java:183)
        at
org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput(Defau
ltNHttpServerConnection.java:158)
        at
org.apache.http.impl.nio.DefaultServerIOEventDispatch.inputReady(Default
ServerIOEventDispatch.java:145)
        at
org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.ja
va:153)
        at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(Abstract
IOReactor.java:314)
        at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(Abstrac
tIOReactor.java:294)
        at
org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIORea
ctor.java:256)
        at
org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.jav
a:96)
        at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run
(AbstractMultiworkerIOReactor.java:556)
        at java.lang.Thread.run(Thread.java:619)

This is due to the fact that one of our client apps (iphone app) sends
some data in UTF-16 format while we are expecting
UTF-8.

Because of the current strict way you have configured the CharSetDecoder
we get an exception and the connection is terminated. The
same code running on the Tomcat servlet connector gets accepted by
tomcat, it replaces the offending characters with a ?

After some research I found that this was a setting of the
CharSetDecoder. I have create a patch locally that calls
CharsetDecoder.onMalformedInput(CodingErrorAction.REPLACE) in the
SessionInputBufferImpl constructor. This give me the desired result.

Maybe you can make this is a configurable option, or make this the
default?

Kind regards

Joost


------------------------------------------------------------------------
-----------------------

 
 
Joost van de Wijgerd
Architect
 
eBuddy BV
Keizersgracht 585
1017 DR  Amsterdam
The Netherlands
www.ebuddy.com
email: jvandewijgerd@...
msn: joost@...
handle: jwijgerd
 
phone: [phone]
mobile: [phone]
fax:      [phone]
 
 
------------------------------------------------------------------------
-----------------------
 


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

[olegk]

Joost van de Wijgerd wrote:

> Hello HttpComponents Developers,
>
> My company just went in production two days ago with a new server partly
> based on your http-nio package. We have created
> a custom http connector as well as a servlet based connector (tomcat).
> We started using httpcore-nio because we expect to
> do a lot more concurrents with it. We are running in the following
> problem:
>
> 2009-06-30 14:14:11,234 ERROR [I/O dispatcher 4] -
> java.nio.charset.MalformedInputException: Input length = 1
> java.nio.charset.MalformedInputException: Input length = 1
>         at
> java.nio.charset.CoderResult.throwException(CoderResult.java:260)
>         at
> org.apache.http.impl.nio.reactor.SessionInputBufferImpl.readLine(Session
> InputBufferImpl.java:190)
>         at
> org.apache.http.impl.nio.codecs.AbstractMessageParser.parse(AbstractMess
> ageParser.java:183)
>         at
> org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput(Defau
> ltNHttpServerConnection.java:158)
>         at
> org.apache.http.impl.nio.DefaultServerIOEventDispatch.inputReady(Default
> ServerIOEventDispatch.java:145)
>         at
> org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.ja
> va:153)
>         at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(Abstract
> IOReactor.java:314)
>         at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(Abstrac
> tIOReactor.java:294)
>         at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIORea
> ctor.java:256)
>         at
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.jav
> a:96)
>         at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run
> (AbstractMultiworkerIOReactor.java:556)
>         at java.lang.Thread.run(Thread.java:619)
>
> This is due to the fact that one of our client apps (iphone app) sends
> some data in UTF-16 format while we are expecting
> UTF-8.
>
> Because of the current strict way you have configured the CharSetDecoder
> we get an exception and the connection is terminated. The
> same code running on the Tomcat servlet connector gets accepted by
> tomcat, it replaces the offending characters with a ?
>
> After some research I found that this was a setting of the
> CharSetDecoder. I have create a patch locally that calls
> CharsetDecoder.onMalformedInput(CodingErrorAction.REPLACE) in the
> SessionInputBufferImpl constructor. This give me the desired result.
>
> Maybe you can make this is a configurable option, or make this the
> default?
>
> Kind regards
>
> Joost
>

Hi Joost

Please do feel free to submit a patch addressing the issue. I personally
would prefer a configurable parameter for controlling the behavior of
CharsetDecoder in that regard.

By the way, you can force HttpCore to use a non-standard Charset for a
group of requests by using the 'http.protocol.element-charset'

Hope this helps

Oleg

> ------------------------------------------------------------------------
> -----------------------
>
>  
>  
> Joost van de Wijgerd
> Architect
>  
> eBuddy BV
> Keizersgracht 585
> 1017 DR  Amsterdam
> The Netherlands
> www.ebuddy.com
> email: jvandewijgerd@...
> msn: joost@...
> handle: jwijgerd
>  
> phone: [phone]
> mobile: [phone]
> fax:      [phone]
>  
>  
> ------------------------------------------------------------------------
> -----------------------
>  
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...