ResponseBodyAsStream Truncation

[Shelley, Ryan]

Hi all... I have, what seems, a pretty basic need, but I'm hitting a
roadblock.  I get a response from an HttpClient/HttpMethod GET request
and when I use "responseBody" I get the full response, but also a
warning that I should be using "responseBodyAsStream".  I changed my
code to use "responseBodyAsStream" and when I look at what I get, it is
always truncated.  Switching back to "responseBody," it once again comes
back complete.  Am I not implementing the stream properly?

 

int size = 0;

byte[] buffer = new byte[1024];

StringBuffer responseBuff = new StringBuffer();

InputStream in = null;

 

try{

      in = method.getResponseBodyAsStream();

      while ((size = in.read(buffer)) != -1)

            responseBuff.append(new String(buffer));

}finally{

      in.close();

}

                       

String xmlStr = responseBuff.toString();

 

When I run the above code and attach my debugger, "xmlStr" is always
truncated when I use "responseBodyAsStream".  As I said, switching back
to "responseBody" works fine and the response is complete.  I've tried
using a byte array and a char array (the reason I use the byte array
here is that the response is saved to a file).

 

I'm using HttpClient 3.1, and am not currently able to upgrade to 4.0
due to library dependencies.

 

Thoughts?  Thanks!

 

-Ryan

[Sam Berlin]

Although I can't explain the truncation, there is a clear bug in the code.
When doing a read from an InputStream, it's not guaranteed to fill in the
entire buffer.  You're appending a String built from the entire buffer, not
the amount of bytes that were read into the buffer.  Also, the code is
ignoring the character set.  Can you guarantee the response character set is
the same as the machine's default?  Otherwise, appending byte-by-byte can go
very very wrong.  (Even if it's the same, if the character set supports
multibyte characters, appending byte-by-byte could still go horribly wrong.)

Sam

On Fri, Sep 11, 2009 at 5:44 PM, Shelley, Ryan <Ryan.Shelley@...>wrote:

> Hi all... I have, what seems, a pretty basic need, but I'm hitting a
> roadblock.  I get a response from an HttpClient/HttpMethod GET request
> and when I use "responseBody" I get the full response, but also a
> warning that I should be using "responseBodyAsStream".  I changed my
> code to use "responseBodyAsStream" and when I look at what I get, it is
> always truncated.  Switching back to "responseBody," it once again comes
> back complete.  Am I not implementing the stream properly?
>
>
>
> int size = 0;
>
> byte[] buffer = new byte[1024];
>
> StringBuffer responseBuff = new StringBuffer();
>
> InputStream in = null;
>
>
>
> try{
>
>      in = method.getResponseBodyAsStream();
>
>      while ((size = in.read(buffer)) != -1)
>
>            responseBuff.append(new String(buffer));
>
> }finally{
>
>      in.close();
>
> }
>
>
>
> String xmlStr = responseBuff.toString();
>
>
>
> When I run the above code and attach my debugger, "xmlStr" is always
> truncated when I use "responseBodyAsStream".  As I said, switching back
> to "responseBody" works fine and the response is complete.  I've tried
> using a byte array and a char array (the reason I use the byte array
> here is that the response is saved to a file).
>
>
>
> I'm using HttpClient 3.1, and am not currently able to upgrade to 4.0
> due to library dependencies.
>
>
>
> Thoughts?  Thanks!
>
>
>
> -Ryan
>
>

[Shelley, Ryan]

Well, I'm not certain how to block until I know the InputStream is done
receiving all possible bytes.  I'm going off when "read" returns -1,
meaning end of stream.  Do I loop until "read" throws an IOException
(meaning the connection is closed)?

I absolutely understand your concern about the byte array.  I saw the
charset parameter, but I haven't yet researched getting the charset from
the HttpMethod object.  Once I get the stream issue resolved, I'll make
sure to address that.

-Ryan

-----Original Message-----
From: httpclient-users-return-7294-Ryan.Shelley=disney.com@...
[mailto:httpclient-users-return-7294-Ryan.Shelley=disney.com@...
rg] On Behalf Of Sam Berlin
Sent: Friday, September 11, 2009 3:00 PM
To: HttpClient User Discussion
Subject: Re: ResponseBodyAsStream Truncation

Although I can't explain the truncation, there is a clear bug in the
code.
When doing a read from an InputStream, it's not guaranteed to fill in
the
entire buffer.  You're appending a String built from the entire buffer,
not
the amount of bytes that were read into the buffer.  Also, the code is
ignoring the character set.  Can you guarantee the response character
set is
the same as the machine's default?  Otherwise, appending byte-by-byte
can go
very very wrong.  (Even if it's the same, if the character set supports
multibyte characters, appending byte-by-byte could still go horribly
wrong.)

Sam

On Fri, Sep 11, 2009 at 5:44 PM, Shelley, Ryan
<Ryan.Shelley@...>wrote:

> Hi all... I have, what seems, a pretty basic need, but I'm hitting a
> roadblock.  I get a response from an HttpClient/HttpMethod GET request
> and when I use "responseBody" I get the full response, but also a
> warning that I should be using "responseBodyAsStream".  I changed my
> code to use "responseBodyAsStream" and when I look at what I get, it
is
> always truncated.  Switching back to "responseBody," it once again
comes

> back complete.  Am I not implementing the stream properly?
>
>
>
> int size = 0;
>
> byte[] buffer = new byte[1024];
>
> StringBuffer responseBuff = new StringBuffer();
>
> InputStream in = null;
>
>
>
> try{
>
>      in = method.getResponseBodyAsStream();
>
>      while ((size = in.read(buffer)) != -1)
>
>            responseBuff.append(new String(buffer));
>
> }finally{
>
>      in.close();
>
> }
>
>
>
> String xmlStr = responseBuff.toString();
>
>
>
> When I run the above code and attach my debugger, "xmlStr" is always
> truncated when I use "responseBodyAsStream".  As I said, switching

back

> to "responseBody" works fine and the response is complete.  I've tried
> using a byte array and a char array (the reason I use the byte array
> here is that the response is saved to a file).
>
>
>
> I'm using HttpClient 3.1, and am not currently able to upgrade to 4.0
> due to library dependencies.
>
>
>
> Thoughts?  Thanks!
>
>
>
> -Ryan
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...