making a client for downloading file through http

[Håkon Sagehaug]

Hi all,

I'm making a web service that should be bale to download files from ftp,
http etc. And now I-m making the http part and wondering the best way of
doing that using the httpcomponents libraries. The files that can be
downloaded can be small or large(100->Mb).  The scenario is that the client
gives the service a ftp or http url and the service downloads it to the
services local file system.
For now I've used the http core package for downloading files, without
connection re-use but will add that. So my question is what is the most
efficient way of doing this using the libraries. I've looked a little at nio
component. What I want i guess is that the client calling the service should
not wait until the download is complete before an response is given, but
just be notified that the download has been started. I guess one option is
to do that in a thread, another is to use httpnio

Any tips or suggestions how to solve this?

cheers, Håkon

--
Håkon Sagehaug, Scientific Programmer
Parallab, Bergen Center for Computational Science (BCCS)
UNIFOB AS (University of Bergen Research Company)
Hakon.Sagehaug@..., phone [phone]

[olegk]

Håkon Sagehaug wrote:

> Hi all,
>
> I'm making a web service that should be bale to download files from ftp,
> http etc. And now I-m making the http part and wondering the best way of
> doing that using the httpcomponents libraries. The files that can be
> downloaded can be small or large(100->Mb).  The scenario is that the client
> gives the service a ftp or http url and the service downloads it to the
> services local file system.
> For now I've used the http core package for downloading files, without
> connection re-use but will add that. So my question is what is the most
> efficient way of doing this using the libraries.

Hi Håkon

I think HttpCore (classic) is a good choice as long as you do not mind
managing connection persistence in your code.

  I've looked a little at nio
> component. What I want i guess is that the client calling the service should
> not wait until the download is complete before an response is given, but
> just be notified that the download has been started. I guess one option is
> to do that in a thread, another is to use httpnio
>

I do not think HttpCore NIO is necessary in such a case. NIO is
_significantly_ more complex and in my opinion offers little benefit for
client side HTTP. HttpCore (classic) will only block until a response
head is fully received. So, your could can examine the status code and
headers before downloading the response body from the content input
stream. I see little wrong with using a worker thread to download the
response content as long as the number of worker threads remains low
(<50 or so).

> Any tips or suggestions how to solve this?
>

Hope this answers your question.

Cheers

Oleg

> cheers, Håkon
>


---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

[Håkon Sagehaug]

Hi Oleg

Thanks for the tips, I think I'll go for the classic way. I followed the [1]
instroduction and I guess thats a good starting point.

I get sometimes this error

org.apache.http.NoHttpResponseException: The target server failed to respond

Waht can i do to avoid this, is it a timeout isssue. My code is mroe or less
that same setup as shown in [1].

cheers, håkon

[1] http://wiki.apache.org/HttpComponents/GeneralHttpCoreIntroduction



2009/11/6 Oleg Kalnichevski <olegk@...>

> Håkon Sagehaug wrote:
>
>> Hi all,
>>
>> I'm making a web service that should be bale to download files from ftp,
>> http etc. And now I-m making the http part and wondering the best way of
>> doing that using the httpcomponents libraries. The files that can be
>> downloaded can be small or large(100->Mb).  The scenario is that the
>> client
>> gives the service a ftp or http url and the service downloads it to the
>> services local file system.
>> For now I've used the http core package for downloading files, without
>> connection re-use but will add that. So my question is what is the most
>> efficient way of doing this using the libraries.
>>
>
> Hi Håkon
>
> I think HttpCore (classic) is a good choice as long as you do not mind
> managing connection persistence in your code.
>
>
>  I've looked a little at nio
>
>> component. What I want i guess is that the client calling the service
>> should
>> not wait until the download is complete before an response is given, but
>> just be notified that the download has been started. I guess one option is
>> to do that in a thread, another is to use httpnio
>>
>>
> I do not think HttpCore NIO is necessary in such a case. NIO is
> _significantly_ more complex and in my opinion offers little benefit for
> client side HTTP. HttpCore (classic) will only block until a response head
> is fully received. So, your could can examine the status code and headers
> before downloading the response body from the content input stream. I see
> little wrong with using a worker thread to download the response content as
> long as the number of worker threads remains low (<50 or so).
>
>
>  Any tips or suggestions how to solve this?
>>
>>
> Hope this answers your question.
>
> Cheers
>
> Oleg
>
>  cheers, Håkon
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>
>

--
Håkon Sagehaug, Scientific Programmer
Parallab, Bergen Center for Computational Science (BCCS)
UNIFOB AS (University of Bergen Research Company)
Hakon.Sagehaug@..., phone [phone]

[olegk]

Håkon Sagehaug wrote:
> Hi Oleg
>
> Thanks for the tips, I think I'll go for the classic way. I followed the [1]
> instroduction and I guess thats a good starting point.
>

Hi Håkon

Make sure you have also read this tutorial:

http://hc.apache.org/httpcomponents-core/tutorial/html/index.html


> I get sometimes this error
>
> org.apache.http.NoHttpResponseException: The target server failed to respond
>
> Waht can i do to avoid this, is it a timeout isssue. My code is mroe or less
> that same setup as shown in [1].
>

HttpCore throws this exception if the target server fails to return
_any_ response of what so ever and prematurely terminates the
connection. This can happen, for instance, when the target server is
being under heavy load and is unable to allocate a worker for a new
connection, or due to unexpected error in the request processing code.
However, there can be no 100% certainty about that. Usually the server
logs is the only definitive source of information about that kind of
problems.

Cheers

Oleg

> cheers, håkon
>
> [1] http://wiki.apache.org/HttpComponents/GeneralHttpCoreIntroduction
>
>
>
> 2009/11/6 Oleg Kalnichevski <olegk@...>
>
>> Håkon Sagehaug wrote:
>>
>>> Hi all,
>>>
>>> I'm making a web service that should be bale to download files from ftp,
>>> http etc. And now I-m making the http part and wondering the best way of
>>> doing that using the httpcomponents libraries. The files that can be
>>> downloaded can be small or large(100->Mb).  The scenario is that the
>>> client
>>> gives the service a ftp or http url and the service downloads it to the
>>> services local file system.
>>> For now I've used the http core package for downloading files, without
>>> connection re-use but will add that. So my question is what is the most
>>> efficient way of doing this using the libraries.
>>>
>> Hi Håkon
>>
>> I think HttpCore (classic) is a good choice as long as you do not mind
>> managing connection persistence in your code.
>>
>>
>>  I've looked a little at nio
>>
>>> component. What I want i guess is that the client calling the service
>>> should
>>> not wait until the download is complete before an response is given, but
>>> just be notified that the download has been started. I guess one option is
>>> to do that in a thread, another is to use httpnio
>>>
>>>
>> I do not think HttpCore NIO is necessary in such a case. NIO is
>> _significantly_ more complex and in my opinion offers little benefit for
>> client side HTTP. HttpCore (classic) will only block until a response head
>> is fully received. So, your could can examine the status code and headers
>> before downloading the response body from the content input stream. I see
>> little wrong with using a worker thread to download the response content as
>> long as the number of worker threads remains low (<50 or so).
>>
>>
>>  Any tips or suggestions how to solve this?
>>>
>> Hope this answers your question.
>>
>> Cheers
>>
>> Oleg
>>
>>  cheers, Håkon
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
>> For additional commands, e-mail: httpclient-users-help@...
>>
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

[Håkon Sagehaug]

Hi Oleg

see inline

2009/11/6 Oleg Kalnichevski <olegk@...>

> Håkon Sagehaug wrote:
>
>> Hi Oleg
>>
>> Thanks for the tips, I think I'll go for the classic way. I followed the
>> [1]
>> instroduction and I guess thats a good starting point.
>>
>>
> Hi Håkon
>
> Make sure you have also read this tutorial:
>
> http://hc.apache.org/httpcomponents-core/tutorial/html/index.html

Yes, I've actually looked at this also

>
>
>
>  I get sometimes this error
>>
>> org.apache.http.NoHttpResponseException: The target server failed to
>> respond
>>
>> Waht can i do to avoid this, is it a timeout isssue. My code is mroe or
>> less
>> that same setup as shown in [1].
>>
>>
> HttpCore throws this exception if the target server fails to return _any_
> response of what so ever and prematurely terminates the connection. This can
> happen, for instance, when the target server is being under heavy load and
> is unable to allocate a worker for a new connection, or due to unexpected
> error in the request processing code. However, there can be no 100%
> certainty about that. Usually the server logs is the only definitive source
> of information about that kind of problems.
>

The strange thing is that if I used HTTPClient for the job of downloading I
don't seem to have the problem. So just wondering if there where some
parameters one have to set in the http-core that is set in the http-client
component. But I can check the server logs also

cheers, Håkon

>
> Cheers
>
> Oleg
>
>
>
>
>  cheers, håkon
>>
>> [1] http://wiki.apache.org/HttpComponents/GeneralHttpCoreIntroduction
>>
>>
>>
>> 2009/11/6 Oleg Kalnichevski <olegk@...>
>>
>>  Håkon Sagehaug wrote:
>>>
>>>  Hi all,
>>>>
>>>> I'm making a web service that should be bale to download files from ftp,
>>>> http etc. And now I-m making the http part and wondering the best way of
>>>> doing that using the httpcomponents libraries. The files that can be
>>>> downloaded can be small or large(100->Mb).  The scenario is that the
>>>> client
>>>> gives the service a ftp or http url and the service downloads it to the
>>>> services local file system.
>>>> For now I've used the http core package for downloading files, without
>>>> connection re-use but will add that. So my question is what is the most
>>>> efficient way of doing this using the libraries.
>>>>
>>>>  Hi Håkon
>>>
>>> I think HttpCore (classic) is a good choice as long as you do not mind
>>> managing connection persistence in your code.
>>>
>>>
>>>  I've looked a little at nio
>>>
>>>  component. What I want i guess is that the client calling the service
>>>> should
>>>> not wait until the download is complete before an response is given, but
>>>> just be notified that the download has been started. I guess one option
>>>> is
>>>> to do that in a thread, another is to use httpnio
>>>>
>>>>
>>>>  I do not think HttpCore NIO is necessary in such a case. NIO is
>>> _significantly_ more complex and in my opinion offers little benefit for
>>> client side HTTP. HttpCore (classic) will only block until a response
>>> head
>>> is fully received. So, your could can examine the status code and headers
>>> before downloading the response body from the content input stream. I see
>>> little wrong with using a worker thread to download the response content
>>> as
>>> long as the number of worker threads remains low (<50 or so).
>>>
>>>
>>>  Any tips or suggestions how to solve this?
>>>
>>>>
>>>>  Hope this answers your question.
>>>
>>> Cheers
>>>
>>> Oleg
>>>
>>>  cheers, Håkon
>>>
>>>>
>>>>  ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
>>> For additional commands, e-mail: httpclient-users-help@...
>>>
>>>
>>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>
>

--
Håkon Sagehaug, Scientific Programmer
Parallab, Bergen Center for Computational Science (BCCS)
UNIFOB AS (University of Bergen Research Company)
Hakon.Sagehaug@..., phone [phone]

[olegk]

Håkon Sagehaug wrote:

> Hi Oleg
>
> see inline
>
> 2009/11/6 Oleg Kalnichevski <olegk@...>
>
>> Håkon Sagehaug wrote:
>>
>>> Hi Oleg
>>>
>>> Thanks for the tips, I think I'll go for the classic way. I followed the
>>> [1]
>>> instroduction and I guess thats a good starting point.
>>>
>>>
>> Hi Håkon
>>
>> Make sure you have also read this tutorial:
>>
>> http://hc.apache.org/httpcomponents-core/tutorial/html/index.html
>
> Yes, I've actually looked at this also
>
>>
>>
>>  I get sometimes this error
>>> org.apache.http.NoHttpResponseException: The target server failed to
>>> respond
>>>
>>> Waht can i do to avoid this, is it a timeout isssue. My code is mroe or
>>> less
>>> that same setup as shown in [1].
>>>
>>>
>> HttpCore throws this exception if the target server fails to return _any_
>> response of what so ever and prematurely terminates the connection. This can
>> happen, for instance, when the target server is being under heavy load and
>> is unable to allocate a worker for a new connection, or due to unexpected
>> error in the request processing code. However, there can be no 100%
>> certainty about that. Usually the server logs is the only definitive source
>> of information about that kind of problems.
>>
> The strange thing is that if I used HTTPClient for the job of downloading I
> don't seem to have the problem. So just wondering if there where some
> parameters one have to set in the http-core that is set in the http-client
> component. But I can check the server logs also
>

There is no special parameter. HttpClient simply employs a retry handler
to re-try such requests automatically:

http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d4e280

Oleg

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

[Håkon Sagehaug]

Hi

can one add a retry handler to the http core way of doing it also?

cheers, Håkon

2009/11/6 Oleg Kalnichevski <olegk@...>

> Håkon Sagehaug wrote:
>
>> Hi Oleg
>>
>> see inline
>>
>> 2009/11/6 Oleg Kalnichevski <olegk@...>
>>
>>  Håkon Sagehaug wrote:
>>>
>>>  Hi Oleg
>>>>
>>>> Thanks for the tips, I think I'll go for the classic way. I followed the
>>>> [1]
>>>> instroduction and I guess thats a good starting point.
>>>>
>>>>
>>>>  Hi Håkon
>>>
>>> Make sure you have also read this tutorial:
>>>
>>> http://hc.apache.org/httpcomponents-core/tutorial/html/index.html
>>>
>>
>> Yes, I've actually looked at this also
>>
>>
>>>
>>>  I get sometimes this error
>>>
>>>> org.apache.http.NoHttpResponseException: The target server failed to
>>>> respond
>>>>
>>>> Waht can i do to avoid this, is it a timeout isssue. My code is mroe or
>>>> less
>>>> that same setup as shown in [1].
>>>>
>>>>
>>>>  HttpCore throws this exception if the target server fails to return
>>> _any_
>>> response of what so ever and prematurely terminates the connection. This
>>> can
>>> happen, for instance, when the target server is being under heavy load
>>> and
>>> is unable to allocate a worker for a new connection, or due to unexpected
>>> error in the request processing code. However, there can be no 100%
>>> certainty about that. Usually the server logs is the only definitive
>>> source
>>> of information about that kind of problems.
>>>
>>>  The strange thing is that if I used HTTPClient for the job of
>> downloading I
>> don't seem to have the problem. So just wondering if there where some
>> parameters one have to set in the http-core that is set in the http-client
>> component. But I can check the server logs also
>>
>>
> There is no special parameter. HttpClient simply employs a retry handler to
> re-try such requests automatically:
>
>
> http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d4e280
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>
>

--
Håkon Sagehaug, Scientific Programmer
Parallab, Bergen Center for Computational Science (BCCS)
UNIFOB AS (University of Bergen Research Company)
Hakon.Sagehaug@..., phone [phone]

[sebb-2-2]

You might also want to look at Commons VFS:

http://commons.apache.org/vfs/

On 06/11/2009, Håkon Sagehaug <Hakon.Sagehaug@...> wrote:

> Hi
>
>  can one add a retry handler to the http core way of doing it also?
>
>  cheers, Håkon
>
>
>  2009/11/6 Oleg Kalnichevski <olegk@...>
>
>  > Håkon Sagehaug wrote:
>  >
>  >> Hi Oleg
>  >>
>  >> see inline
>  >>
>  >> 2009/11/6 Oleg Kalnichevski <olegk@...>
>  >>
>  >>  Håkon Sagehaug wrote:
>  >>>
>  >>>  Hi Oleg
>  >>>>
>  >>>> Thanks for the tips, I think I'll go for the classic way. I followed the
>  >>>> [1]
>  >>>> instroduction and I guess thats a good starting point.
>  >>>>
>  >>>>
>  >>>>  Hi Håkon
>  >>>
>  >>> Make sure you have also read this tutorial:
>  >>>
>  >>> http://hc.apache.org/httpcomponents-core/tutorial/html/index.html
>  >>>
>  >>
>  >> Yes, I've actually looked at this also
>  >>
>  >>
>  >>>
>  >>>  I get sometimes this error
>  >>>
>  >>>> org.apache.http.NoHttpResponseException: The target server failed to
>  >>>> respond
>  >>>>
>  >>>> Waht can i do to avoid this, is it a timeout isssue. My code is mroe or
>  >>>> less
>  >>>> that same setup as shown in [1].
>  >>>>
>  >>>>
>  >>>>  HttpCore throws this exception if the target server fails to return
>  >>> _any_
>  >>> response of what so ever and prematurely terminates the connection. This
>  >>> can
>  >>> happen, for instance, when the target server is being under heavy load
>  >>> and
>  >>> is unable to allocate a worker for a new connection, or due to unexpected
>  >>> error in the request processing code. However, there can be no 100%
>  >>> certainty about that. Usually the server logs is the only definitive
>  >>> source
>  >>> of information about that kind of problems.
>  >>>
>  >>>  The strange thing is that if I used HTTPClient for the job of
>  >> downloading I
>  >> don't seem to have the problem. So just wondering if there where some
>  >> parameters one have to set in the http-core that is set in the http-client
>  >> component. But I can check the server logs also
>  >>
>  >>
>  > There is no special parameter. HttpClient simply employs a retry handler to
>  > re-try such requests automatically:
>  >
>  >
>  > http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d4e280
>  >
>  > Oleg
>  >
>  >
>  > ---------------------------------------------------------------------
>  > To unsubscribe, e-mail: httpclient-users-unsubscribe@...
>  > For additional commands, e-mail: httpclient-users-help@...
>  >
>  >
>
>
>
> --
>  Håkon Sagehaug, Scientific Programmer
>  Parallab, Bergen Center for Computational Science (BCCS)
>  UNIFOB AS (University of Bergen Research Company)
>  Hakon.Sagehaug@..., phone [phone]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

[olegk]

Håkon Sagehaug wrote:
> Hi
>
> can one add a retry handler to the http core way of doing it also?
>

Sure. HttpClient uses HttpCore internally.

Oleg

> cheers, Håkon
>
> 2009/11/6 Oleg Kalnichevski <olegk@...>
>
>> Håkon Sagehaug wrote:
>>
>>> Hi Oleg
>>>
>>> see inline
>>>
>>> 2009/11/6 Oleg Kalnichevski <olegk@...>
>>>
>>>  Håkon Sagehaug wrote:
>>>>  Hi Oleg
>>>>> Thanks for the tips, I think I'll go for the classic way. I followed the
>>>>> [1]
>>>>> instroduction and I guess thats a good starting point.
>>>>>
>>>>>
>>>>>  Hi Håkon
>>>> Make sure you have also read this tutorial:
>>>>
>>>> http://hc.apache.org/httpcomponents-core/tutorial/html/index.html
>>>>
>>> Yes, I've actually looked at this also
>>>
>>>
>>>>  I get sometimes this error
>>>>
>>>>> org.apache.http.NoHttpResponseException: The target server failed to
>>>>> respond
>>>>>
>>>>> Waht can i do to avoid this, is it a timeout isssue. My code is mroe or
>>>>> less
>>>>> that same setup as shown in [1].
>>>>>
>>>>>
>>>>>  HttpCore throws this exception if the target server fails to return
>>>> _any_
>>>> response of what so ever and prematurely terminates the connection. This
>>>> can
>>>> happen, for instance, when the target server is being under heavy load
>>>> and
>>>> is unable to allocate a worker for a new connection, or due to unexpected
>>>> error in the request processing code. However, there can be no 100%
>>>> certainty about that. Usually the server logs is the only definitive
>>>> source
>>>> of information about that kind of problems.
>>>>
>>>>  The strange thing is that if I used HTTPClient for the job of
>>> downloading I
>>> don't seem to have the problem. So just wondering if there where some
>>> parameters one have to set in the http-core that is set in the http-client
>>> component. But I can check the server logs also
>>>
>>>
>> There is no special parameter. HttpClient simply employs a retry handler to
>> re-try such requests automatically:
>>
>>
>> http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d4e280
>>
>> Oleg
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
>> For additional commands, e-mail: httpclient-users-help@...
>>
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

[Håkon Sagehaug]

Hi

Soemthing like

HttpRequestRetryHandler myRetryHandler = new HttpRequestRetryHandler() {

        public boolean retryRequest(IOException exception,
            int executionCount, HttpContext context) {
            if (executionCount >= 5) {
            // Do not retry if over max retry count
            return false;
            }
            if (exception instanceof NoHttpResponseException) {
            // Retry if the server dropped connection on us
            return true;
            }
            if (exception instanceof SSLHandshakeException) {
            // Do not retry on SSL handshake exception
            return false;
            }
            HttpRequest request = (HttpRequest) context
                .getAttribute(ExecutionContext.HTTP_REQUEST);
            boolean idempotent = !(request instanceof
HttpEntityEnclosingRequest);
            if (idempotent) {
            // Retry if the request is considered idempotent
            return true;
            }
            return false;
        }

        };

        params.setParameter(HttpMethodParams.RETRY_HANDLER, myRetryHandler);

Where params is a object of HttpParams class.

Håkon

2009/11/6 Oleg Kalnichevski <olegk@...>

> Håkon Sagehaug wrote:
>
>> Hi
>>
>> can one add a retry handler to the http core way of doing it also?
>>
>>
> Sure. HttpClient uses HttpCore internally.
>
> Oleg
>
>
>  cheers, Håkon
>>
>> 2009/11/6 Oleg Kalnichevski <olegk@...>
>>
>>  Håkon Sagehaug wrote:
>>>
>>>  Hi Oleg
>>>>
>>>> see inline
>>>>
>>>> 2009/11/6 Oleg Kalnichevski <olegk@...>
>>>>
>>>>  Håkon Sagehaug wrote:
>>>>
>>>>>  Hi Oleg
>>>>>
>>>>>> Thanks for the tips, I think I'll go for the classic way. I followed
>>>>>> the
>>>>>> [1]
>>>>>> instroduction and I guess thats a good starting point.
>>>>>>
>>>>>>
>>>>>>  Hi Håkon
>>>>>>
>>>>> Make sure you have also read this tutorial:
>>>>>
>>>>> http://hc.apache.org/httpcomponents-core/tutorial/html/index.html
>>>>>
>>>>>  Yes, I've actually looked at this also
>>>>
>>>>
>>>>   I get sometimes this error
>>>>>
>>>>>  org.apache.http.NoHttpResponseException: The target server failed to
>>>>>> respond
>>>>>>
>>>>>> Waht can i do to avoid this, is it a timeout isssue. My code is mroe
>>>>>> or
>>>>>> less
>>>>>> that same setup as shown in [1].
>>>>>>
>>>>>>
>>>>>>  HttpCore throws this exception if the target server fails to return
>>>>>>
>>>>> _any_
>>>>> response of what so ever and prematurely terminates the connection.
>>>>> This
>>>>> can
>>>>> happen, for instance, when the target server is being under heavy load
>>>>> and
>>>>> is unable to allocate a worker for a new connection, or due to
>>>>> unexpected
>>>>> error in the request processing code. However, there can be no 100%
>>>>> certainty about that. Usually the server logs is the only definitive
>>>>> source
>>>>> of information about that kind of problems.
>>>>>
>>>>>  The strange thing is that if I used HTTPClient for the job of
>>>>>
>>>> downloading I
>>>> don't seem to have the problem. So just wondering if there where some
>>>> parameters one have to set in the http-core that is set in the
>>>> http-client
>>>> component. But I can check the server logs also
>>>>
>>>>
>>>>  There is no special parameter. HttpClient simply employs a retry
>>> handler to
>>> re-try such requests automatically:
>>>
>>>
>>>
>>> http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d4e280
>>>
>>> Oleg
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
>>> For additional commands, e-mail: httpclient-users-help@...
>>>
>>>
>>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>
>

--
Håkon Sagehaug, Scientific Programmer
Parallab, Bergen Center for Computational Science (BCCS)
UNIFOB AS (University of Bergen Research Company)
Hakon.Sagehaug@..., phone [phone]

[olegk]

Håkon Sagehaug wrote:

> Hi
>
> Soemthing like
>
> HttpRequestRetryHandler myRetryHandler = new HttpRequestRetryHandler() {
>
>         public boolean retryRequest(IOException exception,
>             int executionCount, HttpContext context) {
>             if (executionCount >= 5) {
>             // Do not retry if over max retry count
>             return false;
>             }
>             if (exception instanceof NoHttpResponseException) {
>             // Retry if the server dropped connection on us
>             return true;
>             }
>             if (exception instanceof SSLHandshakeException) {
>             // Do not retry on SSL handshake exception
>             return false;
>             }
>             HttpRequest request = (HttpRequest) context
>                 .getAttribute(ExecutionContext.HTTP_REQUEST);
>             boolean idempotent = !(request instanceof
> HttpEntityEnclosingRequest);
>             if (idempotent) {
>             // Retry if the request is considered idempotent
>             return true;
>             }
>             return false;
>         }
>
>         };
>
>         params.setParameter(HttpMethodParams.RETRY_HANDLER, myRetryHandler);
>
> Where params is a object of HttpParams class.
>

Yeah, right, but you would still have to write some code that actually
makes use of that retry handler like DefaultRequestDirector in HttpClient.

Oleg

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...