Nabble - HttpComponents

[jira] Resolved: (HTTPCORE-211) Memory leak outside of the Heap when using non-keepalive SSL

2009-12-17T03:42:18Z

[ https://issues.apache.org/jira/browse/HTTPCORE-211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Asankha C. Perera resolved HTTPCORE-211.
----------------------------------------

Resolution: Fixed

Fixed with svn commit: r891654

Sorry about the many line separator differences, the main fix was only to replace the "ByteBuffer.allocateDirect" calls with "ByteBuffer.allocate" to allocate on heap

> Memory leak outside of the Heap when using non-keepalive SSL
> ------------------------------------------------------------
>
> Key: HTTPCORE-211
> URL: https://issues.apache.org/jira/browse/HTTPCORE-211
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore NIO
> Affects Versions: 4.0.1
> Reporter: Asankha C. Perera
> Assignee: Asankha C. Perera
> Fix For: 4.0.2
>
>
> The issue is discussed here : http://markmail.org/message/7ni2qrgebajkvrzn
> Although seen during load testing with non-keepalive SSL, the leak would be present in the keep-alive scenario too, although the leak would be less. The issue was seen on RHEL and CentOS, but not on Ubuntu.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

From forum: HttpComponents-Dev

[jira] Created: (HTTPCORE-211) Memory leak outside of the Heap when using non-keepalive SSL

2009-12-17T03:24:18Z

Memory leak outside of the Heap when using non-keepalive SSL
------------------------------------------------------------

Key: HTTPCORE-211
URL: https://issues.apache.org/jira/browse/HTTPCORE-211
Project: HttpComponents HttpCore
Issue Type: Bug
Components: HttpCore NIO
Affects Versions: 4.0.1
Reporter: Asankha C. Perera
Assignee: Asankha C. Perera
Fix For: 4.0.2

The issue is discussed here : http://markmail.org/message/7ni2qrgebajkvrzn

Although seen during load testing with non-keepalive SSL, the leak would be present in the keep-alive scenario too, although the leak would be less. The issue was seen on RHEL and CentOS, but not on Ubuntu.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

From forum: HttpComponents-Dev

Re: Using multiple IP addresses

2009-12-16T17:28:32Z

Thanks, that is much easier than expected!
Tony

> Message du 16/12/09 21:44
> De : "Oleg Kalnichevski"
> A : "HttpClient User Discussion"
> Copie à :
> Objet : Re: Using multiple IP addresses
>
>
> Tony Poppleton wrote:
> > Hi Christine,
> >
> > I am just looking for the simplest solution. Using one http client and one connection manager would be nice, although not necessary if that is overcomplicated.
> >
> > Thanks,
> > Tony
> >
> >
>
> 'http.route.local-address' parameter is your friend.
>
> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e473
>
> Oleg
>
> >
> >> Message du 15/12/09 14:30
> >> De : "Christine Karman"
> >> A : "HttpClient User Discussion"
> >> Copie à :
> >> Objet : Re: Using multiple IP addresses
> >>
> >>
> >> Tony Poppleton wrote:
> >>> Hi,
> >>>
> >>> I have two IP addresses on my computer, and would like to send out requests on both using a single Java application and HttpClient. Is this possible? I.e. one request sent out from the first IP, and the next request sent out from the second IP.
> >>>
> >> Do you want one instance of HttpClient to do so, and one connectionManager?
> >>
> >> dagdag
> >> Christine
> >>> Thanks,
> >>> Tony
> >>>
> >>
> >> --
> >> dagdag is just a two-character rotation of byebye.
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> >> For additional commands, e-mail: httpclient-users-help@...
> >>
> >> ---------------------------------------------------------------------------------------
> >> Orange vous informe que cet e-mail a ete controle par l'anti-virus mail.
> >> Aucun virus connu a ce jour par nos services n'a ete detecte.
> >>
> >>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>
> ---------------------------------------------------------------------------------------
> Orange vous informe que cet e-mail a ete controle par l'anti-virus mail.
> Aucun virus connu a ce jour par nos services n'a ete detecte.
>
>
>

From forum: HttpClient-User

Re: Using multiple IP addresses

2009-12-16T12:44:17Z

Tony Poppleton wrote:
> Hi Christine,
>
> I am just looking for the simplest solution. Using one http client and one connection manager would be nice, although not necessary if that is overcomplicated.
>
> Thanks,
> Tony
>
>

'http.route.local-address' parameter is your friend.

http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e473

Oleg

>
>> Message du 15/12/09 14:30
>> De : "Christine Karman"
>> A : "HttpClient User Discussion"
>> Copie à :
>> Objet : Re: Using multiple IP addresses
>>
>>
>> Tony Poppleton wrote:
>>> Hi,
>>>
>>> I have two IP addresses on my computer, and would like to send out requests on both using a single Java application and HttpClient. Is this possible? I.e. one request sent out from the first IP, and the next request sent out from the second IP.
>>>
>> Do you want one instance of HttpClient to do so, and one connectionManager?
>>
>> dagdag
>> Christine
>>> Thanks,
>>> Tony
>>>
>>
>> --
>> dagdag is just a two-character rotation of byebye.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
>> For additional commands, e-mail: httpclient-users-help@...
>>
>> ---------------------------------------------------------------------------------------
>> Orange vous informe que cet e-mail a ete controle par l'anti-virus mail.
>> Aucun virus connu a ce jour par nos services n'a ete detecte.
>>
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User

Re: Kerberos proxy authentication issue

2009-12-16T12:40:50Z

Sebastiaan van Erk wrote:

> Hi Oleg,
>
> Thanks for adding this. I'm currently a bit stuck on the caching of the
> tickets, which really needs to be fixed, but it looks like I'll have to
> dive into protocols/APIs deeply (JAAS, Java GSSAPI, SPNEGO) to figure it
> out.
>
> There is however one thing about the SPENGO authentication protocol that
> does not yet fit nicely into the httpclient API, namely, if you look at
> the diagram:
>
> http://issues.apache.org/jira/secure/attachment/12419383/SPNEGO_cropped.png
>
> What you see is a series of requests back and forth, with the last
> response containing the final negotiation token NOT having a response
> code of 401. In the diagram it's a 200, but in my case (with the
> redirect), it was a 3xx. In any case, the token should go back into the
> authentication scheme.
>

This is somewhat similar to NTLM authentication, where the final
response code may also be 3xx, so HttpClient should be able to handler
such situations, worst case with some API extensions.

> BTW, I quickly hacked a response inteceptor which would do this for me
> (it was a hack, because it just called authenticate() again with the
> token and a null HttpRequest (since you don't have a request in the
> response interceptor) and at least GSSAPI was able to complete the
> negotiation. I did this because I was hoping this would solve the ticket
> caching problem, i.e, hoping that the tickets would be "committed" if
> the negotiation completed, but unfortunately this was not the case.
>
> I don't really know the authentication API should look to support this
> protocol... and it might in the end not really be necessary, except
> perhaps for mutual authentication.
>
> I'm going to look into the SPNEGO/GSSAPI stuff now to fix the caching,
> so I'm still on this. I just wanted to keep you posted.
>
> Regards,
> Sebastiaan

Keep us posted on the progress.

Cheers

Oleg

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User

Re: Using multiple IP addresses

2009-12-15T08:44:30Z

Hi Christine,

I am just looking for the simplest solution. Using one http client and one connection manager would be nice, although not necessary if that is overcomplicated.

Thanks,
Tony

> Message du 15/12/09 14:30
> De : "Christine Karman"
> A : "HttpClient User Discussion"
> Copie à :
> Objet : Re: Using multiple IP addresses
>
>
> Tony Poppleton wrote:
> > Hi,
> >
> > I have two IP addresses on my computer, and would like to send out requests on both using a single Java application and HttpClient. Is this possible? I.e. one request sent out from the first IP, and the next request sent out from the second IP.
> >
> Do you want one instance of HttpClient to do so, and one connectionManager?
>
> dagdag
> Christine
> > Thanks,
> > Tony
> >
>
>
> --
> dagdag is just a two-character rotation of byebye.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>
> ---------------------------------------------------------------------------------------
> Orange vous informe que cet e-mail a ete controle par l'anti-virus mail.
> Aucun virus connu a ce jour par nos services n'a ete detecte.
>
>
>

From forum: HttpClient-User

Re: Using multiple IP addresses

2009-12-15T05:29:28Z

Tony Poppleton wrote:
> Hi,
>
> I have two IP addresses on my computer, and would like to send out requests on both using a single Java application and HttpClient. Is this possible? I.e. one request sent out from the first IP, and the next request sent out from the second IP.
>
Do you want one instance of HttpClient to do so, and one connectionManager?

dagdag
Christine
> Thanks,
> Tony
>

--
dagdag is just a two-character rotation of byebye.

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User

Using multiple IP addresses

2009-12-14T14:31:36Z

Hi,

I have two IP addresses on my computer, and would like to send out requests on both using a single Java application and HttpClient. Is this possible? I.e. one request sent out from the first IP, and the next request sent out from the second IP.

Thanks,
Tony

From forum: HttpClient-User

Re: Memory leak with non-keepalive SSL connection load test

2009-12-14T07:49:01Z

If IOSessions are disposed of properly, ByteBufferAllocater can become
ByteBufferManager & the ByteBuffers can be returned & recycled/reused.

Sam

On Mon, Dec 14, 2009 at 10:20 AM, Oleg Kalnichevski <olegk@...>wrote:

> Asankha C. Perera wrote:
>
>> Hi Oleg
>>
>>> Has anyone encountered this issue before? my immediate next step is to
>>>> try on a 64bit OS version .. but I am curious to know how we could avoid
>>>> this problem
>>>>
>>>>
>>> Asankha,
>>>
>>> I think the easiest fix to the problem should be avoiding direct
>>> bytebuffers altogether. Feel free to go ahead and patch SSLIOSession in
>>> SVN trunk.
>>>
>>>
>> Thanks for the reply, I still got the same issue with the 64 bit OS
>> version too, so probably Ubuntu has something that causes the JVM to GC
>> before its too late, which is not there in RHEL. As a temporary fix, I
>> replaced the direct buffers with heap buffers which solved the problem,
>> and I think it may indeed be the safer bet right now.
>>
>> thanks
>> asankha
>>
>>
> Asankha,
>
> Actually I do not think this should be seen as a temporary fix. Direct
> buffers should be used for objects with a relatively long life span. I/O
> sessions may well be short lived. Alternatively, feel free to change
> SSLIOsession to make use of ByteBufferAllocator, which would allow for
> customizable allocation strategies.
>
> Cheers
>
> Oleg
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>

From forum: HttpComponents-Dev

Re: Memory leak with non-keepalive SSL connection load test

2009-12-14T07:20:52Z

Asankha C. Perera wrote:

> Hi Oleg
>>> Has anyone encountered this issue before? my immediate next step is to
>>> try on a 64bit OS version .. but I am curious to know how we could avoid
>>> this problem
>>>
>> Asankha,
>>
>> I think the easiest fix to the problem should be avoiding direct
>> bytebuffers altogether. Feel free to go ahead and patch SSLIOSession in
>> SVN trunk.
>>
> Thanks for the reply, I still got the same issue with the 64 bit OS
> version too, so probably Ubuntu has something that causes the JVM to GC
> before its too late, which is not there in RHEL. As a temporary fix, I
> replaced the direct buffers with heap buffers which solved the problem,
> and I think it may indeed be the safer bet right now.
>
> thanks
> asankha
>

Asankha,

Actually I do not think this should be seen as a temporary fix. Direct
buffers should be used for objects with a relatively long life span. I/O
sessions may well be short lived. Alternatively, feel free to change
SSLIOsession to make use of ByteBufferAllocator, which would allow for
customizable allocation strategies.

Cheers

Oleg

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

From forum: HttpComponents-Dev

Re: Kerberos proxy authentication issue

2009-12-14T04:56:40Z

Oleg Kalnichevski wrote:

> Sebastiaan van Erk wrote:
>> Oleg Kalnichevski wrote:
>>> Oleg Kalnichevski wrote:
>>>> Sebastiaan van Erk wrote:
>>>>> Oleg Kalnichevski wrote:
>>>>>> Sebastiaan van Erk wrote:
>>>>>>> Hi Oleg,
>>>>>>>
>>>>>>> Thanks for your reply.
>>>>>>>
>>>>>>> There's a good chance I'm going to have to get this working, even
>>>>>>> if it
>>>>>>> means I'm going to have to delve into this myself. I'll contact the
>>>>>>> original developer and see if he sees anything obvious, but in
>>>>>>> any case,
>>>>>>> if I succeed in getting it working, I will happily contribute the
>>>>>>> patches.
>>>>>>>
>>>>>>> Best regards,
>>>>>>> Sebastiaan
>>>>>>>
>>>>>>
>>>>>> Hi Sebastiaan
>>>>>>
>>>>>> Cool. In my turn I will happily help with HttpClient specific stuff.
>>>>>
>>>>> Hi Oleg,
>>>>>
>>>>> I got the proxy authentication working already. I even tested it
>>>>> when both proxy auth and http auth are required and it works.
>>>>>
>>>>> There is just one small issue: I need the proxy host name to
>>>>> generate the kerberos service name (HTTP/proxyhost@REALM). The
>>>>> current NegotiateScheme code doesn't handle proxy auth at all and
>>>>> always uses the target host name which it retrieves from the Host
>>>>> header (is that the best way to get the target host?):
>>>>>
>>>>> if (isStripPort()) {
>>>>>
>>>>> init((request.getLastHeader("Host")).getValue().replaceAll(":[0-9]+$",
>>>>> "") );
>>>>> } else {
>>>>> init( (request.getLastHeader("Host")).getValue());
>>>>> }
>>>>>
>>>>> I changed the NegotiateScheme to extend AuthSchemeBase (like
>>>>> NTLMScheme). Now I have access to the isProxy() method and can add
>>>>> the correct header (Proxy-Authorization or Authorization). However,
>>>>> to determine the hostname I currently have this:
>>>>>
>>>>> String host;
>>>>> if (isProxy()) {
>>>>> // FIXME this should actually be determined by the route planner?
>>>>> HttpHost proxy =
>>>>> ConnRouteParams.getDefaultProxy(request.getParams());
>>>>> host = proxy.getHostName();
>>>>> } else {
>>>>> host = request.getLastHeader("Host").getValue();
>>>>> }
>>>>> if (isStripPort()) {
>>>>> host = host.replaceAll(":[0-9]+$", "");
>>>>> }
>>>>> init(host);
>>>>>
>>>>> I noticed a few frames up in DefaultRequestDirector.handleResponse
>>>>> the actual proxy host is known:
>>>>>
>>>>> if (this.proxyAuthHandler.isAuthenticationRequested(response,
>>>>> context)) {
>>>>>
>>>>> HttpHost proxy = route.getProxyHost();
>>>>>
>>>>> this.log.debug("Proxy requested authentication");
>>>>> Map<String, Header> challenges =
>>>>> this.proxyAuthHandler.getChallenges(response, context);
>>>>> try {
>>>>> processChallenges(challenges,
>>>>> this.proxyAuthState, this.proxyAuthHandler,
>>>>> response, context);
>>>>> } catch (AuthenticationException ex) {
>>>>> ...
>>>>>
>>>>> But this information is not available to me at the FIXME location.
>>>>> I'm also ignoring any forced route now, but it seems wrong to copy
>>>>> paste the code from DefaultHttpRoutePlanner anyway, especially
>>>>> since that's just the default implementation anyhow and could be
>>>>> overridden.
>>>>>
>>>>> Next thing I'll look into is why the redirect fails.
>>>>>
>>>>> Regards,
>>>>> Sebastiaan
>>>>>
>>>>
>>>> Hi Sebastiaan
>>>>
>>>> This looks nasty. The usual way of obtaining contextual details in
>>>> HttpClient is by examining attributes of the HttpContext instance
>>>> associated with the request being executed. The trouble is that auth
>>>> schemes factories presently have no means of getting hold of the
>>>> HttpContext.
>>>>
>>>> One possibility of fixing it would be extending or replacing the
>>>> AuthSchemeFactory with something better.
>>>>
>>>> ---
>>>> public interface BetterAuthSchemeFactory extends AuthSchemeFactory {
>>>>
>>>> AuthScheme newInstance(HttpContext context, HttpParams params);
>>>>
>>>> }
>>>> ---
>>>>
>>>> This is double but is far from trivial if 100% binary compatibility
>>>> with 4.0 is to be retained.
>>>>
>>>> Feel free to go ahead and open a change request in JIRA for this issue.
>>>>
>>>> Oleg
>>>>
>>>
>>> Probably a better alternative would be something like that
>>>
>>> ---------
>>> /**
>>> * This interface represents an extended authentication scheme
>>> * that requires access to {@link HttpContext} in order to
>>> * generate an authorization string.
>>> *
>>> * @since 4.1
>>> */
>>>
>>> public interface ContextAwareAuthScheme extends AuthScheme {
>>>
>>> /**
>>> * Produces an authorization string for the given set of
>>> * {@link Credentials}.
>>> *
>>> * @param credentials The set of credentials to be used for
>>> athentication
>>> * @param request The request being authenticated
>>> * @param context HTTP context
>>> * @throws AuthenticationException if authorization string cannot
>>> * be generated due to an authentication failure
>>> *
>>> * @return the authorization string
>>> */
>>> Header authenticate(
>>> Credentials credentials,
>>> HttpRequest request,
>>> HttpContext context) throws AuthenticationException;
>>>
>>> }
>>>
>>> ---------
>>>
>>> Oleg
>>
>> Hi Oleg,
>>
>> The ContextAwareAuthScheme looks perfect for the job.
>>
>> Should I open a JIRA issue for it?
>>
>
> Yes, you should
>
>> How will binary compatibility be maintained? An instanceof check at
>> the location where AuthScheme is used at the moment?
>>
>
> Yep
>
> Oleg

Hi Oleg,

Thanks for adding this. I'm currently a bit stuck on the caching of the
tickets, which really needs to be fixed, but it looks like I'll have to
dive into protocols/APIs deeply (JAAS, Java GSSAPI, SPNEGO) to figure it
out.

There is however one thing about the SPENGO authentication protocol that
does not yet fit nicely into the httpclient API, namely, if you look at
the diagram:

http://issues.apache.org/jira/secure/attachment/12419383/SPNEGO_cropped.png

What you see is a series of requests back and forth, with the last
response containing the final negotiation token NOT having a response
code of 401. In the diagram it's a 200, but in my case (with the
redirect), it was a 3xx. In any case, the token should go back into the
authentication scheme.

BTW, I quickly hacked a response inteceptor which would do this for me
(it was a hack, because it just called authenticate() again with the
token and a null HttpRequest (since you don't have a request in the
response interceptor) and at least GSSAPI was able to complete the
negotiation. I did this because I was hoping this would solve the ticket
caching problem, i.e, hoping that the tickets would be "committed" if
the negotiation completed, but unfortunately this was not the case.

I don't really know the authentication API should look to support this
protocol... and it might in the end not really be necessary, except
perhaps for mutual authentication.

I'm going to look into the SPNEGO/GSSAPI stuff now to fix the caching,
so I'm still on this. I just wanted to keep you posted.

Regards,
Sebastiaan

smime.p7s (4K) Download Attachment

From forum: HttpClient-User

Re: Memory leak with non-keepalive SSL connection load test

2009-12-14T04:16:02Z

As I see it direct buffers are allocated using glibc functions. Could be the
glibc version that causes the different behaviour.

On 14.12.2009 12:54, Asankha C. Perera wrote:
> Thanks for the reply, I still got the same issue with the 64 bit OS
> version too, so probably Ubuntu has something that causes the JVM to GC
> before its too late, which is not there in RHEL. As a temporary fix, I
> replaced the direct buffers with heap buffers which solved the problem,
> and I think it may indeed be the safer bet right now.
>
> thanks
> asankha

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

From forum: HttpComponents-Dev

Re: Memory leak with non-keepalive SSL connection load test

2009-12-14T03:54:12Z

Hi Oleg

>> Has anyone encountered this issue before? my immediate next step is to
>> try on a 64bit OS version .. but I am curious to know how we could avoid
>> this problem
>>
> Asankha,
>
> I think the easiest fix to the problem should be avoiding direct
> bytebuffers altogether. Feel free to go ahead and patch SSLIOSession in
> SVN trunk.
>

Thanks for the reply, I still got the same issue with the 64 bit OS
version too, so probably Ubuntu has something that causes the JVM to GC
before its too late, which is not there in RHEL. As a temporary fix, I
replaced the direct buffers with heap buffers which solved the problem,
and I think it may indeed be the safer bet right now.

thanks
asankha

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

From forum: HttpComponents-Dev

[jira] Closed: (HTTPCLIENT-755) Use of java.net.URI.resolve() is buggy

2009-12-14T02:34:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johannes Koch closed HTTPCLIENT-755.
------------------------------------

> Use of java.net.URI.resolve() is buggy
> --------------------------------------
>
> Key: HTTPCLIENT-755
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-755
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Reporter: Johannes Koch
> Fix For: 4.0 Alpha 4
>
> Attachments: URIresolve_patch_20080228.txt
>
>
> The use of java.net.URI.resolve() is buggy (see <http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4708535>). Affected class: org.apache.http.impl.client.DefaultRedirectHandler.
> Proposed solution: Create a resolve(URI, String) method in o.a.h.client.utils.URLUtils.

From forum: HttpComponents-Dev

[jira] Closed: (HTTPCLIENT-705) Handle URIs with path component null

2009-12-14T02:34:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johannes Koch closed HTTPCLIENT-705.
------------------------------------

Thanks

> Handle URIs with path component null
> ------------------------------------
>
> Key: HTTPCLIENT-705
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-705
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Reporter: Johannes Koch
> Fix For: 4.0 Alpha 3
>
>
> HttpClient does not handle URIs with path component null (e.g. http://google.com) the same as path component '/'. This results e.g. in a ProtocolException "The server failed to respond with a valid HTTP response".

From forum: HttpComponents-Dev

[jira] Closed: (HTTPCLIENT-758) Wrong method signatures in AbstractHttpClient

2009-12-14T02:34:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johannes Koch closed HTTPCLIENT-758.
------------------------------------

> Wrong method signatures in AbstractHttpClient
> ---------------------------------------------
>
> Key: HTTPCLIENT-758
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-758
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Reporter: Johannes Koch
> Fix For: 4.0 Alpha 4
>
> Attachments: HTTPCLIENT-758_patch.txt
>
>
> The method signatures for removeRequestInterceptorByClass and removeResponseInterceptorByClass in AbstractHttpClient are wrong. Must be
> public void removeRequestInterceptorByClass(Class<? extends
> HttpRequestInterceptor> clazz);
> and
> public void removeResponseInterceptorByClass(Class<? extends
> HttpRequestInterceptor> clazz);

From forum: HttpComponents-Dev

[jira] Closed: (HTTPCLIENT-861) URI reference resolution fails examples in RFC 3986

2009-12-14T02:34:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johannes Koch closed HTTPCLIENT-861.
------------------------------------

> URI reference resolution fails examples in RFC 3986
> ---------------------------------------------------
>
> Key: HTTPCLIENT-861
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-861
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 4.0 Beta 2
> Reporter: Johannes Koch
> Priority: Minor
> Fix For: 4.0 Final
>
> Attachments: HTTPCLIENT-861-patch.txt, TestURIUtils.java, TestURIUtils.java
>
>
> org.apache.http.client.utils.URIUtils.resolve(final URI baseURI, URI reference) fails to resolve some examples from RFC 3986 section 5.3 correctly. See TestCase.

From forum: HttpComponents-Dev

[jira] Closed: (HTTPCLIENT-847) Handling of a buggy response to HEAD request

2009-12-14T02:34:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johannes Koch closed HTTPCLIENT-847.
------------------------------------

> Handling of a buggy response to HEAD request
> --------------------------------------------
>
> Key: HTTPCLIENT-847
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-847
> Project: HttpComponents HttpClient
> Issue Type: Question
> Affects Versions: 4.0 Beta 2
> Environment: Java 1.5.0_14
> Reporter: Johannes Koch
> Priority: Minor
> Attachments: TestHeadRequestWithBuggyServer.java
>
>
> I tried to make a HEAD request to some server and discovered, that the server seems to respond in some buggy way. The response is a 302 and contains an entity.
> This makes HttpClient waiting and wainting and ... when redirecting is true.
> What would be the preferred way to make HttpClient stop processing the response in this situation?

From forum: HttpComponents-Dev

[jira] Closed: (HTTPCLIENT-702) Compilation errors in DefaultResponseParser

2009-12-14T02:32:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johannes Koch closed HTTPCLIENT-702.
------------------------------------

> Compilation errors in DefaultResponseParser
> -------------------------------------------
>
> Key: HTTPCLIENT-702
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-702
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Reporter: Johannes Koch
>
> I have two compilation errors in o.a.h.impl.conn.DefaultResponseParser:
> 1. in
> protected HttpMessage parseHead(
> final SessionInputBuffer sessionBuffer) throws IOException, HttpException:
> if (lineParser.hasProtocolVersion(this.lineBuf, 0)) {
> The method hasProtocolVersion(CharArrayBuffer, ParserCursor) in the type LineParser is not applicable for the arguments (CharArrayBuffer, int)
> 2. in
> protected HttpMessage parseHead(
> final SessionInputBuffer sessionBuffer) throws IOException, HttpException:
> StatusLine statusline = lineParser.parseStatusLine(this.lineBuf, 0, this.lineBuf.length());
> The method parseStatusLine(CharArrayBuffer, ParserCursor) in the type LineParser is not applicable for the arguments (CharArrayBuffer, int, int)

From forum: HttpComponents-Dev

[jira] Closed: (HTTPCLIENT-704) Exception not caugh in DefaultResponseParser

2009-12-14T02:32:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johannes Koch closed HTTPCLIENT-704.
------------------------------------

> Exception not caugh in DefaultResponseParser
> --------------------------------------------
>
> Key: HTTPCLIENT-704
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-704
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Reporter: Johannes Koch
> Fix For: 4.0 Alpha 3
>
>
> The method hasProtocolVersion in o.a.h.message.BaseLineParser (httpcore-alpha6) throws an IndexOutOfBoundsException which is not caught by the parseHead method in the o.a.h.impl.conn.DefaultResponseParser.

From forum: HttpComponents-Dev

[jira] Closed: (HTTPCLIENT-703) Compilation error in RFC2617Scheme

2009-12-14T02:32:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Johannes Koch closed HTTPCLIENT-703.
------------------------------------

> Compilation error in RFC2617Scheme
> ----------------------------------
>
> Key: HTTPCLIENT-703
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-703
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Reporter: Johannes Koch
>
> I have a compilation error in o.a.h.impl.auth.RFC2617Scheme:
> in
> public void processChallenge(final Header header) throws MalformedChallengeException:
> HeaderElement[] elements = BasicHeaderValueParser.DEFAULT.parseElements(buffer, pos, buffer.length());
> The method parseElements(CharArrayBuffer, ParserCursor) in the type BasicHeaderValueParser is not applicable for the arguments (CharArrayBuffer, int, int)

From forum: HttpComponents-Dev

Re: Memory leak with non-keepalive SSL connection load test

2009-12-14T01:13:50Z

On Mon, 2009-12-14 at 01:15 +0530, Asankha C. Perera wrote:

> Hi All
>
> I've encountered a strange problem from a Synapse user using RHEL 32 bit
> system, where a load test of non-keepalive SSL connections causes the
> process to leak memory and ultimately crash. This does not happen on
> Ubuntu which I use by default, and I had to run a CentOS instance on EC2
> to reproduce and see this problem happening. The JDK I used was the
> latest 1.6.17 - but I came across the links [1] and [2] from the past
> which seems to indicate the problem
>
> The problem seems to be due to the four direct byte buffers allocated
> per each SSL connection, and possibly because the JVM does not GC since
> the byte buffers are allocated outside of it. The memory leak is visible
> to vmstat by rapidly declining free memory on the system, but not
> visible on the heap memory usage. The stack trace before the crash I got
> was:
>
> Exception in thread "https-Listener I/O dispatcher-2"
> java.lang.OutOfMemoryError: Direct buffer memory
> at java.nio.Bits.reserveMemory(Bits.java:633)
> at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
> at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
> at
> org.apache.http.impl.nio.reactor.SSLIOSession.<init>(SSLIOSession.java:113)
> at
> org.apache.http.impl.nio.SSLServerIOEventDispatch.connected(SSLServerIOEventDispatch.java:110)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.keyCreated(BaseIOReactor.java:182)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processNewChannels(AbstractIOReactor.java:246)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:153)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:70)
> at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318)
> at java.lang.Thread.run(Thread.java:619)
>
> Has anyone encountered this issue before? my immediate next step is to
> try on a 64bit OS version .. but I am curious to know how we could avoid
> this problem
>

Asankha,

I think the easiest fix to the problem should be avoiding direct
bytebuffers altogether. Feel free to go ahead and patch SSLIOSession in
SVN trunk.

Cheers

Oleg

> cheers
> asankha
>
> [1] http://lists.apple.com/archives/Java-dev/2006/Feb/msg00059.html
> [2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092131
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

From forum: HttpComponents-Dev

Memory leak with non-keepalive SSL connection load test

2009-12-13T11:45:37Z

Hi All

I've encountered a strange problem from a Synapse user using RHEL 32 bit
system, where a load test of non-keepalive SSL connections causes the
process to leak memory and ultimately crash. This does not happen on
Ubuntu which I use by default, and I had to run a CentOS instance on EC2
to reproduce and see this problem happening. The JDK I used was the
latest 1.6.17 - but I came across the links [1] and [2] from the past
which seems to indicate the problem

The problem seems to be due to the four direct byte buffers allocated
per each SSL connection, and possibly because the JVM does not GC since
the byte buffers are allocated outside of it. The memory leak is visible
to vmstat by rapidly declining free memory on the system, but not
visible on the heap memory usage. The stack trace before the crash I got
was:

Exception in thread "https-Listener I/O dispatcher-2"
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at
org.apache.http.impl.nio.reactor.SSLIOSession.<init>(SSLIOSession.java:113)
at
org.apache.http.impl.nio.SSLServerIOEventDispatch.connected(SSLServerIOEventDispatch.java:110)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.keyCreated(BaseIOReactor.java:182)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.processNewChannels(AbstractIOReactor.java:246)
at
org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:153)
at
org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:70)
at
org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:318)
at java.lang.Thread.run(Thread.java:619)

Has anyone encountered this issue before? my immediate next step is to
try on a 64bit OS version .. but I am curious to know how we could avoid
this problem

cheers
asankha

[1] http://lists.apple.com/archives/Java-dev/2006/Feb/msg00059.html
[2] http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5092131

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

From forum: HttpComponents-Dev

[jira] Commented: (HTTPCLIENT-876) Calling httpClient.execute(post) on a shared server causes security error (WRITE not allowed to protected area on disk)

2009-12-12T13:17:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12789793#action_12789793 ]

Kiran Kumar commented on HTTPCLIENT-876:
----------------------------------------

Hi Cliff,

Were you able to set up connection pooling on your Godaddy hosting environment using context.xml ? I have been trying this for around 10 days , but no luck . My environment Linux Shared Hosting with java supported

My directory structure
/WEB-INF/web.xml
/META-INF/context.xml

what I tried

1) Copied manually context.xml file under /META-INF , this didn't work
2) Created war file and deployed under root directory even this not helped me
3) context file parameters
<Context path="/" docBase=""

since I am copying under root folder , not sure with what names it creates under conf\Catalina\localhost .

Could you please help me ? ( Searching internet , talking to godaddy and trying to explain them the problem , almost stopped all my work)

Thanks for your help

> Calling httpClient.execute(post) on a shared server causes security error (WRITE not allowed to protected area on disk)
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-876
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-876
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Components: HttpClient
> Affects Versions: 4.0 Final
> Environment: Java 5.0, Tomcat
> Reporter: Clifford
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> I run my JSP modules on a shared server at GoDaddy.com, one of the largest hosting companies in the USA. They have strict security on the servers which disallows writing to any disk files unless they are in the /temp directory.
>
> When I first tried to execute a module I wrote using HttpClient, I got a security write-not-allowed error. I looked at the stack trace and found out that org.apache.http.impl.client.DefaultHttpClient.java (at source line 197) calls org.apache.http.util.VersionInfo method loadVersionInfo, and that class (at source line 248) tries to do a FILE WRITE after not finding a property file containing the version#. That WRITE is disallowed by my hosting, thus causing my HttpClient call to fail. I can provide more details if you like.
>
> I worked around the problem by commenting out the call to loadVersionInfo and recompiling DefaultHttpClient, but MANY MANY programmers will run into this issue, so I would label it an urgent bug that needs to be fixed. Suggestions for the fix could be 1) hard-code the version in a new final static variable of DefaultHttpClient, or 2) Write the Properties file containing the HttpClient version# to a directory within /temp.
> The stack trace (transcribed from a printout) is:
> java.security.AccessControlException: access denied (java.io.FilePermission /web/tomcat/work/hosting/dir.dotgreen.org/.../loader/META-INF write) at ... 5 levels of java.security.* then
> java.io.File.mkdir
> WebappClassLoader.findResourceInternal
> WebappClassLoader.findResource
> WebappClassLoader.getResourceAsStream
> VersionInfo.loadVersionInfo (line 244)
> DefaultHttpClient.createHttpParams (line 197)
> AbstractHttpClient.getParams (line 293)
> DefaultHttpClient.createClient (line 2)
> AbstractHttpClient.getConnectionManager (line 312)
> DefaultHttpClient.createHttpContext (line 254)
> AbstractHttpClient.execute (line 618)
> AbstractHttpClient.execute (line 576)
> AbstractHttpClient.execute (line 554)
> then a dozen JSP/catalina locations that are irrelevant

From forum: HttpComponents-Dev

Re: How to read chunked response entity? (hc 4.0.1)

2009-12-12T06:18:53Z

Sam Crawford wrote:

> It's demonstrated right at the top of
> http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html
> - that will work for both chunked and non-chunked responses.

"Chapter 1. Fundamentals" Ouch! :-)

If I am right, you mean just reading the content InputStream,
regardless of the Transfer-Encoding. Ahem, this is what I did,
in a separate, reproducible, standalone main class, and it
failed. And now that I re-run this "reproducible" example, it
works :-/ Quite embarrassing...

Sorry for the noise, and thanks for the reminder of the
tutorial! Regards,

--
Florent Georges
http://www..fgeorges.org/

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User

Re: How to read chunked response entity? (hc 4.0.1)

2009-12-12T05:33:52Z

It's demonstrated right at the top of
http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html
- that will work for both chunked and non-chunked responses.

As you probably know, you won't be able to call getContentLength() on
a chunked response entity. You just have to read the InputStream until
EOF.

Thanks,

Sam

2009/12/12 Florent Georges <lists@...>:

> Hi,
>
> I send a GET request to a server which responds with a chunked
> entity content. I've looked in the documentation, but I only
> found info about how to produce chunked content in a request, not
> about how to consume chunked content from a response.
>
> Did I miss something? How can I retrieve the entity content?
> I use HttpCore 4.0.1 and HttpClient 4.0.1.
>
> Regards,
>
> --
> Florent Georges
> http://www.fgeorges.org/
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User

How to read chunked response entity? (hc 4.0.1)

2009-12-12T05:21:45Z

Hi,

I send a GET request to a server which responds with a chunked
entity content. I've looked in the documentation, but I only
found info about how to produce chunked content in a request, not
about how to consume chunked content from a response.

Did I miss something? How can I retrieve the entity content?
I use HttpCore 4.0.1 and HttpClient 4.0.1.

Regards,

--
Florent Georges
http://www.fgeorges.org/

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User

[jira] Updated: (HTTPCLIENT-901) Add a ContextAwareAuthScheme that has access to the HttpContext in the authenticate method

2009-12-11T15:38:18Z

[ https://issues.apache.org/jira/browse/HTTPCLIENT-901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Oleg Kalnichevski updated HTTPCLIENT-901:
-----------------------------------------

Priority: Major (was: Minor)

I added ContextAwareAuthScheme interface and tweaked RequestProxyAuthentication and RequestTargetAuthentication interceptors

http://svn.apache.org/viewvc?view=revision&revision=889860

It is all yours now.

Oleg

> Add a ContextAwareAuthScheme that has access to the HttpContext in the authenticate method
> ------------------------------------------------------------------------------------------
>
> Key: HTTPCLIENT-901
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-901
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpAuth
> Affects Versions: 4.1 Alpha1
> Reporter: Sebastiaan van Erk
>
> The interface to be added would be:
> /**
> * This interface represents an extended authentication scheme
> * that requires access to {@link HttpContext} in order to
> * generate an authorization string.
> *
> * @since 4.1
> */
> public interface ContextAwareAuthScheme extends AuthScheme {
> /**
> * Produces an authorization string for the given set of
> * {@link Credentials}.
> *
> * @param credentials The set of credentials to be used for athentication
> * @param request The request being authenticated
> * @param context HTTP context
> * @throws AuthenticationException if authorization string cannot
> * be generated due to an authentication failure
> *
> * @return the authorization string
> */
> Header authenticate(
> Credentials credentials,
> HttpRequest request,
> HttpContext context) throws AuthenticationException;
> }
> Binary compatibility can be maintained by doing an instanceof check at the location where AuthScheme.authenticate() is called at the moment, and calling the context aware version if available.
> This interface is necessary for the NegotiateScheme authentication scheme because the service names for the authentication tickets are based on the hostname of the target host or proxy host, depending on whether it's normal or proxy authentication, and this information is only available from the HttpContext.
> Without the HttpContext there is a workaround that works most of the time, which looks like this:
> String host;
> if (isProxy()) {
> // FIXME this should actually taken from the HttpContext.
> HttpHost proxy = ConnRouteParams.getDefaultProxy(request.getParams());
> host = proxy.getHostName();
> } else {
> host = request.getLastHeader("Host").getValue();
> }

From forum: HttpComponents-Dev

Re: Kerberos proxy authentication issue

2009-12-11T15:16:25Z

Oleg Kalnichevski wrote:

Ah, but this is only if the target host is different. The problem here
is that the target host is the same, but the it's a different resource,
so the negotiation must begin anew... (In my test, the redirect is from
http://tunneltest.servoy.com/private ->
http://tunneltest.servoy.com/private/

>> Another thing I want to look into is if the auth header is reusable
>> for a specific period of time, so that you can do preemptive auth
>> instead of a challenge response cycle every time. Not sure if it's
>> possible though, because that would seen to be open to a replay
>> attack. On the other hand, one can generate the first token of the
>> negotiation preemptively I think, assuming the first challenge from
>> the server is empty.
>>
>
> Take a look at how preemptive Digest auth works in 4.1. It might be
> similar to what you want:
>
> http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientPreemptiveDigestAuthentication.java

Thanks for the tip. Will look at it right after I fix the ticket caching
issue.

Regards,
Sebastiaan

> Oleg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>

smime.p7s (4K) Download Attachment

From forum: HttpClient-User

[jira] Created: (HTTPCLIENT-901) Add a ContextAwareAuthScheme that has access to the HttpContext in the authenticate method

2009-12-11T15:14:18Z

Add a ContextAwareAuthScheme that has access to the HttpContext in the authenticate method
------------------------------------------------------------------------------------------

Key: HTTPCLIENT-901
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-901
Project: HttpComponents HttpClient
Issue Type: Improvement
Components: HttpAuth
Affects Versions: 4.1 Alpha1
Reporter: Sebastiaan van Erk
Priority: Minor

The interface to be added would be:

/**
* This interface represents an extended authentication scheme
* that requires access to {@link HttpContext} in order to
* generate an authorization string.
*
* @since 4.1
*/

public interface ContextAwareAuthScheme extends AuthScheme {

/**
* Produces an authorization string for the given set of
* {@link Credentials}.
*
* @param credentials The set of credentials to be used for athentication
* @param request The request being authenticated
* @param context HTTP context
* @throws AuthenticationException if authorization string cannot
* be generated due to an authentication failure
*
* @return the authorization string
*/
Header authenticate(
Credentials credentials,
HttpRequest request,
HttpContext context) throws AuthenticationException;

}

Binary compatibility can be maintained by doing an instanceof check at the location where AuthScheme.authenticate() is called at the moment, and calling the context aware version if available.

This interface is necessary for the NegotiateScheme authentication scheme because the service names for the authentication tickets are based on the hostname of the target host or proxy host, depending on whether it's normal or proxy authentication, and this information is only available from the HttpContext.

Without the HttpContext there is a workaround that works most of the time, which looks like this:

String host;
if (isProxy()) {
// FIXME this should actually taken from the HttpContext.
HttpHost proxy = ConnRouteParams.getDefaultProxy(request.getParams());
host = proxy.getHostName();
} else {
host = request.getLastHeader("Host").getValue();
}

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

From forum: HttpComponents-Dev

Re: Kerberos proxy authentication issue

2009-12-11T15:13:25Z

Oleg Kalnichevski wrote:

https://issues.apache.org/jira/browse/HTTPCLIENT-901

Regards,
Sebastiaan

smime.p7s (4K) Download Attachment

From forum: HttpClient-User

Re: Kerberos proxy authentication issue

2009-12-11T15:03:05Z

Sebastiaan van Erk wrote:

> Oleg Kalnichevski wrote:
>> Sebastiaan van Erk wrote:
>>> Hi Oleg,
>>>
>>> Thanks for your reply.
>>>
>>> There's a good chance I'm going to have to get this working, even if it
>>> means I'm going to have to delve into this myself. I'll contact the
>>> original developer and see if he sees anything obvious, but in any case,
>>> if I succeed in getting it working, I will happily contribute the
>>> patches.
>>>
>>> Best regards,
>>> Sebastiaan
>>>
>>
>> Hi Sebastiaan
>>
>> Cool. In my turn I will happily help with HttpClient specific stuff.
>
> Hi Oleg,
>
> I also found the reason why the redirect doesn't work. Basically it
> comes down to the following:
>
> The Kerberos auth can have a sequence of challenge/responses, that keep
> resulting in 401 until the authentication succeeds. After each
> challenge, the authenticate method in the same NegotiateScheme instance
> is called, and it processes this new information. So far so good.
>
> But when the request gets redirected to a new URL it basically means the
> auth succeeded, and the NegotiateScheme instance is in "ESTABLISHED"
> state for the OLD request.
>
> Now the new request also gets a 401, and authenticate is called again,
> however, it is called in the OLD instance, for the OLD request.
> Basically a *new* instance from the scheme factory is necessary for the
> new request.
>
> I can think of two fixes:
> 1) if http client detects a redirect, it also throws out the old
> AuthScheme and uses the auth scheme factory to generate a new one
> 2) the NegotiateScheme instance detects that the URL has changed by
> inspecting the request, and resets its internal state.
>
> I like solution number 1 better, but I'm not sure of what the
> consequences might be for the other auth schemes. It seems to me that it
> is the right approach though, no matter what the scheme; if you follow a
> redirect, just create a new instance.
>

hhhm. HttpClient should be doing exactly that

http://hc.apache.org/httpcomponents-client/httpclient/xref/org/apache/http/impl/client/DefaultRequestDirector.html#999

>
> Another thing I want to look into is if the auth header is reusable for
> a specific period of time, so that you can do preemptive auth instead of
> a challenge response cycle every time. Not sure if it's possible though,
> because that would seen to be open to a replay attack. On the other
> hand, one can generate the first token of the negotiation preemptively I
> think, assuming the first challenge from the server is empty.
>

Take a look at how preemptive Digest auth works in 4.1. It might be
similar to what you want:

http://svn.apache.org/repos/asf/httpcomponents/httpclient/trunk/httpclient/src/examples/org/apache/http/examples/client/ClientPreemptiveDigestAuthentication.java

Oleg

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User

Re: Kerberos proxy authentication issue

2009-12-11T14:58:02Z

Sebastiaan van Erk wrote:

> Oleg Kalnichevski wrote:
>> Oleg Kalnichevski wrote:
>>> Sebastiaan van Erk wrote:
>>>> Oleg Kalnichevski wrote:
>>>>> Sebastiaan van Erk wrote:
>>>>>> Hi Oleg,
>>>>>>
>>>>>> Thanks for your reply.
>>>>>>
>>>>>> There's a good chance I'm going to have to get this working, even
>>>>>> if it
>>>>>> means I'm going to have to delve into this myself. I'll contact the
>>>>>> original developer and see if he sees anything obvious, but in any
>>>>>> case,
>>>>>> if I succeed in getting it working, I will happily contribute the
>>>>>> patches.
>>>>>>
>>>>>> Best regards,
>>>>>> Sebastiaan
>>>>>>
>>>>>
>>>>> Hi Sebastiaan
>>>>>
>>>>> Cool. In my turn I will happily help with HttpClient specific stuff.
>>>>
>>>> Hi Oleg,
>>>>
>>>> I got the proxy authentication working already. I even tested it
>>>> when both proxy auth and http auth are required and it works.
>>>>
>>>> There is just one small issue: I need the proxy host name to
>>>> generate the kerberos service name (HTTP/proxyhost@REALM). The
>>>> current NegotiateScheme code doesn't handle proxy auth at all and
>>>> always uses the target host name which it retrieves from the Host
>>>> header (is that the best way to get the target host?):
>>>>
>>>> if (isStripPort()) {
>>>>
>>>> init((request.getLastHeader("Host")).getValue().replaceAll(":[0-9]+$",
>>>> "") );
>>>> } else {
>>>> init( (request.getLastHeader("Host")).getValue());
>>>> }
>>>>
>>>> I changed the NegotiateScheme to extend AuthSchemeBase (like
>>>> NTLMScheme). Now I have access to the isProxy() method and can add
>>>> the correct header (Proxy-Authorization or Authorization). However,
>>>> to determine the hostname I currently have this:
>>>>
>>>> String host;
>>>> if (isProxy()) {
>>>> // FIXME this should actually be determined by the route planner?
>>>> HttpHost proxy =
>>>> ConnRouteParams.getDefaultProxy(request.getParams());
>>>> host = proxy.getHostName();
>>>> } else {
>>>> host = request.getLastHeader("Host").getValue();
>>>> }
>>>> if (isStripPort()) {
>>>> host = host.replaceAll(":[0-9]+$", "");
>>>> }
>>>> init(host);
>>>>
>>>> I noticed a few frames up in DefaultRequestDirector.handleResponse
>>>> the actual proxy host is known:
>>>>
>>>> if (this.proxyAuthHandler.isAuthenticationRequested(response,
>>>> context)) {
>>>>
>>>> HttpHost proxy = route.getProxyHost();
>>>>
>>>> this.log.debug("Proxy requested authentication");
>>>> Map<String, Header> challenges =
>>>> this.proxyAuthHandler.getChallenges(response, context);
>>>> try {
>>>> processChallenges(challenges,
>>>> this.proxyAuthState, this.proxyAuthHandler,
>>>> response, context);
>>>> } catch (AuthenticationException ex) {
>>>> ...
>>>>
>>>> But this information is not available to me at the FIXME location.
>>>> I'm also ignoring any forced route now, but it seems wrong to copy
>>>> paste the code from DefaultHttpRoutePlanner anyway, especially since
>>>> that's just the default implementation anyhow and could be overridden.
>>>>
>>>> Next thing I'll look into is why the redirect fails.
>>>>
>>>> Regards,
>>>> Sebastiaan
>>>>
>>>
>>> Hi Sebastiaan
>>>
>>> This looks nasty. The usual way of obtaining contextual details in
>>> HttpClient is by examining attributes of the HttpContext instance
>>> associated with the request being executed. The trouble is that auth
>>> schemes factories presently have no means of getting hold of the
>>> HttpContext.
>>>
>>> One possibility of fixing it would be extending or replacing the
>>> AuthSchemeFactory with something better.
>>>
>>> ---
>>> public interface BetterAuthSchemeFactory extends AuthSchemeFactory {
>>>
>>> AuthScheme newInstance(HttpContext context, HttpParams params);
>>>
>>> }
>>> ---
>>>
>>> This is double but is far from trivial if 100% binary compatibility
>>> with 4.0 is to be retained.
>>>
>>> Feel free to go ahead and open a change request in JIRA for this issue.
>>>
>>> Oleg
>>>
>>
>> Probably a better alternative would be something like that
>>
>> ---------
>> /**
>> * This interface represents an extended authentication scheme
>> * that requires access to {@link HttpContext} in order to
>> * generate an authorization string.
>> *
>> * @since 4.1
>> */
>>
>> public interface ContextAwareAuthScheme extends AuthScheme {
>>
>> /**
>> * Produces an authorization string for the given set of
>> * {@link Credentials}.
>> *
>> * @param credentials The set of credentials to be used for
>> athentication
>> * @param request The request being authenticated
>> * @param context HTTP context
>> * @throws AuthenticationException if authorization string cannot
>> * be generated due to an authentication failure
>> *
>> * @return the authorization string
>> */
>> Header authenticate(
>> Credentials credentials,
>> HttpRequest request,
>> HttpContext context) throws AuthenticationException;
>>
>> }
>>
>> ---------
>>
>> Oleg
>
> Hi Oleg,
>
> The ContextAwareAuthScheme looks perfect for the job.
>
> Should I open a JIRA issue for it?
>

Yes, you should

> How will binary compatibility be maintained? An instanceof check at the
> location where AuthScheme is used at the moment?
>

Yep

Oleg

> Regards,
> Sebastiaan
>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
>> For additional commands, e-mail: httpclient-users-help@...
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User

Re: Kerberos proxy authentication issue

2009-12-11T14:56:39Z

Oleg Kalnichevski wrote:

> Oleg Kalnichevski wrote:
>> Sebastiaan van Erk wrote:
>>> Oleg Kalnichevski wrote:
>>>> Sebastiaan van Erk wrote:
>>>>> Hi Oleg,
>>>>>
>>>>> Thanks for your reply.
>>>>>
>>>>> There's a good chance I'm going to have to get this working, even
>>>>> if it
>>>>> means I'm going to have to delve into this myself. I'll contact the
>>>>> original developer and see if he sees anything obvious, but in any
>>>>> case,
>>>>> if I succeed in getting it working, I will happily contribute the
>>>>> patches.
>>>>>
>>>>> Best regards,
>>>>> Sebastiaan
>>>>>
>>>>
>>>> Hi Sebastiaan
>>>>
>>>> Cool. In my turn I will happily help with HttpClient specific stuff.
>>>
>>> Hi Oleg,
>>>
>>> I got the proxy authentication working already. I even tested it when
>>> both proxy auth and http auth are required and it works.
>>>
>>> There is just one small issue: I need the proxy host name to generate
>>> the kerberos service name (HTTP/proxyhost@REALM). The current
>>> NegotiateScheme code doesn't handle proxy auth at all and always uses
>>> the target host name which it retrieves from the Host header (is that
>>> the best way to get the target host?):
>>>
>>> if (isStripPort()) {
>>>
>>> init((request.getLastHeader("Host")).getValue().replaceAll(":[0-9]+$",
>>> "") );
>>> } else {
>>> init( (request.getLastHeader("Host")).getValue());
>>> }
>>>
>>> I changed the NegotiateScheme to extend AuthSchemeBase (like
>>> NTLMScheme). Now I have access to the isProxy() method and can add
>>> the correct header (Proxy-Authorization or Authorization). However,
>>> to determine the hostname I currently have this:
>>>
>>> String host;
>>> if (isProxy()) {
>>> // FIXME this should actually be determined by the route planner?
>>> HttpHost proxy =
>>> ConnRouteParams.getDefaultProxy(request.getParams());
>>> host = proxy.getHostName();
>>> } else {
>>> host = request.getLastHeader("Host").getValue();
>>> }
>>> if (isStripPort()) {
>>> host = host.replaceAll(":[0-9]+$", "");
>>> }
>>> init(host);
>>>
>>> I noticed a few frames up in DefaultRequestDirector.handleResponse
>>> the actual proxy host is known:
>>>
>>> if (this.proxyAuthHandler.isAuthenticationRequested(response,
>>> context)) {
>>>
>>> HttpHost proxy = route.getProxyHost();
>>>
>>> this.log.debug("Proxy requested authentication");
>>> Map<String, Header> challenges =
>>> this.proxyAuthHandler.getChallenges(response, context);
>>> try {
>>> processChallenges(challenges,
>>> this.proxyAuthState, this.proxyAuthHandler,
>>> response, context);
>>> } catch (AuthenticationException ex) {
>>> ...
>>>
>>> But this information is not available to me at the FIXME location.
>>> I'm also ignoring any forced route now, but it seems wrong to copy
>>> paste the code from DefaultHttpRoutePlanner anyway, especially since
>>> that's just the default implementation anyhow and could be overridden.
>>>
>>> Next thing I'll look into is why the redirect fails.
>>>
>>> Regards,
>>> Sebastiaan
>>>
>>
>> Hi Sebastiaan
>>
>> This looks nasty. The usual way of obtaining contextual details in
>> HttpClient is by examining attributes of the HttpContext instance
>> associated with the request being executed. The trouble is that auth
>> schemes factories presently have no means of getting hold of the
>> HttpContext.
>>
>> One possibility of fixing it would be extending or replacing the
>> AuthSchemeFactory with something better.
>>
>> ---
>> public interface BetterAuthSchemeFactory extends AuthSchemeFactory {
>>
>> AuthScheme newInstance(HttpContext context, HttpParams params);
>>
>> }
>> ---
>>
>> This is double but is far from trivial if 100% binary compatibility
>> with 4.0 is to be retained.
>>
>> Feel free to go ahead and open a change request in JIRA for this issue.
>>
>> Oleg
>>
>
> Probably a better alternative would be something like that
>
> ---------
> /**
> * This interface represents an extended authentication scheme
> * that requires access to {@link HttpContext} in order to
> * generate an authorization string.
> *
> * @since 4.1
> */
>
> public interface ContextAwareAuthScheme extends AuthScheme {
>
> /**
> * Produces an authorization string for the given set of
> * {@link Credentials}.
> *
> * @param credentials The set of credentials to be used for
> athentication
> * @param request The request being authenticated
> * @param context HTTP context
> * @throws AuthenticationException if authorization string cannot
> * be generated due to an authentication failure
> *
> * @return the authorization string
> */
> Header authenticate(
> Credentials credentials,
> HttpRequest request,
> HttpContext context) throws AuthenticationException;
>
> }
>
> ---------
>
> Oleg

Hi Oleg,

The ContextAwareAuthScheme looks perfect for the job.

Should I open a JIRA issue for it?

How will binary compatibility be maintained? An instanceof check at the
location where AuthScheme is used at the moment?

Regards,
Sebastiaan

> ---------------------------------------------------------------------
> To unsubscribe, e-mail: httpclient-users-unsubscribe@...
> For additional commands, e-mail: httpclient-users-help@...
>

smime.p7s (4K) Download Attachment

From forum: HttpClient-User

Re: Kerberos proxy authentication issue

2009-12-11T14:52:28Z

Oleg Kalnichevski wrote:

> Sebastiaan van Erk wrote:
>> Oleg Kalnichevski wrote:
>>> Sebastiaan van Erk wrote:
>>>> Hi Oleg,
>>>>
>>>> Thanks for your reply.
>>>>
>>>> There's a good chance I'm going to have to get this working, even if it
>>>> means I'm going to have to delve into this myself. I'll contact the
>>>> original developer and see if he sees anything obvious, but in any
>>>> case,
>>>> if I succeed in getting it working, I will happily contribute the
>>>> patches.
>>>>
>>>> Best regards,
>>>> Sebastiaan
>>>>
>>>
>>> Hi Sebastiaan
>>>
>>> Cool. In my turn I will happily help with HttpClient specific stuff.
>>
>> Hi Oleg,
>>
>> I got the proxy authentication working already. I even tested it when
>> both proxy auth and http auth are required and it works.
>>
>> There is just one small issue: I need the proxy host name to generate
>> the kerberos service name (HTTP/proxyhost@REALM). The current
>> NegotiateScheme code doesn't handle proxy auth at all and always uses
>> the target host name which it retrieves from the Host header (is that
>> the best way to get the target host?):
>>
>> if (isStripPort()) {
>>
>> init((request.getLastHeader("Host")).getValue().replaceAll(":[0-9]+$",
>> "") );
>> } else {
>> init( (request.getLastHeader("Host")).getValue());
>> }
>>
>> I changed the NegotiateScheme to extend AuthSchemeBase (like
>> NTLMScheme). Now I have access to the isProxy() method and can add the
>> correct header (Proxy-Authorization or Authorization). However, to
>> determine the hostname I currently have this:
>>
>> String host;
>> if (isProxy()) {
>> // FIXME this should actually be determined by the route planner?
>> HttpHost proxy =
>> ConnRouteParams.getDefaultProxy(request.getParams());
>> host = proxy.getHostName();
>> } else {
>> host = request.getLastHeader("Host").getValue();
>> }
>> if (isStripPort()) {
>> host = host.replaceAll(":[0-9]+$", "");
>> }
>> init(host);
>>
>> I noticed a few frames up in DefaultRequestDirector.handleResponse the
>> actual proxy host is known:
>>
>> if (this.proxyAuthHandler.isAuthenticationRequested(response,
>> context)) {
>>
>> HttpHost proxy = route.getProxyHost();
>>
>> this.log.debug("Proxy requested authentication");
>> Map<String, Header> challenges =
>> this.proxyAuthHandler.getChallenges(response, context);
>> try {
>> processChallenges(challenges,
>> this.proxyAuthState, this.proxyAuthHandler,
>> response, context);
>> } catch (AuthenticationException ex) {
>> ...
>>
>> But this information is not available to me at the FIXME location. I'm
>> also ignoring any forced route now, but it seems wrong to copy paste
>> the code from DefaultHttpRoutePlanner anyway, especially since that's
>> just the default implementation anyhow and could be overridden.
>>
>> Next thing I'll look into is why the redirect fails.
>>
>> Regards,
>> Sebastiaan
>>
>
> Hi Sebastiaan
>
> This looks nasty. The usual way of obtaining contextual details in
> HttpClient is by examining attributes of the HttpContext instance
> associated with the request being executed. The trouble is that auth
> schemes factories presently have no means of getting hold of the
> HttpContext.
>
> One possibility of fixing it would be extending or replacing the
> AuthSchemeFactory with something better.
>
> ---
> public interface BetterAuthSchemeFactory extends AuthSchemeFactory {
>
> AuthScheme newInstance(HttpContext context, HttpParams params);
>
> }
> ---
>
> This is double but is far from trivial if 100% binary compatibility with
> 4.0 is to be retained.
>
> Feel free to go ahead and open a change request in JIRA for this issue.
>
> Oleg
>

Probably a better alternative would be something like that

---------
/**
* This interface represents an extended authentication scheme
* that requires access to {@link HttpContext} in order to
* generate an authorization string.
*
* @since 4.1
*/

public interface ContextAwareAuthScheme extends AuthScheme {

/**
* Produces an authorization string for the given set of
* {@link Credentials}.
*
* @param credentials The set of credentials to be used for
athentication
* @param request The request being authenticated
* @param context HTTP context
* @throws AuthenticationException if authorization string cannot
* be generated due to an authentication failure
*
* @return the authorization string
*/
Header authenticate(
Credentials credentials,
HttpRequest request,
HttpContext context) throws AuthenticationException;

}

---------

Oleg

---------------------------------------------------------------------
To unsubscribe, e-mail: httpclient-users-unsubscribe@...
For additional commands, e-mail: httpclient-users-help@...

From forum: HttpClient-User