[jira] Created: (HTTPCLIENT-834) Transparent Content Coding support

Transparent Content Coding support
----------------------------------

                 Key: HTTPCLIENT-834
                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
             Project: HttpComponents HttpClient
          Issue Type: New Feature
          Components: HttpClient
    Affects Versions: 4.0 Beta 3
         Environment: Any
            Reporter: James Abley


I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.

This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682176#action_12682176 ]

James Abley commented on HTTPCLIENT-834:
----------------------------------------

I've implemented gzip and deflate support already - patch to follow once I get my git foo sorted.

compress I'm not so familiar with, and identity is a no-op so requires no changes.

The patch has been implemented so that it won't interfere with existing clients if they are already setting an Accept-Encoding header in the request.

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Abley updated HTTPCLIENT-834:
-----------------------------------

    Attachment: 834.patch

Initial patch which introduces gzip and deflate support.

Sorry, mangled git patch. If any of the committers use git, then they should be able to apply it. Otherwise, I'll need to spend some more time understanding how to generate a patch suitable for application with the patch command.

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682318#action_12682318 ]

Oleg Kalnichevski commented on HTTPCLIENT-834:
----------------------------------------------

Hi James

(1) ThreadLocals are often considered evil and ought to be avoided, especially in general purpose libraries. If you need to persist some data between different interceptors please consider using HttpContext. This is precisely what it is meant for.

(2) Please consider providing some means of defining applicable content coding algorithms and their order of preference.

(3) Patch utility compatible diff would be nice.

Oleg

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682434#action_12682434 ]

James Abley commented on HTTPCLIENT-834:
----------------------------------------

Hi Oleg,

Thanks for the feedback.

1) Thanks, I wasn't aware of the idiomatic way to do that, so I'll change it. Can you share why you feel ThreadLocals are so bad? I wasn't aware of any issues and would like to know more.

2) I don't follow. Can you provide more information, or maybe describe it as a test?

3) Will do. I just wanted to get it out there for discussion.

Cheers,

James

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682460#action_12682460 ]

Oleg Kalnichevski commented on HTTPCLIENT-834:
----------------------------------------------

(1) Because ThreadLocals, when misused/abused, can lead to memory leaks or, worse, very subtle bugs, especially in managed environments such as servlet or EJB containers, where threads are frequently pooled / re-used. The most catastrophic bug I have seen involved a ThreadLocal holding a UserPrincipal in a web application, which was not correctly unset with a try-catch-finally. You can pretty much guess the consequence of that bug.

The only two legitimate uses of ThreadLocal I know of is (1) in single-user standalone applications or (2) in web applications when ThreadLocals set and unset using a HttpServletFilter in try-catch-finally. ThreadLocal simply cannot be reliably unset from inside a library. ThreadLocals in general purpose libraries == major trouble.

(2) There should be a way to instruct the interceptor to generate different Accept-Encoding values, for example 'gzip,deflate', "deflate,gzip", "identity", etc, based a configuration parameter.

Oleg

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682434#action_12682434 ]

James Abley edited comment on HTTPCLIENT-834 at 3/16/09 3:12 PM:
-----------------------------------------------------------------

Hi Oleg,

Thanks for the feedback.

1) Thanks, I wasn't aware of the idiomatic way to do that, so I'll change it. Can you share why you feel ThreadLocals are so bad? I wasn't aware of any issues and would like to know more.

2) I don't follow. Can you provide more information, or maybe describe it as a test?

3) Will do. I just wanted to get it out there for discussion. UPDATE: Actually, patch -p1 <path/to/patch works for me.

Cheers,

James

      was (Author: jabley):
    Hi Oleg,

Thanks for the feedback.

1) Thanks, I wasn't aware of the idiomatic way to do that, so I'll change it. Can you share why you feel ThreadLocals are so bad? I wasn't aware of any issues and would like to know more.

2) I don't follow. Can you provide more information, or maybe describe it as a test?

3) Will do. I just wanted to get it out there for discussion.

Cheers,

James
 

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682495#action_12682495 ]

Sebb commented on HTTPCLIENT-834:
---------------------------------

Two discussions of ThreadLocal that might be useful:

http://blog.arendsen.net/index.php/2005/02/16/threadlocals-are-evil/

http://crazybob.org/2006/07/hard-core-java-threadlocal.html

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682497#action_12682497 ]

James Abley commented on HTTPCLIENT-834:
----------------------------------------

1) OK, thanks for the clarification. That pretty much ties with my understanding, but I thought it was a reasonable approach at the time and seemed in line with recommendations from Java Concurrency in Practice. I've removed the usage as you suggested.

2) I initially thought you wanted to specify quality values. But you're describing something similar. I'm still not clear why that requires configuration. If the user doesn't want the interceptor to indicate to the server that various content codings are supported, the user can specify their own Accept-Encoding header for the request and the request won't be altered and the response won't be processed by the new interceptors. Indeed, that behaviour is there to ensure that it doesn't break any existing clients that may be using an interceptor already to support this functionality. Although thinking about that, the new interceptor is getting added before the client has a chance to add an interceptor, so the new ones that I've written will fire before a user-provided one gets a chance to handle the request response. Is that the issue? What else am I missing? Can you point me at an example of something similar that already exists, for configuration parameters?

As as aside, is there any reason why the Javadocs aren't built as part of the site hosted at apache?

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682639#action_12682639 ]

Oleg Kalnichevski commented on HTTPCLIENT-834:
----------------------------------------------

I was thinking about something along this line:

{{{
enum ContentCoding { identity, gzip, deflate };
}}}    

{{{
@SuppressWarnings("unchecked")
Collection<ContentCoding> codings = (Collection<ContentCoding>) request.getParams().getParameter(
                "http.protocol.accept-encoding");
if (codings != null) {
     // Add Accept-Encoding header
}
}}}

>> Although thinking about that, the new interceptor is getting added before the client has a chance to add an interceptor, so the new ones that I've written
>> will fire before a user-provided one gets a chance to handle the request response.Is that the issue?

I am afraid it is. The content processor in its current form will break existing code that relies on protocol interceptors to transparently handle content encoding. This is the reason I would like to have an explicit configuration parameter the user would have to set in order to enable content processing.

>> Can you point me at an example of something similar that already exists, for configuration parameters?

Take a look at the default header interceptor, which populates HTTP request headers dynamically based on a configuration parameter

http://hc.apache.org/httpcomponents-client/httpclient/xref/org/apache/http/client/protocol/RequestDefaultHeaders.html

Oleg

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682656#action_12682656 ]

James Abley commented on HTTPCLIENT-834:
----------------------------------------

Cool.

I am still aiming for a transparent implementation that won't do anything if the user has configured their own behaviour, and will do the right thing (be in effect) if they haven't. I'm hopeful that we're not yet at the stage where this will require explicitly enabling to work.

I'll create a failing test to demonstrate how it will impact existing protocol interceptors, then fix it and see what falls out of that.

Thanks for your help; it's always good to collaborate with new people and get new insights.

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Abley updated HTTPCLIENT-834:
-----------------------------------

    Attachment: 834-svn-754998.patch
                834-2009-03-17.patch

Git based patch and same again for SVN.
SVN patch sets svn:eol-style=native.

This patch is re-worked to avoid breaking existing clients that may already be doing something like this.

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch, 834.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

James Abley updated HTTPCLIENT-834:
-----------------------------------

    Attachment:     (was: 834.patch)

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12682176#action_12682176 ]

James Abley edited comment on HTTPCLIENT-834 at 3/17/09 2:55 PM:
-----------------------------------------------------------------

I've implemented gzip and deflate support already - patch to follow once I get my git fu sorted.

compress I'm not so familiar with, and identity is a no-op so requires no changes.

The patch has been implemented so that it won't interfere with existing clients if they are already setting an Accept-Encoding header in the request.

      was (Author: jabley):
    I've implemented gzip and deflate support already - patch to follow once I get my git foo sorted.

compress I'm not so familiar with, and identity is a no-op so requires no changes.

The patch has been implemented so that it won't interfere with existing clients if they are already setting an Accept-Encoding header in the request.
 

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683066#action_12683066 ]

Oleg Kalnichevski commented on HTTPCLIENT-834:
----------------------------------------------

I reviewed the patch and checked in the patch to the 4.1 branch:

http://svn.apache.org/repos/asf/httpcomponents/httpclient/branches/branch_4_1/
http://svn.apache.org/viewvc?rev=755629&view=rev

Many thanks for this contribution, James.

I am also going to add your name to the list of project contributors, if do not mind.

The only problem with the current approach is that there is no way to disable or customize the way content coding is handled per default. One can remove any standard protocol interceptor and replace it with a custom one. This obviously will not work for ContentProcessor, which is not nice. An alternative solution might be to simply use a different implementation class derived from DefaultHttpClient.

I am looking forward to follow-up patches ;-)

Would you also be willing to look into content caching at some point of time?

Oleg

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683150#action_12683150 ]

James Abley commented on HTTPCLIENT-834:
----------------------------------------

W00t! By all means list me as a contributor. I wasn't expecting it to be accepted yet; I thought it was still a little rough. I hadn't found a coding style guide, so I was expecting some feedback there, apache license header, maybe a contributors agreement to submit? Not that I'm complaining. Thank you.

If the demand is there to allow it to be configured / removed, then we can add that.

I would like to look at content caching. I would think that's a bit more involved and I'm not sure that I can commit the time that I think it would require at the moment, but it's definitely something that I'd like to see in the library. Or I might see what else is in JIRA that looks fun, that would take the amount of time that I can commit to at the moment.

I'll also have to give it some thought as to how that might be implemented. I've got a little too much baggage in terms of how I've implemented something like that using httpclient-3.x and ehcache, so I need to step back and think about it using the 4.x API and (pluggable) cache implementations.

James

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12683196#action_12683196 ]

Oleg Kalnichevski commented on HTTPCLIENT-834:
----------------------------------------------

> I wasn't expecting it to be accepted yet; I thought it was still a little rough

The work is far from being done. I just prefer smaller, incremental patches.

> maybe a contributors agreement to submit

Yes, you should consider submitting an indivudual CLA, maybe even a corporate CLA, if you are working on HttpClient during office hours

> If the demand is there to allow it to be configured / removed, then we can add that

It surely must be possible to disable content encoding. There are enough situations when it is not desired, for instance when trouble-shooting.  I am still not convinced it should be enabled per default.

Oleg

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

     [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Oleg Kalnichevski updated HTTPCLIENT-834:
-----------------------------------------

    Fix Version/s: 4.1.0

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 3
>         Environment: Any
>            Reporter: James Abley
>             Fix For: 4.1.0
>
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737570#action_12737570 ]

James Abley commented on HTTPCLIENT-834:
----------------------------------------

Sorry, didn't mean to go dark for so long. Moving house and Real Work got in the way. I'd like to bring this to a conclusion.

Where in SVN should I target now, and is there IRC or something?

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Any
>            Reporter: James Abley
>             Fix For: 4.1.0
>
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

    [ https://issues.apache.org/jira/browse/HTTPCLIENT-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12737575#action_12737575 ]

Oleg Kalnichevski commented on HTTPCLIENT-834:
----------------------------------------------

Welcome back, James.

Not much has changed since I committed your patch. The code is still in the 4.1 branch (to be merged down to trunk after 4.0 GA release) [1]. I am still of the same opinion that DefaultHttpClient ought not have the transparent content coding enabled per default (there are different options of solving the problem, HTTPCLIENT-838 being one). You should still consider submitting an an individual or a corporate CLA.

All discussions about HttpCore / HttpClient development should take place on the dev mailing list.

Oleg

[1] http://svn.apache.org/repos/asf/httpcomponents/httpclient/branches/branch_4_1/ 

> Transparent Content Coding support
> ----------------------------------
>
>                 Key: HTTPCLIENT-834
>                 URL: https://issues.apache.org/jira/browse/HTTPCLIENT-834
>             Project: HttpComponents HttpClient
>          Issue Type: New Feature
>          Components: HttpClient
>    Affects Versions: 4.0 Beta 2
>         Environment: Any
>            Reporter: James Abley
>             Fix For: 4.1.0
>
>         Attachments: 834-2009-03-17.patch, 834-svn-754998.patch
>
>
> I would like to see HttpClient features brought up to parity with other libraries, both in Java and other languages. c.f. Python's httplib2 (not yet in the standard library, but many would like to see it in there). That library transparently handles gzip and compress content codings.
> This issue is to capture possible solutions to providing this sort of innate functionality in HttpClient, so that users aren't required to know RFC2616 intimately. The HttpClient library should do the right thing and use the network in the most efficient manner possible.

--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...