HttpClient with ssl
|
View:
New views
12 Messages
—
Rating Filter:
Alert me
|
 |
HttpClient with ssl
by Douglas Ferguson-2
::
Rate this Message:
I have been unsuccessful in finding information on how to setup
HttpClient to retrieve random https:// pages. Does anybody know how to do this? Douglas --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by Tobias N. Sasse
::
Rate this Message:
Douglas Ferguson schrieb:
> I have been unsuccessful in finding information on how to setup > HttpClient to retrieve random https:// pages. > > Does anybody know how to do this You should look at the following pages: http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 HTH, Tobi --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by Douglas Ferguson-2
::
Rate this Message:
I did read that page, but I ran into a couple of issues.
1) This is deprecated: Protocol.registerProtocol("myhttps", new Protocol("https", new MySSLSocketFactory(), 9443)); 2) I wasn't sure which factory I should be using. 3) Is the contribution package avaible through a maven repository? I'm using commons version HttpClient. On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote: > Douglas Ferguson schrieb: >> I have been unsuccessful in finding information on how to setup >> HttpClient to retrieve random https:// pages. >> >> Does anybody know how to do this > > You should look at the following pages: > > http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) > http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 > > HTH, > Tobi > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@... > For additional commands, e-mail: httpclient-users-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by olegk
::
Rate this Message:
Douglas Ferguson wrote:
> I did read that page, but I ran into a couple of issues. > > 1) This is deprecated: Protocol.registerProtocol("myhttps", > new Protocol("https", new MySSLSocketFactory(), 9443)); > Cast your custom socket factory to ProtocolSocketFactory. Though, make sure it implements SecureProtocolSocketFactory. new Protocol("https", (ProtocolSocketFactory) new MySSLSocketFactory(), 9443) > 2) I wasn't sure which factory I should be using. > SecureProtocolSocketFactory > 3) Is the contribution package avaible through a maven repository? > No, it is not. Contrib code is distributed as source only > I'm using commons version HttpClient. > Upgrade to HttpClient 4.0. The 3.x branch is nearing end of life. Oleg > > On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote: > >> Douglas Ferguson schrieb: >>> I have been unsuccessful in finding information on how to setup >>> HttpClient to retrieve random https:// pages. >>> >>> Does anybody know how to do this >> You should look at the following pages: >> >> http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) >> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 >> >> HTH, >> Tobi >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >> For additional commands, e-mail: httpclient-users-help@... >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@... > For additional commands, e-mail: httpclient-users-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by Ken Krugler
::
Rate this Message:
Hi Douglas,
I'm assuming "commons version HttpClient" means version 3.1. I haven't directly set up https with 3.1, but I've used the Nutch protocol-httpclient plugin, which does. I'd suggest looking at the code in the Http class, e.g. configureClient() which does: // Set up an HTTPS socket factory that accepts self-signed certs. Protocol https = new Protocol("https", new DummySSLProtocolSocketFactory(), 443); Protocol.registerProtocol("https", https); where DummySSLProtocolSocketFactory is another class in the same package. All of this code is in the Nutch project, at src/plugin/protocol-httpclient/src/java/org/apache/nutch/protocol/ httpclient -- Ken On Sep 27, 2009, at 11:25am, Douglas Ferguson wrote: > I did read that page, but I ran into a couple of issues. > > 1) This is deprecated: Protocol.registerProtocol("myhttps", > new Protocol("https", new MySSLSocketFactory(), 9443)); > > 2) I wasn't sure which factory I should be using. > > 3) Is the contribution package avaible through a maven repository? > > I'm using commons version HttpClient. > > > On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote: > >> Douglas Ferguson schrieb: >>> I have been unsuccessful in finding information on how to setup >>> HttpClient to retrieve random https:// pages. >>> >>> Does anybody know how to do this >> >> You should look at the following pages: >> >> http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) >> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 >> >> HTH, >> Tobi --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by Douglas Ferguson-2
::
Rate this Message:
> > Upgrade to HttpClient 4.0. The 3.x branch is nearing end of life. > Is HttpClient 4.0 available from a maven repository? > Oleg > >> >> On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote: >> >>> Douglas Ferguson schrieb: >>>> I have been unsuccessful in finding information on how to setup >>>> HttpClient to retrieve random https:// pages. >>>> >>>> Does anybody know how to do this >>> You should look at the following pages: >>> >>> http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) >>> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 >>> >>> HTH, >>> Tobi >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>> For additional commands, e-mail: httpclient-users-help@... >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >> For additional commands, e-mail: httpclient-users-help@... >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@... > For additional commands, e-mail: httpclient-users-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by Douglas Ferguson-2
::
Rate this Message:
This is an interface. Is there an impl available that will allow
retrieval of any https page? I found some examples but they all involved installing certs for the pages you want to allow. I need to be able to download any random page from the web. Douglas On Sep 27, 2009, at 2:02 PM, Oleg Kalnichevski wrote: >> 2) I wasn't sure which factory I should be using. >> > > SecureProtocolSocketFactory --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by Douglas Ferguson-2
::
Rate this Message:
I believe so.
I have this in my pom.. > <dependency> > <groupId>commons-httpclient</groupId> > <artifactId>commons-httpclient</artifactId> > <version>3.1</version> > </dependency> And my code looks like this: Note: I override getResponseCharSeta and plugin the Mozilla charset detector. Also, I currently replace https:// with http:// because https is throwing an exception. Which is what I'm not trying to solve. > public static GetMethod newGetMethod(String url2, int timeOut) > throws IOException, HttpException, Exception { > GetMethod getMethod; > HttpClient httpClient = new HttpClient(); > > > String url = url2.replaceFirst("(?i)https://", "http://"); > System.out.println(url); > > httpClient.getHttpConnectionManager().getParams > ().setConnectionTimeout(timeOut); > httpClient.getHttpConnectionManager().getParams().setSoTimeout > (timeOut); > > getMethod = new GetMethod(url) { > > String charset = null; > > @Override > public String getResponseCharSet() { > if (charset == null) { > charset = super.getResponseCharSet(); > if (charset == null || CHARSET_NOT_FOUND.equals(charset)) { > try { > charset = detectCharSet(getResponseBodyAsStream()); > } catch (Exception e) { > charset = UTF_8; > } > } > } > return charset; > } > }; > getMethod.getParams().setUriCharset(UTF_8); > getMethod.getParams().setContentCharset(CHARSET_NOT_FOUND); > > getMethod.setRequestHeader("User-Agent", "Mozilla/3.0 (Win95; I)"); > getMethod.addRequestHeader("Accept-Charset", UTF_8); > > getMethod.setFollowRedirects(true); > > int statusCode = httpClient.executeMethod(getMethod); > if (statusCode != HttpStatus.SC_OK) { > throw new Exception("GetMethod failed: " + statusCode + ": " + > getMethod.getStatusLine()); > } > > return getMethod; > } On Sep 27, 2009, at 2:12 PM, Ken Krugler wrote: > Hi Douglas, > > I'm assuming "commons version HttpClient" means version 3.1. > > I haven't directly set up https with 3.1, but I've used the Nutch > protocol-httpclient plugin, which does. > > I'd suggest looking at the code in the Http class, e.g. > configureClient() which does: > > // Set up an HTTPS socket factory that accepts self-signed certs. > Protocol https = new Protocol("https", > new DummySSLProtocolSocketFactory(), 443); > Protocol.registerProtocol("https", https); > > where DummySSLProtocolSocketFactory is another class in the same > package. > > All of this code is in the Nutch project, at > > src/plugin/protocol-httpclient/src/java/org/apache/nutch/protocol/ > httpclient > > -- Ken > > On Sep 27, 2009, at 11:25am, Douglas Ferguson wrote: > >> I did read that page, but I ran into a couple of issues. >> >> 1) This is deprecated: Protocol.registerProtocol("myhttps", >> new Protocol("https", new MySSLSocketFactory(), 9443)); >> >> 2) I wasn't sure which factory I should be using. >> >> 3) Is the contribution package avaible through a maven repository? >> >> I'm using commons version HttpClient. >> >> >> On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote: >> >>> Douglas Ferguson schrieb: >>>> I have been unsuccessful in finding information on how to setup >>>> HttpClient to retrieve random https:// pages. >>>> >>>> Does anybody know how to do this >>> >>> You should look at the following pages: >>> >>> http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) >>> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 >>> >>> HTH, >>> Tobi > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@... > For additional commands, e-mail: httpclient-users-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by olegk
::
Rate this Message:
Douglas Ferguson wrote:
>> Upgrade to HttpClient 4.0. The 3.x branch is nearing end of life. >> > > Is HttpClient 4.0 available from a maven repository? > > Yes, it is http://hc.apache.org/httpcomponents-client/download.html Oleg >> Oleg >> >>> On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote: >>> >>>> Douglas Ferguson schrieb: >>>>> I have been unsuccessful in finding information on how to setup >>>>> HttpClient to retrieve random https:// pages. >>>>> >>>>> Does anybody know how to do this >>>> You should look at the following pages: >>>> >>>> http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) >>>> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 >>>> >>>> HTH, >>>> Tobi >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>>> For additional commands, e-mail: httpclient-users-help@... >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>> For additional commands, e-mail: httpclient-users-help@... >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >> For additional commands, e-mail: httpclient-users-help@... >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@... > For additional commands, e-mail: httpclient-users-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by olegk
::
Rate this Message:
Douglas Ferguson wrote:
> This is an interface. Is there an impl available that will allow > retrieval of any https page? > > I found some examples but they all involved installing certs for the > pages you want to allow. > I need to be able to download any random page from the web. > > Douglas > You'll have to create a custom impl that simply trusts any certificate without verification. See these sections of the HttpClient tutorial: http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e497 Generally, this kind of practice is strongly discouraged for security reasons. Oleg > On Sep 27, 2009, at 2:02 PM, Oleg Kalnichevski wrote: > >>> 2) I wasn't sure which factory I should be using. >>> >> SecureProtocolSocketFactory > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@... > For additional commands, e-mail: httpclient-users-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by Douglas Ferguson-2
::
Rate this Message:
Ah.. Yes. Thanks..
Looks like the api changed drastically. Is there an upgrade guide from 3.1 to 4.0? I'd mostly be worried about how I would go about doing this again: getMethod = new GetMethod(url) { String charset = null; @Override public String getResponseCharSet() { if (charset == null) { charset = super.getResponseCharSet(); if (charset == null || CHARSET_NOT_FOUND.equals(charset)) { try { charset = detectCharSet(getResponseBodyAsStream()); } catch (Exception e) { charset = UTF_8; } } } return charset; } }; On Sep 27, 2009, at 4:05 PM, Oleg Kalnichevski wrote: > Douglas Ferguson wrote: >>> Upgrade to HttpClient 4.0. The 3.x branch is nearing end of life. >>> >> >> Is HttpClient 4.0 available from a maven repository? >> >> > > Yes, it is > > http://hc.apache.org/httpcomponents-client/download.html > > Oleg > > >>> Oleg >>> >>>> On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote: >>>> >>>>> Douglas Ferguson schrieb: >>>>>> I have been unsuccessful in finding information on how to setup >>>>>> HttpClient to retrieve random https:// pages. >>>>>> >>>>>> Does anybody know how to do this >>>>> You should look at the following pages: >>>>> >>>>> http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) >>>>> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 >>>>> >>>>> HTH, >>>>> Tobi >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>>>> For additional commands, e-mail: httpclient-users-help@... >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>>> For additional commands, e-mail: httpclient-users- >>>> help@... >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>> For additional commands, e-mail: httpclient-users-help@... >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >> For additional commands, e-mail: httpclient-users-help@... >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@... > For additional commands, e-mail: httpclient-users-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
|
|
Re: HttpClient with ssl
by Douglas Ferguson-2
::
Rate this Message:
Ok.. Something is odd.. I was getting an exception on https pages
before, but now I'm not. Perhaps the page I happened to be testing on before had a cert problem. D/ On Sep 27, 2009, at 4:05 PM, Oleg Kalnichevski wrote: > Douglas Ferguson wrote: >>> Upgrade to HttpClient 4.0. The 3.x branch is nearing end of life. >>> >> >> Is HttpClient 4.0 available from a maven repository? >> >> > > Yes, it is > > http://hc.apache.org/httpcomponents-client/download.html > > Oleg > > >>> Oleg >>> >>>> On Sep 27, 2009, at 1:18 PM, Tobias N. Sasse wrote: >>>> >>>>> Douglas Ferguson schrieb: >>>>>> I have been unsuccessful in finding information on how to setup >>>>>> HttpClient to retrieve random https:// pages. >>>>>> >>>>>> Does anybody know how to do this >>>>> You should look at the following pages: >>>>> >>>>> http://hc.apache.org/httpclient-3.x/sslguide.html (version 3) >>>>> http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e484 >>>>> >>>>> HTH, >>>>> Tobi >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>>>> For additional commands, e-mail: httpclient-users-help@... >>>>> >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>>> For additional commands, e-mail: httpclient-users- >>>> help@... >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >>> For additional commands, e-mail: httpclient-users-help@... >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: httpclient-users-unsubscribe@... >> For additional commands, e-mail: httpclient-users-help@... >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: httpclient-users-unsubscribe@... > For additional commands, e-mail: httpclient-users-help@... > --------------------------------------------------------------------- To unsubscribe, e-mail: httpclient-users-unsubscribe@... For additional commands, e-mail: httpclient-users-help@... |
| Free embeddable forum powered by Nabble | Forum Help |
