Changing proxy settings dynamically

[Moritz Post-2]

Hello jets3t list

I am currently using the jetse3t library to access the s3 infrastructure
(obviously ;) ). Now some of my clients are located behind a proxy
server. Of course you can use the httpclient directives to configure
host and port of the proxy (which works) but problems arise when
changing these settings during runtime.

We are not using property file based config settings but supply all the
configuration directives during runtime to the Jets3tProperties (using a
dummy filename to obtain an instance). When looking into the code this
properties file is backed up by a java.util.Properties class which is
immutable by nature. It is though backed up by a HashMap which is of
course editable.

The problem we are facing is that once we provide proxy settings via
Jets3tProperties.setProperty(key,value), we are not able to remove them
(aka tell jets3t not to use a proxy any more). This is because of the
immutable nature of the Jets3tProperties and the underlying
java.util.Properties.

Therefore i would be very interessted to hear of other possible
solutions for this problem or to get a release of the Jets3t lib where
such limitations are overcome

regards
Moritz Post

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[James Murty-3]

Hi Moritz,

There are two distinct issues you will come up against if your client's proxy settings can change dynamically.

The first is the design of the JetS3tProperties object, which is intended to store static configuration values. To gain more control over how the properties are read and updated, I would recommend using the latest version of JetS3t which allows you to provide a unique properties object to each S3Service you create. If you create your own subclass of JetS3tProperties and override all its methods, you can make it behave the way you want, perhaps looking up and returning the current proxy settings on demand.

A pre-release package of the latest code is available here: http://jets3t.s3.amazonaws.com/jets3t-0.6.1-alpha_3.zip

The second issue is that changing proxy settings in the JetS3tProperties object will not actually affect the rest of the library. When you create an S3Service object, the proxy settings are read once and provided to the underlying HttpClient library. To force future HTTP connections to use a different proxy, you will need to recreate the relevant S3Service object. The best way to do this would be to have your code detect proxy-related errors thrown by JetS3t, and automatically recreate the S3Service with the latest proxy settings when this happens.

As an alternative to modifying JetS3t, you may want to look into the HttpClient library that JetS3t relies on to see if you can hook-in some code to detect and update proxy settings more directly.

I hope this helps,
James

---
http://www.jamesmurty.com

On Sat, Aug 2, 2008 at 1:13 AM, Moritz Post <mpost@...> wrote:

Hello jets3t list

I am currently using the jetse3t library to access the s3 infrastructure (obviously ;) ). Now some of my clients are located behind a proxy server. Of course you can use the httpclient directives to configure host and port of the proxy (which works) but problems arise when changing these settings during runtime.

We are not using property file based config settings but supply all the configuration directives during runtime to the Jets3tProperties (using a dummy filename to obtain an instance). When looking into the code this properties file is backed up by a java.util.Properties class which is immutable by nature. It is though backed up by a HashMap which is of course editable.

The problem we are facing is that once we provide proxy settings via Jets3tProperties.setProperty(key,value), we are not able to remove them (aka tell jets3t not to use a proxy any more). This is because of the immutable nature of the Jets3tProperties and the underlying java.util.Properties.

Therefore i would be very interessted to hear of other possible solutions for this problem or to get a release of the Jets3t lib where such limitations are overcome

regards
Moritz Post

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Moritz Post-2]

Hi James

Thanks for answering so promptly.

There are two distinct issues you will come up against if your client's proxy settings can change dynamically.

The first is the design of the JetS3tProperties object, which is intended to store static configuration values. To gain more control over how the properties are read and updated, I would recommend using the latest version of JetS3t which allows you to provide a unique properties object to each S3Service you create. If you create your own subclass of JetS3tProperties and override all its methods, you can make it behave the way you want, perhaps looking up and returning the current proxy settings on demand.

That sounded like a viable solution. But the problem here is that the RestS3Service uses the JetS3tProperties class as a singleton so that I am not able to inject my extended properties class.

A pre-release package of the latest code is available here: http://jets3t.s3.amazonaws.com/jets3t-0.6.1-alpha_3.zip

The second issue is that changing proxy settings in the JetS3tProperties object will not actually affect the rest of the library. When you create an S3Service object, the proxy settings are read once and provided to the underlying HttpClient library. To force future HTTP connections to use a different proxy, you will need to recreate the relevant S3Service object. The best way to do this would be to have your code detect proxy-related errors thrown by JetS3t, and automatically recreate the S3Service with the latest proxy settings when this happens.

We are currently recreating all the required S3Service objects when the proxy settings change so this is not really a problem for us.

As an alternative to modifying JetS3t, you may want to look into the HttpClient library that JetS3t relies on to see if you can hook-in some code to detect and update proxy settings more directly.

I have checked this alternative but as it seams there is no was to interact with the HttpClient used within the RestS3Service.

So would it be possible to either open up the singleton behaviour of the JetS3t properties or make the JetS3tProperties editable by default (which would be the preferred way).

Anything you could do to make our use case possible would be appreciated :)

Greets Moritz

PS. could you please please please... add a trim() when entering the AWS credentials in the cockpit login form. it is very annoying to have to input the access and secret id in the trimmed form... especially when just copy/pasting them from the AWS website. thank you :)

I hope this helps,
James

---
http://www.jamesmurty.com

On Sat, Aug 2, 2008 at 1:13 AM, Moritz Post <mpost@...> wrote:

Hello jets3t list

I am currently using the jetse3t library to access the s3 infrastructure (obviously ;) ). Now some of my clients are located behind a proxy server. Of course you can use the httpclient directives to configure host and port of the proxy (which works) but problems arise when changing these settings during runtime.

We are not using property file based config settings but supply all the configuration directives during runtime to the Jets3tProperties (using a dummy filename to obtain an instance). When looking into the code this properties file is backed up by a java.util.Properties class which is immutable by nature. It is though backed up by a HashMap which is of course editable.

The problem we are facing is that once we provide proxy settings via Jets3tProperties.setProperty(key,value), we are not able to remove them (aka tell jets3t not to use a proxy any more). This is because of the immutable nature of the Jets3tProperties and the underlying java.util.Properties.

Therefore i would be very interessted to hear of other possible solutions for this problem or to get a release of the Jets3t lib where such limitations are overcome

regards
Moritz Post

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[James Murty-2]

Hi Moritz,

The first is the design of the JetS3tProperties object, which is intended to store static configuration values. To gain more control over how the properties are read and updated, I would recommend using the latest version of JetS3t which allows you to provide a unique properties object to each S3Service you create. If you create your own subclass of JetS3tProperties and override all its methods, you can make it behave the way you want, perhaps looking up and returning the current proxy settings on demand.

That sounded like a viable solution. But the problem here is that the RestS3Service uses the JetS3tProperties class as a singleton so that I am not able to inject my extended properties class.

If you obtain the latest codebase from CVS or the alpha package I mentioned in the prior message, you can provide a JetS3tProperties object as a variable to the S3Service constructors. With JetS3t version 0.6.0 this will not be possible, because as you point out the JetS3tProperties class is loaded as a singleton.

 

We are currently recreating all the required S3Service objects when the proxy settings change so this is not really a problem for us.

That's good. I think that with the latest version of the JetS3t code you should be able to solve your problem. The JetS3tProperties object in the latest version also provides setProperty() and loadAndReplaceProperties() methods that allow you to manually set or override property values.
 

PS. could you please please please... add a trim() when entering the AWS credentials in the cockpit login form. it is very annoying to have to input the access and secret id in the trimmed form... especially when just copy/pasting them from the AWS website. thank you :)

That's a good point. I've just made this change in the latest codebase, so if you can obtain the code from CVS you should be happy ;)

Cheers,
James

 

[Moritz Post-2]

Hi James

The first is the design of the JetS3tProperties object, which is intended to store static configuration values. To gain more control over how the properties are read and updated, I would recommend using the latest version of JetS3t which allows you to provide a unique properties object to each S3Service you create. If you create your own subclass of JetS3tProperties and override all its methods, you can make it behave the way you want, perhaps looking up and returning the current proxy settings on demand.

That sounded like a viable solution. But the problem here is that the RestS3Service uses the JetS3tProperties class as a singleton so that I am not able to inject my extended properties class.

If you obtain the latest codebase from CVS or the alpha package I mentioned in the prior message, you can provide a JetS3tProperties object as a variable to the S3Service constructors. With JetS3t version 0.6.0 this will not be possible, because as you point out the JetS3tProperties class is loaded as a singleton.

That sounds like a desirable approach. I am looking forward to use this feature.

I am currently working on the g-Eclipse project (http://www.geclipse.eu/ and http://www.eclipse.org/geclipse/), which integrated support for EC2, S3 and various other AWS technologies. In order to use any third party library, it has to go through an IP process so we can not update that easily. Once a 0.6.1 release has been made we will send that through the IP process so that we can use the new features.

Is there any schedule on when to expect the new release?

 

We are currently recreating all the required S3Service objects when the proxy settings change so this is not really a problem for us.

That's good. I think that with the latest version of the JetS3t code you should be able to solve your problem. The JetS3tProperties object in the latest version also provides setProperty() and loadAndReplaceProperties() methods that allow you to manually set or override property values.

I have tried to change my properties by storing the properties to a byte array and writing it back into the the Jets3tProperties via loadAndReplaceProperties() but currently it is only overwriting existing properties or adding new ones... items which do not exist in the new properties are not removed from old properties so this doesn't help. Also setProperty() doesn't help since you can not set a value to null.

Btw, this bug tracks the issue in the eclipse bugzilla: https://bugs.eclipse.org/bugs/show_bug.cgi?id=242798

 

PS. could you please please please... add a trim() when entering the AWS credentials in the cockpit login form. it is very annoying to have to input the access and secret id in the trimmed form... especially when just copy/pasting them from the AWS website. thank you :)

That's a good point. I've just made this change in the latest codebase, so if you can obtain the code from CVS you should be happy ;)

Great. :)

Greets
Moritz Post

[James Murty-3]

I am currently working on the g-Eclipse project (http://www.geclipse.eu/ and http://www.eclipse.org/geclipse/), which integrated support for EC2, S3 and various other AWS technologies. In order to use any third party library, it has to go through an IP process so we can not update that easily. Once a 0.6.1 release has been made we will send that through the IP process so that we can use the new features.

Is there any schedule on when to expect the new release?

I'm hopeful that I will be able to officially release the new version (0.6.1) this coming weekend. The main work remaining is around documentation, and it's almost ready. Fingers crossed ;)
 

I have tried to change my properties by storing the properties to a byte array and writing it back into the the Jets3tProperties via loadAndReplaceProperties() but currently it is only overwriting existing properties or adding new ones... items which do not exist in the new properties are not removed from old properties so this doesn't help. Also setProperty() doesn't help since you can not set a value to null.

Btw, this bug tracks the issue in the eclipse bugzilla: https://bugs.eclipse.org/bugs/show_bug.cgi?id=242798

Thanks for this pointer to the bug tracking on your side. I have added two new methods to JetS3tProperties that will make it possible to remove a specific property, or all the properties encapsulated by the class. I hope this change, in combination with the others that will be included with version 0.6.1, will resolve the issue.

Cheers,
James

[Moritz Post-2]

Hi James

Is there any schedule on when to expect the new release?

I'm hopeful that I will be able to officially release the new version (0.6.1) this coming weekend. The main work remaining is around documentation, and it's almost ready. Fingers crossed ;)

That sounds like a good schedule. I am looking forward to try out this new release.

Btw, this bug tracks the issue in the eclipse bugzilla: https://bugs.eclipse.org/bugs/show_bug.cgi?id=242798

Thanks for this pointer to the bug tracking on your side. I have added two new methods to JetS3tProperties that will make it possible to remove a specific property, or all the properties encapsulated by the class. I hope this change, in combination with the others that will be included with version 0.6.1, will resolve the issue.

As far as i can see, this will solve the issues we were having.

Thanks for your help on that problem. I will report back once the release is out.

Greets
Moritz