Redirect (and cookies) support in HTTP_Request2 / HTTP_Client2?

[Alexey Borzov]

Hi,

I'd like to gather some feedback on how to properly implement redirect and
cookies support in HTTP_Request2 and / or HTTP_Client2.


First, a bit of background. Redirect support was first done in HTTP_Request by
its original author, but as the class wasn't designed from the ground up for
performing multiple requests, redirect implementation was a bit of a hack. It is
currently disabled by default and its use is discouraged.

HTTP_Client has a cleaner implementation, it uses a new instance of HTTP_Request
for doing a redirect, has proper code for resolving relative URLs.

HTTP_Client also has the following useful features:
  * Stores cookies between requests (persistent storage is also possible);
  * Stores default headers, default request parameters and default listeners.
These are added to all created instances of HTTP_Request;
  * Can store complete response history;
  * get() / post() / head() / put() / delete() convenience methods.


Now, skip to HTTP_Request2. I suspect that if one enables redirect support in
cURL adapter, it will work out of the box. It also isn't that difficult to add
naïve redirect implementation to Socket adapter, only a matter of porting the
relevant code from HTTP_Client::_performRequest().


I'd also like to implement cookie storage support, though, and make us of Public
Suffix List (http://publicsuffix.org/). The big question is implementing that
*and* redirect support. Cookies can easily be extracted from redirect responses
via Observers, but can they also be injected there (esp. with Curl)? If it's not
possible, then Curl's native redirect support can not be used and the same
PHP-based implementation should be used for both adapters. It will be easier to
push that to a separate class (HTTP_Client2?).


Also I'm interested whether other features of HTTP_Client are still as relevant
with HTTP_Request2 as they were with HTTP_Request.

--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[tfk]

On Sun, Nov 1, 2009 at 4:40 PM, Alexey Borzov <borz_off@...> wrote:

> Hi,
>
> I'd like to gather some feedback on how to properly implement redirect and
> cookies support in HTTP_Request2 and / or HTTP_Client2.
>
>
> First, a bit of background. Redirect support was first done in HTTP_Request
> by its original author, but as the class wasn't designed from the ground up
> for performing multiple requests, redirect implementation was a bit of a
> hack. It is currently disabled by default and its use is discouraged.
>
> HTTP_Client has a cleaner implementation, it uses a new instance of
> HTTP_Request for doing a redirect, has proper code for resolving relative
> URLs.
>
> HTTP_Client also has the following useful features:
>  * Stores cookies between requests (persistent storage is also possible);
>  * Stores default headers, default request parameters and default listeners.
> These are added to all created instances of HTTP_Request;
>  * Can store complete response history;
>  * get() / post() / head() / put() / delete() convenience methods.
>
>
> Now, skip to HTTP_Request2. I suspect that if one enables redirect support
> in cURL adapter, it will work out of the box. It also isn't that difficult
> to add naïve redirect implementation to Socket adapter, only a matter of
> porting the relevant code from HTTP_Client::_performRequest().
>
>
> I'd also like to implement cookie storage support, though, and make us of
> Public Suffix List (http://publicsuffix.org/). The big question is
> implementing that *and* redirect support. Cookies can easily be extracted
> from redirect responses via Observers, but can they also be injected there
> (esp. with Curl)? If it's not possible, then Curl's native redirect support
> can not be used and the same PHP-based implementation should be used for
> both adapters. It will be easier to push that to a separate class
> (HTTP_Client2?).
>
>
> Also I'm interested whether other features of HTTP_Client are still as
> relevant with HTTP_Request2 as they were with HTTP_Request.
>

Cookie-support (well, a cookie jar :-)) and redirects would be great.
Those two are (IMHO), the most important features.

Default headers, storing request history and the convenience methods
are a nice to have, I wouldn't say no to them.

I'm already a fan and using HTTP_Request2 in almost a dozen projects.
Thanks for working on this, I'm really looking forward to
HTTP_Client2.

Till

--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Bill Shupp]

On Nov 1, 2009, at 7:40 AM, Alexey Borzov wrote:

> Hi,
>
> I'd like to gather some feedback on how to properly implement  
> redirect and cookies support in HTTP_Request2 and / or HTTP_Client2.
>
>
> First, a bit of background. Redirect support was first done in  
> HTTP_Request by its original author, but as the class wasn't  
> designed from the ground up for performing multiple requests,  
> redirect implementation was a bit of a hack. It is currently  
> disabled by default and its use is discouraged.
>
> HTTP_Client has a cleaner implementation, it uses a new instance of  
> HTTP_Request for doing a redirect, has proper code for resolving  
> relative URLs.
>
> HTTP_Client also has the following useful features:
> * Stores cookies between requests (persistent storage is also  
> possible);
> * Stores default headers, default request parameters and default  
> listeners. These are added to all created instances of HTTP_Request;
> * Can store complete response history;
> * get() / post() / head() / put() / delete() convenience methods.
>
>
> Now, skip to HTTP_Request2. I suspect that if one enables redirect  
> support in cURL adapter, it will work out of the box. It also isn't  
> that difficult to add naïve redirect implementation to Socket  
> adapter, only a matter of porting the relevant code from  
> HTTP_Client::_performRequest().
>
>
> I'd also like to implement cookie storage support, though, and make  
> us of Public Suffix List (http://publicsuffix.org/). The big  
> question is implementing that *and* redirect support. Cookies can  
> easily be extracted from redirect responses via Observers, but can  
> they also be injected there (esp. with Curl)? If it's not possible,  
> then Curl's native redirect support can not be used and the same PHP-
> based implementation should be used for both adapters. It will be  
> easier to push that to a separate class (HTTP_Client2?).
>
>
> Also I'm interested whether other features of HTTP_Client are still  
> as relevant with HTTP_Request2 as they were with HTTP_Request.
>

My feeling is that I don't think we need both HTTP_Client2 and  
HTTP_Request2 going forward.  Just put the missing features in  
HTTP_Request2.  My current feature interest is that of redirects, but  
I can see the other HTTP_Client features being useful as well if you  
want to bring them over.

Regards,

Bill
--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Michael Gauthier]

On Sun, 2009-11-01 at 18:40 +0300, Alexey Borzov wrote:

> Hi,
>
> I'd like to gather some feedback on how to properly implement redirect and
> cookies support in HTTP_Request2 and / or HTTP_Client2.
>
>
> First, a bit of background. Redirect support was first done in HTTP_Request by
> its original author, but as the class wasn't designed from the ground up for
> performing multiple requests, redirect implementation was a bit of a hack. It is
> currently disabled by default and its use is discouraged.
>
> HTTP_Client has a cleaner implementation, it uses a new instance of HTTP_Request
> for doing a redirect, has proper code for resolving relative URLs.
>
> HTTP_Client also has the following useful features:
>   * Stores cookies between requests (persistent storage is also possible);
>   * Stores default headers, default request parameters and default listeners.
> These are added to all created instances of HTTP_Request;
>   * Can store complete response history;
>   * get() / post() / head() / put() / delete() convenience methods.
>
>
> Now, skip to HTTP_Request2. I suspect that if one enables redirect support in
> cURL adapter, it will work out of the box. It also isn't that difficult to add
> naïve redirect implementation to Socket adapter, only a matter of porting the
> relevant code from HTTP_Client::_performRequest().
>
>
> I'd also like to implement cookie storage support, though, and make us of Public
> Suffix List (http://publicsuffix.org/). The big question is implementing that
> *and* redirect support. Cookies can easily be extracted from redirect responses
> via Observers, but can they also be injected there (esp. with Curl)? If it's not
> possible, then Curl's native redirect support can not be used and the same
> PHP-based implementation should be used for both adapters. It will be easier to
> push that to a separate class (HTTP_Client2?).
>
>
> Also I'm interested whether other features of HTTP_Client are still as relevant
> with HTTP_Request2 as they were with HTTP_Request.
>

I think splitting cookie-handling and redirect support into a separate
package makes sense. It's the easiest way to make it possible to get the
complete response history for redirects, and to have persistent cookie
storage backends.

By putting these features in HTTP_Client2, you can keep the
HTTP_Request2 API lean and focused, like it is now.

I don't (currently) have a need for either of these features, but a
separate package seems like the logical place to put them.

Cheers,


Mike





--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Alexey Borzov]

Hi Michael,

Michael Gauthier wrote:

>> Now, skip to HTTP_Request2. I suspect that if one enables redirect support in
>> cURL adapter, it will work out of the box. It also isn't that difficult to add
>> naïve redirect implementation to Socket adapter, only a matter of porting the
>> relevant code from HTTP_Client::_performRequest().
>>
>>
>> I'd also like to implement cookie storage support, though, and make us of Public
>> Suffix List (http://publicsuffix.org/). The big question is implementing that
>> *and* redirect support. Cookies can easily be extracted from redirect responses
>> via Observers, but can they also be injected there (esp. with Curl)? If it's not
>> possible, then Curl's native redirect support can not be used and the same
>> PHP-based implementation should be used for both adapters. It will be easier to
>> push that to a separate class (HTTP_Client2?).
>>
>>
>> Also I'm interested whether other features of HTTP_Client are still as relevant
>> with HTTP_Request2 as they were with HTTP_Request.
>>
>
> I think splitting cookie-handling and redirect support into a separate
> package makes sense. It's the easiest way to make it possible to get the
> complete response history for redirects, and to have persistent cookie
> storage backends.

Note that complete response history for redirects will be available in either
case due to Observers.

I am also not planning to do any persistent cookie storage myself, I'll only
implement serialization helpers __sleep() and __wakeup() like in current
HTTP_Client_CookieManager.

> By putting these features in HTTP_Client2, you can keep the
> HTTP_Request2 API lean and focused, like it is now.

API bloat is a justified concern, but these features may only add a couple of
configuration parameters ('allow_redirects' and 'max_redirects') and a couple of
methods (setCookieJar() and getCookieJar()).

> I don't (currently) have a need for either of these features, but a
> separate package seems like the logical place to put them.

I made a quick check with Curl adapter, and as I expected enabling redirects
with that required only changing
CURLOPT_FOLLOWLOCATION => false
to
CURLOPT_FOLLOWLOCATION => true
in one strategic place. It worked correctly, sending proper events to an
Observer and returning the response with the page it was redirected to.

Setting CURLOPT_COOKIE from within the callback (i.e. after the original request
but before the redirect) also worked.

So Curl can use its native redirect handling and will only need Cookie Jar
implemented. Since this Cookie Jar will be used from within Curl Adapter, it
makes no sense putting it into the separate package...

Looks like we won't see HTTP_Client2 after all, no sense implenenting a package
for a few convenience methods. :]

--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Alexey Borzov]

Hi Bill,

Bill Shupp wrote:
> My feeling is that I don't think we need both HTTP_Client2 and
> HTTP_Request2 going forward.  Just put the missing features in
> HTTP_Request2.  My current feature interest is that of redirects, but I
> can see the other HTTP_Client features being useful as well if you want
> to bring them over.

I tend to agree that a separate package doesn't make sense. I'll probably be
able to add redirect support to HTTP_Request2 in a few weeks, Cookie jar support
is a more exotic feature and can wait a bit longer.

--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Bill Shupp]

On Nov 1, 2009, at 1:38 PM, Alexey Borzov wrote:

> Hi Bill,
>
> Bill Shupp wrote:
>> My feeling is that I don't think we need both HTTP_Client2 and  
>> HTTP_Request2 going forward.  Just put the missing features in  
>> HTTP_Request2.  My current feature interest is that of redirects,  
>> but I can see the other HTTP_Client features being useful as well  
>> if you want to bring them over.
>
> I tend to agree that a separate package doesn't make sense. I'll  
> probably be able to add redirect support to HTTP_Request2 in a few  
> weeks, Cookie jar support is a more exotic feature and can wait a  
> bit longer.

In the short term, can I get ssl peer validation working with  
HTTP_Request?  This doesn't work for me:

$http = new HTTP_Request('https://example.com', array('socketOptions'  
=> array('verify_peer' => true)));

I end up with a PEAR_Error, the message being "Operation timed out".

Thanks,

Bill

--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Bill Shupp]

On Nov 1, 2009, at 3:53 PM, Bill Shupp wrote:

> On Nov 1, 2009, at 1:38 PM, Alexey Borzov wrote:
>
>> Hi Bill,
>>
>> Bill Shupp wrote:
>>> My feeling is that I don't think we need both HTTP_Client2 and  
>>> HTTP_Request2 going forward.  Just put the missing features in  
>>> HTTP_Request2.  My current feature interest is that of redirects,  
>>> but I can see the other HTTP_Client features being useful as well  
>>> if you want to bring them over.
>>
>> I tend to agree that a separate package doesn't make sense. I'll  
>> probably be able to add redirect support to HTTP_Request2 in a few  
>> weeks, Cookie jar support is a more exotic feature and can wait a  
>> bit longer.
>
> In the short term, can I get ssl peer validation working with  
> HTTP_Request?  This doesn't work for me:
>
> $http = new HTTP_Request('https://example.com', array
> ('socketOptions' => array('verify_peer' => true)));
>
> I end up with a PEAR_Error, the message being "Operation timed out".

That kinda sounded like a feature request, which it isn't.  I'm really  
just trying to find out if it's possible with the current HTTP_Request  
version.

Cheers,

Bill

--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Alexey Borzov]

Hi Bill,

Bill Shupp wrote:

>> In the short term, can I get ssl peer validation working with
>> HTTP_Request?  This doesn't work for me:
>>
>> $http = new HTTP_Request('https://example.com', array('socketOptions'
>> => array('verify_peer' => true)));
>>
>> I end up with a PEAR_Error, the message being "Operation timed out".
>
> That kinda sounded like a feature request, which it isn't.  I'm really
> just trying to find out if it's possible with the current HTTP_Request
> version.

Sorry, can't really help here. socketOptions aren't used by HTTP_Request, they
are only passed to Net_Socket's constructor, which uses them to create socket
context. So if this doesn't work, there are most probably some issues with PHP's
SSL support.

--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Alexey Borzov]

Hi,

A bit of heads-up: I committed redirect support in HTTP_Request2 to SVN, those
interested may test it. It will be released as 0.5.0 after I make a few more
unrelated small fixes, so please give feedback if something works not as expected.

Alexey Borzov wrote:
> I'd like to gather some feedback on how to properly implement redirect
> and cookies support in HTTP_Request2 and / or HTTP_Client2.

--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php

[Bill Shupp]

On Nov 15, 2009, at 3:33 AM, Alexey Borzov wrote:

> Hi,
>
> A bit of heads-up: I committed redirect support in HTTP_Request2 to SVN, those interested may test it. It will be released as 0.5.0 after I make a few more unrelated small fixes, so please give feedback if something works not as expected.
>

Alexey,

Out of the box it's working as expected for me.  Thanks a bunch for getting this done!  I'll start moving some of my packages that require redirects to it.

Cheers,

Bill
--
PEAR Development Mailing List (http://pear.php.net/)
To unsubscribe, visit: http://www.php.net/unsub.php