Cookies not set for webclient

View: New views
6 Messages — Rating Filter:   Alert me  

Cookies not set for webclient

by swarupa_shet :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello,

I am trying to post the request with Cookie; where when cookie is submitted in request headers ; additional authentication information (like: questions set on website for secure login) is not asked. I am trying to replicate similar behaviour with HTMLUNIT. I am setting the cookies in cookiemanager but somehow the cookie is not sent in headers. Does any one has any clue?

Following are the responses and code snippet:
WITHOUT HTMLUNIT
HTTP/1.1 302 Found
Connection: close
Date: Fri, 16 Oct 2009 11:34:56 GMT
Server: Microsoft-IIS/6.0
P3P: CP="CAO"
Expires: -1
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: /PassMarkFrame.aspx
Set-Cookie: PMData=PMV2CbS6jYCFjbEq6Jgu3TdlZXmV1qPPlzXgoRrKhicuOcfMFIpPZyYf/RrCsd2e5an/5NUqHDOK8MiE0f/j5BtlCbzbv/BH9Y4ScqHr3C9CsNv9ueym0xBt1zLGn45Rg4msat; expires=Sat, 16-Oct-2010 11:34:56 GMT; path=/; secure; HttpOnly
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-length: 12534

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fPassMarkFrame.aspx">here</a>.</h2>
</body></html>
 
WITH HTMLUNIT:
HTTP/1.1 302 Found
Date: Fri, 16 Oct 2009 11:33:52 GMT
Server: Microsoft-IIS/6.0
P3P: CP="CAO"
Expires: -1
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: /PassMarkFrame.aspx
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-length: 12534

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fPassMarkFrame.aspx">here</a>.</h2>
</body></html>
 

As seen above; HTMLUNIT is not sending "Set-Cookie:" header.

Following is the java code snippet:

CookieManager cookieManager = _webClient.getCookieManager();
            Cookie cookie = new Cookie();
            cookie.setName(name);
            cookie.setDomain(domain);
            cookie.setPath(path);
            cookie.setSecure(secure);
            cookie.setExpiryDate(cal.getTime());
            cookie.setValue(value);
            cookieManager.addCookie(cookie);
           
            _webClient.setCookieManager(cookieManager);
           
I am using webrequestsetting to pass on the request to server.
I also tried explictly setting/ adding additional header in webrequestsetting object as  "Set-Cookie:"

Is there any thing that i am missing?
Thanks for any help in advance.

Thanks,
--
Swarupa


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: Cookies not set for webclient

by gredler :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Swarupa,

I'm pretty sure Set-Cookie is an HTTP header sent by the server to the client, not by the client to the server; thus, I don't think HtmlUnit (or a real browser) will send this header... it just has the ability to *receive* this header from the server.

Take care,

Daniel


On Fri, Oct 16, 2009 at 7:50 AM, Swarupa Shet <swarupa.shet@...> wrote:
Hello,

I am trying to post the request with Cookie; where when cookie is submitted in request headers ; additional authentication information (like: questions set on website for secure login) is not asked. I am trying to replicate similar behaviour with HTMLUNIT. I am setting the cookies in cookiemanager but somehow the cookie is not sent in headers. Does any one has any clue?

Following are the responses and code snippet:
WITHOUT HTMLUNIT
HTTP/1.1 302 Found
Connection: close
Date: Fri, 16 Oct 2009 11:34:56 GMT
Server: Microsoft-IIS/6.0
P3P: CP="CAO"
Expires: -1
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: /PassMarkFrame.aspx
Set-Cookie: PMData=PMV2CbS6jYCFjbEq6Jgu3TdlZXmV1qPPlzXgoRrKhicuOcfMFIpPZyYf/RrCsd2e5an/5NUqHDOK8MiE0f/j5BtlCbzbv/BH9Y4ScqHr3C9CsNv9ueym0xBt1zLGn45Rg4msat; expires=Sat, 16-Oct-2010 11:34:56 GMT; path=/; secure; HttpOnly
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-length: 12534

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fPassMarkFrame.aspx">here</a>.</h2>
</body></html>
 
WITH HTMLUNIT:
HTTP/1.1 302 Found
Date: Fri, 16 Oct 2009 11:33:52 GMT
Server: Microsoft-IIS/6.0
P3P: CP="CAO"
Expires: -1
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: /PassMarkFrame.aspx
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-length: 12534

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fPassMarkFrame.aspx">here</a>.</h2>
</body></html>
 

As seen above; HTMLUNIT is not sending "Set-Cookie:" header.

Following is the java code snippet:

CookieManager cookieManager = _webClient.getCookieManager();
            Cookie cookie = new Cookie();
            cookie.setName(name);
            cookie.setDomain(domain);
            cookie.setPath(path);
            cookie.setSecure(secure);
            cookie.setExpiryDate(cal.getTime());
            cookie.setValue(value);
            cookieManager.addCookie(cookie);
           
            _webClient.setCookieManager(cookieManager);
           
I am using webrequestsetting to pass on the request to server.
I also tried explictly setting/ adding additional header in webrequestsetting object as  "Set-Cookie:"

Is there any thing that i am missing?
Thanks for any help in advance.

Thanks,
--
Swarupa


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user




--
Daniel Gredler
http://daniel.gredler.net/

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: Cookies not set for webclient

by swarupa_shet :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,
Thaks for response Daniel, Yes the information i sent is response of html request. It adds cookies values to the response; then the client also needs to sent same/appropriate cookie values but somehow HTMLUNIT is not sending that ; which in turn is asking additional questions instead of login to the site.

I still did not get working example for cookie manager , I am unable to find the solution to problem posted earlier (refer previous mail).

DO i need to write following for every cookie?
webClient.addRequestHeader("Cookie", this.cookie);

Or if any one has working code for cookiemanager please post it here.

Thanks,

On Tue, Oct 20, 2009 at 7:11 AM, Daniel Gredler <djgredler@...> wrote:
Hi Swarupa,

I'm pretty sure Set-Cookie is an HTTP header sent by the server to the client, not by the client to the server; thus, I don't think HtmlUnit (or a real browser) will send this header... it just has the ability to *receive* this header from the server.

Take care,

Daniel


On Fri, Oct 16, 2009 at 7:50 AM, Swarupa Shet <swarupa.shet@...> wrote:
Hello,

I am trying to post the request with Cookie; where when cookie is submitted in request headers ; additional authentication information (like: questions set on website for secure login) is not asked. I am trying to replicate similar behaviour with HTMLUNIT. I am setting the cookies in cookiemanager but somehow the cookie is not sent in headers. Does any one has any clue?

Following are the responses and code snippet:
WITHOUT HTMLUNIT
HTTP/1.1 302 Found
Connection: close
Date: Fri, 16 Oct 2009 11:34:56 GMT
Server: Microsoft-IIS/6.0
P3P: CP="CAO"
Expires: -1
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: /PassMarkFrame.aspx
Set-Cookie: PMData=PMV2CbS6jYCFjbEq6Jgu3TdlZXmV1qPPlzXgoRrKhicuOcfMFIpPZyYf/RrCsd2e5an/5NUqHDOK8MiE0f/j5BtlCbzbv/BH9Y4ScqHr3C9CsNv9ueym0xBt1zLGn45Rg4msat; expires=Sat, 16-Oct-2010 11:34:56 GMT; path=/; secure; HttpOnly
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-length: 12534

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fPassMarkFrame.aspx">here</a>.</h2>
</body></html>
 
WITH HTMLUNIT:
HTTP/1.1 302 Found
Date: Fri, 16 Oct 2009 11:33:52 GMT
Server: Microsoft-IIS/6.0
P3P: CP="CAO"
Expires: -1
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Location: /PassMarkFrame.aspx
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-length: 12534

<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fPassMarkFrame.aspx">here</a>.</h2>
</body></html>
 

As seen above; HTMLUNIT is not sending "Set-Cookie:" header.

Following is the java code snippet:

CookieManager cookieManager = _webClient.getCookieManager();
            Cookie cookie = new Cookie();
            cookie.setName(name);
            cookie.setDomain(domain);
            cookie.setPath(path);
            cookie.setSecure(secure);
            cookie.setExpiryDate(cal.getTime());
            cookie.setValue(value);
            cookieManager.addCookie(cookie);
           
            _webClient.setCookieManager(cookieManager);
           
I am using webrequestsetting to pass on the request to server.
I also tried explictly setting/ adding additional header in webrequestsetting object as  "Set-Cookie:"

Is there any thing that i am missing?
Thanks for any help in advance.

Thanks,
--
Swarupa


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user




--
Daniel Gredler
http://daniel.gredler.net/

------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user




--
Swarupa


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: Cookies not set for webclient

by Ahmed Ashour :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Some parts of this message have been removed. Learn more about Nabble's security policy.
Dear Swarupa,
 
>> It adds cookies values to the response; then the client also needs to sent same/appropriate cookie values but somehow HTMLUNIT is not sending that 
 
Fair enough, send the detailed response from server, and the next request from HtmlUnit/real browser. You may see that HtmlUnit is not sending the 'Cookie' header, but real browsers do.
 
The feature is already implemented, but we need reproducible case to investigate further. Do you have a public site?
 
Yours,
Ahmed
----
Blog: http://asashour.blogspot.com

From: Swarupa Shet <swarupa.shet@...>
To: htmlunit-user@...
Sent: Tue, October 20, 2009 9:42:08 AM
Subject: Re: [Htmlunit-user] Cookies not set for webclient

Hi,
Thaks for response Daniel, Yes the information i sent is response of html request. It adds cookies values to the response; then the client also needs to sent same/appropriate cookie values but somehow HTMLUNIT is not sending that ; which in turn is asking additional questions instead of login to the site.

I still did not get working example for cookie manager , I am unable to find the solution to problem posted earlier (refer previous mail).

DO i need to write following for every cookie?
webClient.addRequestHeader("Cookie", this.cookie);

Or if any one has working code for cookiemanager please post it here.

Thanks,


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: Cookies not set for webclient

by swarupa_shet :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

Thanks for your response Ahmed,
I am trying to login to bank's website , which is sensitive info and i do not want to post it here.
You said in your mail about the cookie header not sent through HTMLUNIT, i checked request of HTMLUNIT and plain browser , both sends cookie as header

Following is the scenario :

Plain browser
cookie is sent with first request , browser sends set-cookie header with updated value of the cookie ; Upon receiving updated value of the same cookie ; Plain browsers sends this new cookie values(not old one) .

HTMLUNIT
cookie is sent with first request , browser sends set-cookie header with updated value of the cookie ; Upon receiving updated value of the same cookie ; HTMLUNIT sends this new cookie values(not old one). But here , HTMLUNIT cashes the page which requires redirect

Following is code snippent:
<html><head><title>Object moved</title></head><body>
<h2>Object moved to <a href="%2fPassMarkFrame.aspx">here</a>.</h2>
</body></html>

When we try to get the page by "/PassMarkFrame.aspx" ; it sends old cooki value which server does not recognizes and asks for aditional questions.

I hope, i am clear on it.

Thanks,
Swarupa

On Tue, Oct 20, 2009 at 12:57 PM, Ahmed Ashour <asashour@...> wrote:
Dear Swarupa,
 
>> It adds cookies values to the response; then the client also needs to sent same/appropriate cookie values but somehow HTMLUNIT is not sending that 
 
Fair enough, send the detailed response from server, and the next request from HtmlUnit/real browser. You may see that HtmlUnit is not sending the 'Cookie' header, but real browsers do.
 
The feature is already implemented, but we need reproducible case to investigate further. Do you have a public site?
 
Yours,
Ahmed
----
Blog: http://asashour.blogspot.com

From: Swarupa Shet <swarupa.shet@...>
To: htmlunit-user@...
Sent: Tue, October 20, 2009 9:42:08 AM
Subject: Re: [Htmlunit-user] Cookies not set for webclient

Hi,
Thaks for response Daniel, Yes the information i sent is response of html request. It adds cookies values to the response; then the client also needs to sent same/appropriate cookie values but somehow HTMLUNIT is not sending that ; which in turn is asking additional questions instead of login to the site.

I still did not get working example for cookie manager , I am unable to find the solution to problem posted earlier (refer previous mail).

DO i need to write following for every cookie?
webClient.addRequestHeader("Cookie", this.cookie);

Or if any one has working code for cookiemanager please post it here.

Thanks,


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user




--
Swarupa


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user

Re: Cookies not set for webclient

by Marc Guillemot :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

you mean that HtmlUnit sends correctly the new cookie value and receives
a redirect request and perform this request with the old cookie value.
Is that correct?

Cheers,
Marc.

Swarupa Shet a écrit :

> Hi,
>
> Thanks for your response Ahmed,
> I am trying to login to bank's website , which is sensitive info and i
> do not want to post it here.
> You said in your mail about the cookie header not sent through HTMLUNIT,
> i checked request of HTMLUNIT and plain browser , both sends cookie as
> header
>
> Following is the scenario :
>
> *Plain browser*
> cookie is sent with first request , browser sends set-cookie header with
> updated value of the cookie ; Upon receiving updated value of the same
> cookie ; Plain browsers sends this new cookie values(not old one) .
>
> *HTMLUNIT*
> cookie is sent with first request , browser sends set-cookie header with
> updated value of the cookie ; Upon receiving updated value of the same
> cookie ; HTMLUNIT sends this new cookie values(not old one). But here ,
> HTMLUNIT cashes the page which requires redirect
>
> Following is code snippent:
> <html><head><title>Object moved</title></head><body>
> <h2>Object moved to <a href="%2fPassMarkFrame.aspx">here</a>.</h2>
> </body></html>
>
> When we try to get the page by "/PassMarkFrame.aspx" ; it sends old
> cooki value which server does not recognizes and asks for aditional
> questions.
>
> I hope, i am clear on it.
>
> Thanks,
> Swarupa
>
> On Tue, Oct 20, 2009 at 12:57 PM, Ahmed Ashour <asashour@...
> <mailto:asashour@...>> wrote:
>
>     Dear Swarupa,
>      
>      >> It adds cookies values to the response; then the client also
>     needs to sent same/appropriate cookie values but somehow HTMLUNIT is
>     not sending that
>      
>     Fair enough, send the detailed response from server, and the next
>     request from HtmlUnit/real browser. You may see that HtmlUnit is not
>     sending the 'Cookie' header, but real browsers do.
>      
>     The feature is already implemented, but we need reproducible case to
>     investigate further. Do you have a public site?
>      
>     Yours,
>     Ahmed
>     ----
>     Blog: http://asashour.blogspot.com
>     ------------------------------------------------------------------------
>     *From:* Swarupa Shet <swarupa.shet@...
>     <mailto:swarupa.shet@...>>
>     *To:* htmlunit-user@...
>     <mailto:htmlunit-user@...>
>     *Sent:* Tue, October 20, 2009 9:42:08 AM
>     *Subject:* Re: [Htmlunit-user] Cookies not set for webclient
>
>     Hi,
>     Thaks for response Daniel, Yes the information i sent is response of
>     html request. It adds cookies values to the response; then the
>     client also needs to sent same/appropriate cookie values but somehow
>     HTMLUNIT is not sending that ; which in turn is asking additional
>     questions instead of login to the site.
>
>     I still did not get working example for cookie manager , I am unable
>     to find the solution to problem posted earlier (refer previous mail).
>
>     DO i need to write following for every cookie?
>     webClient.addRequestHeader("Cookie", this.cookie);
>
>     Or if any one has working code for cookiemanager please post it here.
>
>     Thanks,
>
>
>     ------------------------------------------------------------------------------
>     Come build with us! The BlackBerry(R) Developer Conference in SF, CA
>     is the only developer event you need to attend this year. Jumpstart your
>     developing skills, take BlackBerry mobile applications to market and
>     stay
>     ahead of the curve. Join us from November 9 - 12, 2009. Register now!
>     http://p.sf.net/sfu/devconference
>     _______________________________________________
>     Htmlunit-user mailing list
>     Htmlunit-user@...
>     <mailto:Htmlunit-user@...>
>     https://lists.sourceforge.net/lists/listinfo/htmlunit-user
>
>
>
>
> --
> Swarupa
>
>
> ------------------------------------------------------------------------
>
> ------------------------------------------------------------------------------
> Come build with us! The BlackBerry(R) Developer Conference in SF, CA
> is the only developer event you need to attend this year. Jumpstart your
> developing skills, take BlackBerry mobile applications to market and stay
> ahead of the curve. Join us from November 9 - 12, 2009. Register now!
> http://p.sf.net/sfu/devconference
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> Htmlunit-user mailing list
> Htmlunit-user@...
> https://lists.sourceforge.net/lists/listinfo/htmlunit-user


------------------------------------------------------------------------------
Come build with us! The BlackBerry(R) Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9 - 12, 2009. Register now!
http://p.sf.net/sfu/devconference
_______________________________________________
Htmlunit-user mailing list
Htmlunit-user@...
https://lists.sourceforge.net/lists/listinfo/htmlunit-user