Pb with Microsoft Integrated Login and Squid 3.1

[NOGUES Jean-Marc (EURIWARE)]

[NOGUES Jean-Marc (EURIWARE)]

Hi,

I have upgraded our squid from 2.5 stable6  to 3.1.0.14 . This because
many remote web servers  want Microsoft  connection oriented
authentication and I 'have seen  that  squid 2.5 doesn't  forward that
kind of authentication. .

Now using squid  3.1, my users can connect such  web servers  but there
is still an issue..

From time to time  , when uploading a file , users  get a blank page and
message "Request not yet fully sent" can be  seen in cache.log file.

Sniffing this (sniffer between proxy  and web servers) I can see that,
from time to time, servers are going on sending authentication requests
although the user has been already authenticated (is it a normal
behaviour ?).

So sometimes it happens that Squid receives an authentication request as
it is  still  sending upload data to the  server.
This stops the upload  and produces the message seen in cache.log

Thank you for help .

Best Regards,

Jean-Marc Nogues
jmnogues@...

[Amos Jeffries-2]

NOGUES Jean-Marc (EURIWARE) wrote:

> Hi,
>
> I have upgraded our squid from 2.5 stable6  to 3.1.0.14 . This because
> many remote web servers  want Microsoft  connection oriented
> authentication and I 'have seen  that  squid 2.5 doesn't  forward that
> kind of authentication. .
>
> Now using squid  3.1, my users can connect such  web servers  but there
> is still an issue..
>
> From time to time  , when uploading a file , users  get a blank page and
> message "Request not yet fully sent" can be  seen in cache.log file.
>
> Sniffing this (sniffer between proxy  and web servers) I can see that,
> from time to time, servers are going on sending authentication requests
> although the user has been already authenticated (is it a normal
> behaviour ?).

Yes this is _usually_ normal.  HTTP being stateless the auth details
need to be sent on every request, or the client will be re-challenged.

I say "usually normal", because the client software should be aware of
that requirement and send the auth for as many requests as needed in the
session.

What is NOT normal here is seeing repeated series of missing-auth
requests followed by auth request from the same clients. This is a sign
of either client software breakage, NAT, or missing keep-alive data in
the requests. Persistent connections, aka keep-alive, is REQUIRED on
both the client and server connections for NTLM based auth along with
connection pinning to force stateless HTTP into stateful behavior
between the client and server.

>
> So sometimes it happens that Squid receives an authentication request as
> it is  still  sending upload data to the  server.
> This stops the upload  and produces the message seen in cache.log

Looks like you have hit a bug. Possibly the one people are struggling
with at present where a connections auth credentials are dropped
mid-session.

Can you supply any more detailed trace of whats going on please?

Amos
--
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.14

[NOGUES Jean-Marc (EURIWARE)]

Hi,

> I say "usually normal", because the client software should be aware of
> that requirement and send the auth for as many requests as needed in the  > session.

Sniffing between Squid and clients shows that clients never send auth data within further requests in the session. Clients  only send auth data just after  receiving an "HTTP/1.1 401 Unauthorized" from the remote web server.

Jean-Marc Nogues


-----Message d'origine-----
De : NOGUES Jean-Marc (EURIWARE)
Envoyé : mardi 3 novembre 2009 10:36
À : 'Amos Jeffries'
Objet : RE: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1

Hi Amos,

All clients have :
Windows XP SP2
and IE 6.0.2900.2180_xpsp_sp2_gdr.070227-2254 crypt=128 bits

At the bottom  of the trace joined we can see an incoming "HTTP/1.1 401 Unauthorized"and then the  rest of the  upload previously initiated  by the client.  

( Sorry but, for security reasons I had to to extract a .txt
 file from the original Winshark trace.
- tell if you need more  )
regards,

Jm Nogues



-----Message d'origine-----
De : Amos Jeffries [mailto:squid3@...]
Envoyé : mardi 3 novembre 2009 05:54
À : NOGUES Jean-Marc (EURIWARE)
Cc : squid-users@...
Objet : Re: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1

NOGUES Jean-Marc (EURIWARE) wrote:

> Hi,
>
> I have upgraded our squid from 2.5 stable6  to 3.1.0.14 . This because
> many remote web servers  want Microsoft  connection oriented
> authentication and I 'have seen  that  squid 2.5 doesn't  forward that
> kind of authentication. .
>
> Now using squid  3.1, my users can connect such  web servers  but there
> is still an issue..
>
> From time to time  , when uploading a file , users  get a blank page and
> message "Request not yet fully sent" can be  seen in cache.log file.
>
> Sniffing this (sniffer between proxy  and web servers) I can see that,
> from time to time, servers are going on sending authentication requests
> although the user has been already authenticated (is it a normal
> behaviour ?).

Yes this is _usually_ normal.  HTTP being stateless the auth details
need to be sent on every request, or the client will be re-challenged.

I say "usually normal", because the client software should be aware of
that requirement and send the auth for as many requests as needed in the
session.

What is NOT normal here is seeing repeated series of missing-auth
requests followed by auth request from the same clients. This is a sign
of either client software breakage, NAT, or missing keep-alive data in
the requests. Persistent connections, aka keep-alive, is REQUIRED on
both the client and server connections for NTLM based auth along with
connection pinning to force stateless HTTP into stateful behavior
between the client and server.

>
> So sometimes it happens that Squid receives an authentication request as
> it is  still  sending upload data to the  server.
> This stops the upload  and produces the message seen in cache.log

Looks like you have hit a bug. Possibly the one people are struggling
with at present where a connections auth credentials are dropped
mid-session.

Can you supply any more detailed trace of whats going on please?

Amos
--
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.14

[Amos Jeffries-2]

NOGUES Jean-Marc (EURIWARE) wrote:
> Hi,
>
>> I say "usually normal", because the client software should be aware of
>> that requirement and send the auth for as many requests as needed in the  > session.
>
> Sniffing between Squid and clients shows that clients never send auth data within further requests in the session.

Strange. Smells like broken client software.

 > Clients  only send auth data just after  receiving an "HTTP/1.1 401
Unauthorized" from the remote web server.
>


What you should be seeing is series of patterns like this:

CLIENT: request
WEB: 401 auth-missing
CLIENT: request+auth+keepalive
WEB: 200 Okay
CLIENT: request+auth+keepalive
WEB: 200 Okay
CLIENT: request+auth+keepalive
WEB: 200 Okay
CLIENT: request+auth+close
WEB: 200 Okay

... some time later (after browser closed and restarted for second session).

CLIENT: request
WEB: 401 auth-missing
CLIENT: request+auth+keepalive
WEB: 200 Okay
CLIENT: request+auth+close
WEB: 200 Okay


Amos

>
> -----Message d'origine-----
> De : NOGUES Jean-Marc (EURIWARE)
> Envoyé : mardi 3 novembre 2009 10:36
> À : 'Amos Jeffries'
> Objet : RE: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1
>
> Hi Amos,
>
> All clients have :
> Windows XP SP2
> and IE 6.0.2900.2180_xpsp_sp2_gdr.070227-2254 crypt=128 bits
>
> At the bottom  of the trace joined we can see an incoming "HTTP/1.1 401 Unauthorized"and then the  rest of the  upload previously initiated  by the client.  
>
> ( Sorry but, for security reasons I had to to extract a .txt
>  file from the original Winshark trace.
> - tell if you need more  )
> regards,
>
> Jm Nogues
>
>
>
> -----Message d'origine-----
> De : Amos Jeffries [mailto:squid3@...]
> Envoyé : mardi 3 novembre 2009 05:54
> À : NOGUES Jean-Marc (EURIWARE)
> Cc : squid-users@...
> Objet : Re: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1
>
> NOGUES Jean-Marc (EURIWARE) wrote:
>> Hi,
>>
>> I have upgraded our squid from 2.5 stable6  to 3.1.0.14 . This because
>> many remote web servers  want Microsoft  connection oriented
>> authentication and I 'have seen  that  squid 2.5 doesn't  forward that
>> kind of authentication. .
>>
>> Now using squid  3.1, my users can connect such  web servers  but there
>> is still an issue..
>>
>> From time to time  , when uploading a file , users  get a blank page and
>> message "Request not yet fully sent" can be  seen in cache.log file.
>>
>> Sniffing this (sniffer between proxy  and web servers) I can see that,
>> from time to time, servers are going on sending authentication requests
>> although the user has been already authenticated (is it a normal
>> behaviour ?).
>
> Yes this is _usually_ normal.  HTTP being stateless the auth details
> need to be sent on every request, or the client will be re-challenged.
>
> I say "usually normal", because the client software should be aware of
> that requirement and send the auth for as many requests as needed in the
> session.
>
> What is NOT normal here is seeing repeated series of missing-auth
> requests followed by auth request from the same clients. This is a sign
> of either client software breakage, NAT, or missing keep-alive data in
> the requests. Persistent connections, aka keep-alive, is REQUIRED on
> both the client and server connections for NTLM based auth along with
> connection pinning to force stateless HTTP into stateful behavior
> between the client and server.
>
>> So sometimes it happens that Squid receives an authentication request as
>> it is  still  sending upload data to the  server.
>> This stops the upload  and produces the message seen in cache.log
>
> Looks like you have hit a bug. Possibly the one people are struggling
> with at present where a connections auth credentials are dropped
> mid-session.
>
> Can you supply any more detailed trace of whats going on please?
>
> Amos

--
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.14

[NOGUES Jean-Marc (EURIWARE)]

Hi,

I have managed to make the clients connect directly the web server (so no proxy in the middle ..)

What I am seing in the same session is this :(according that  by "auth-missing" you mean an "HTTP 401 Unauthorized" ?)

CLIENT: request (post)
WEB: 401 auth-missing (Negociate)
CLIENT: request (post) +auth (Negociate) +keepalive
WEB: 200 Okay
CLIENT: request (post) + keepalive
WEB: 401 auth-missing (Negociate)
CLIENT: request (post) +auth+ (Negociate) +  keepalive
WEB: 200 Okay
CLIENT: request (post) + keepalive
WEB: 401 auth-missing (Negociate)
 
.. and so on ..

- The remote site is here a Publigen site , but this pb generally occurs with Sharepoint sites which also require Integrated Authentication .

- So user data has to be sent twice ( not very good for the bandwith ... )  
- Value of Authorisation header is "Negociate" (Kerberos I presume ..)

I will try soon with another browser than IE .
(actually all browsers  are and IE 6.0.2900.2180_xpsp_sp2_gdr.070227-2254 crypt=128 bits

Regards,

Jm Nogues


-----Message d'origine-----
De : Amos Jeffries [mailto:squid3@...]
Envoyé : jeudi 5 novembre 2009 05:27
Cc : squid-users@...
Objet : Re: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1

NOGUES Jean-Marc (EURIWARE) wrote:
> Hi,
>
>> I say "usually normal", because the client software should be aware of
>> that requirement and send the auth for as many requests as needed in the  > session.
>
> Sniffing between Squid and clients shows that clients never send auth data within further requests in the session.

Strange. Smells like broken client software.

 > Clients  only send auth data just after  receiving an "HTTP/1.1 401
Unauthorized" from the remote web server.
>


What you should be seeing is series of patterns like this:

CLIENT: request
WEB: 401 auth-missing
CLIENT: request+auth+keepalive
WEB: 200 Okay
CLIENT: request+auth+keepalive
WEB: 200 Okay
CLIENT: request+auth+keepalive
WEB: 200 Okay
CLIENT: request+auth+close
WEB: 200 Okay

... some time later (after browser closed and restarted for second session).

CLIENT: request
WEB: 401 auth-missing
CLIENT: request+auth+keepalive
WEB: 200 Okay
CLIENT: request+auth+close
WEB: 200 Okay


Amos

>
> -----Message d'origine-----
> De : NOGUES Jean-Marc (EURIWARE)
> Envoyé : mardi 3 novembre 2009 10:36
> À : 'Amos Jeffries'
> Objet : RE: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1
>
> Hi Amos,
>
> All clients have :
> Windows XP SP2
> and IE 6.0.2900.2180_xpsp_sp2_gdr.070227-2254 crypt=128 bits
>
> At the bottom  of the trace joined we can see an incoming "HTTP/1.1 401 Unauthorized"and then the  rest of the  upload previously initiated  by the client.  
>
> ( Sorry but, for security reasons I had to to extract a .txt
>  file from the original Winshark trace.
> - tell if you need more  )
> regards,
>
> Jm Nogues
>
>
>
> -----Message d'origine-----
> De : Amos Jeffries [mailto:squid3@...]
> Envoyé : mardi 3 novembre 2009 05:54
> À : NOGUES Jean-Marc (EURIWARE)
> Cc : squid-users@...
> Objet : Re: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1
>
> NOGUES Jean-Marc (EURIWARE) wrote:
>> Hi,
>>
>> I have upgraded our squid from 2.5 stable6  to 3.1.0.14 . This because
>> many remote web servers  want Microsoft  connection oriented
>> authentication and I 'have seen  that  squid 2.5 doesn't  forward that
>> kind of authentication. .
>>
>> Now using squid  3.1, my users can connect such  web servers  but there
>> is still an issue..
>>
>> From time to time  , when uploading a file , users  get a blank page and
>> message "Request not yet fully sent" can be  seen in cache.log file.
>>
>> Sniffing this (sniffer between proxy  and web servers) I can see that,
>> from time to time, servers are going on sending authentication requests
>> although the user has been already authenticated (is it a normal
>> behaviour ?).
>
> Yes this is _usually_ normal.  HTTP being stateless the auth details
> need to be sent on every request, or the client will be re-challenged.
>
> I say "usually normal", because the client software should be aware of
> that requirement and send the auth for as many requests as needed in the
> session.
>
> What is NOT normal here is seeing repeated series of missing-auth
> requests followed by auth request from the same clients. This is a sign
> of either client software breakage, NAT, or missing keep-alive data in
> the requests. Persistent connections, aka keep-alive, is REQUIRED on
> both the client and server connections for NTLM based auth along with
> connection pinning to force stateless HTTP into stateful behavior
> between the client and server.
>
>> So sometimes it happens that Squid receives an authentication request as
>> it is  still  sending upload data to the  server.
>> This stops the upload  and produces the message seen in cache.log
>
> Looks like you have hit a bug. Possibly the one people are struggling
> with at present where a connections auth credentials are dropped
> mid-session.
>
> Can you supply any more detailed trace of whats going on please?
>
> Amos

--
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.14

[Amos Jeffries-2]

NOGUES Jean-Marc (EURIWARE) wrote:

> Hi,
>
> I have managed to make the clients connect directly the web server (so no proxy in the middle ..)
>
> What I am seing in the same session is this :(according that  by "auth-missing" you mean an "HTTP 401 Unauthorized" ?)
>
> CLIENT: request (post)
> WEB: 401 auth-missing (Negociate)
> CLIENT: request (post) +auth (Negociate) +keepalive
> WEB: 200 Okay
> CLIENT: request (post) + keepalive
> WEB: 401 auth-missing (Negociate)
> CLIENT: request (post) +auth+ (Negociate) +  keepalive
> WEB: 200 Okay
> CLIENT: request (post) + keepalive
> WEB: 401 auth-missing (Negociate)
>  
> .. and so on ..

Ouch. Definitely a bug in the browser then.

>
> - The remote site is here a Publigen site , but this pb generally occurs with Sharepoint sites which also require Integrated Authentication .
>
> - So user data has to be sent twice ( not very good for the bandwith ... )  
> - Value of Authorisation header is "Negociate" (Kerberos I presume ..)
>
> I will try soon with another browser than IE .
> (actually all browsers  are and IE 6.0.2900.2180_xpsp_sp2_gdr.070227-2254 crypt=128 bits
>
> Regards,
>
> Jm Nogues
>

I've heard a lot of grumbles from many different web people about IE6 in
particular who love other releases. You may find this problem fixed in
later releases of IE.

Amos

>
> -----Message d'origine-----
> De : Amos Jeffries [mailto:squid3@...]
> Envoyé : jeudi 5 novembre 2009 05:27
> Cc : squid-users@...
> Objet : Re: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1
>
> NOGUES Jean-Marc (EURIWARE) wrote:
>> Hi,
>>
>>> I say "usually normal", because the client software should be aware of
>>> that requirement and send the auth for as many requests as needed in the  > session.
>> Sniffing between Squid and clients shows that clients never send auth data within further requests in the session.
>
> Strange. Smells like broken client software.
>
>  > Clients  only send auth data just after  receiving an "HTTP/1.1 401
> Unauthorized" from the remote web server.
>
>
> What you should be seeing is series of patterns like this:
>
> CLIENT: request
> WEB: 401 auth-missing
> CLIENT: request+auth+keepalive
> WEB: 200 Okay
> CLIENT: request+auth+keepalive
> WEB: 200 Okay
> CLIENT: request+auth+keepalive
> WEB: 200 Okay
> CLIENT: request+auth+close
> WEB: 200 Okay
>
> ... some time later (after browser closed and restarted for second session).
>
> CLIENT: request
> WEB: 401 auth-missing
> CLIENT: request+auth+keepalive
> WEB: 200 Okay
> CLIENT: request+auth+close
> WEB: 200 Okay
>
>
> Amos
>
>> -----Message d'origine-----
>> De : NOGUES Jean-Marc (EURIWARE)
>> Envoyé : mardi 3 novembre 2009 10:36
>> À : 'Amos Jeffries'
>> Objet : RE: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1
>>
>> Hi Amos,
>>
>> All clients have :
>> Windows XP SP2
>> and IE 6.0.2900.2180_xpsp_sp2_gdr.070227-2254 crypt=128 bits
>>
>> At the bottom  of the trace joined we can see an incoming "HTTP/1.1 401 Unauthorized"and then the  rest of the  upload previously initiated  by the client.  
>>
>> ( Sorry but, for security reasons I had to to extract a .txt
>>  file from the original Winshark trace.
>> - tell if you need more  )
>> regards,
>>
>> Jm Nogues
>>
>>
>>
>> -----Message d'origine-----
>> De : Amos Jeffries [mailto:squid3@...]
>> Envoyé : mardi 3 novembre 2009 05:54
>> À : NOGUES Jean-Marc (EURIWARE)
>> Cc : squid-users@...
>> Objet : Re: [squid-users] Pb with Microsoft Integrated Login and Squid 3.1
>>
>> NOGUES Jean-Marc (EURIWARE) wrote:
>>> Hi,
>>>
>>> I have upgraded our squid from 2.5 stable6  to 3.1.0.14 . This because
>>> many remote web servers  want Microsoft  connection oriented
>>> authentication and I 'have seen  that  squid 2.5 doesn't  forward that
>>> kind of authentication. .
>>>
>>> Now using squid  3.1, my users can connect such  web servers  but there
>>> is still an issue..
>>>
>>> From time to time  , when uploading a file , users  get a blank page and
>>> message "Request not yet fully sent" can be  seen in cache.log file.
>>>
>>> Sniffing this (sniffer between proxy  and web servers) I can see that,
>>> from time to time, servers are going on sending authentication requests
>>> although the user has been already authenticated (is it a normal
>>> behaviour ?).
>> Yes this is _usually_ normal.  HTTP being stateless the auth details
>> need to be sent on every request, or the client will be re-challenged.
>>
>> I say "usually normal", because the client software should be aware of
>> that requirement and send the auth for as many requests as needed in the
>> session.
>>
>> What is NOT normal here is seeing repeated series of missing-auth
>> requests followed by auth request from the same clients. This is a sign
>> of either client software breakage, NAT, or missing keep-alive data in
>> the requests. Persistent connections, aka keep-alive, is REQUIRED on
>> both the client and server connections for NTLM based auth along with
>> connection pinning to force stateless HTTP into stateful behavior
>> between the client and server.
>>
>>> So sometimes it happens that Squid receives an authentication request as
>>> it is  still  sending upload data to the  server.
>>> This stops the upload  and produces the message seen in cache.log
>> Looks like you have hit a bug. Possibly the one people are struggling
>> with at present where a connections auth credentials are dropped
>> mid-session.
>>
>> Can you supply any more detailed trace of whats going on please?
>>
>> Amos
>
>

--
Please be using
   Current Stable Squid 2.7.STABLE7 or 3.0.STABLE20
   Current Beta Squid 3.1.0.14

[Henrik Nordstrom-5]

ons 2009-11-04 klockan 18:25 +0100 skrev NOGUES Jean-Marc (EURIWARE):
> Hi,
>
> > I say "usually normal", because the client software should be aware of
> > that requirement and send the auth for as many requests as needed in the  > session.
>
> Sniffing between Squid and clients shows that clients never send auth
> data within further requests in the session. Clients  only send auth
> data just after  receiving an "HTTP/1.1 401 Unauthorized" from the
> remote web server.

Negotiate (and NTLM) is connection oriented non-HTTP compliuant auth
schemes (basic principle of HTTP messaging violated). Because of this
aut credentials is only seen on the first request per TCP connection.
Once auth have completed on that connection further requests on tha same
connection looks like they are anonymous but they are in fact not...
(auth silently inherited from the TCP connection).

Regards
Henrik