HTTP PUT strange behavior

[modiadi]

Hello,
I try to upload a file from an iPhone app to Grok application. The basic
idea is that I spent half of day searching for a solution and I believe that
is a bug on the server side(grok,zoppe or paster). After a lot
of frustration I have downloaded php and I tried the same code with a
php server-side application which works ! I also confirms a potential bug on
the grok side.

I am using  Grok 1.0a4 in a very simple code like this:

import grok
from zope.interface import Interface

class ZDUploadContainer(grok.Model):
pass
class IphoneUploadLayer(grok.IRESTLayer):
    grok.restskin('up')

class ZendriveREST(grok.REST):
  grok.context( ZDUploadContainer)
  grok.layer(IphoneUploadLayer)

  def PUT(self):
        import pdb;pdb.set_trace();
        self.response.setStatus(200)
        return "PUT request"
  def GET(self):
        return "GET request"

I am using a REST layer because is the only way I know to access HTTP PUT
data.

Testing from browser http://localhost:8080/++rest++up/zd/i/ works returning
a page with correct response "GET request".

Now from the phone app starts the interesting part...
Here is the HTTP conversation.
The phone sends as HEADER :

PUT /++rest++up/zd/i HTTP/1.1
Host: localhost:8080
User-Agent: zendrive/1.0 CFNetwork/445.3 Darwin/9.7.0
Content-Type: application/octet-stream
Content-Length: 4
Accept: */*
Accept-Language:en-us
Accept-Encoding: gzip, deflate
Connection: keep-alive

The BODY of the message is TEST which has 4 bytes as length. What happens is
that the after sending all the data it doesn't get any response back(in the
HTTP sniffer I have a Waiting ... status).  If I kill the sending app before
timing out, on the grok side I get the debugger and I am inside the PUT
method !

If i try:
(Pdb) p self.body
'TEST'
Which is correct.  The problem is that I had to kill the iphone app to get
there.

Two interesting things :
if I don't send Content-Length at all, the HTTP header and the body is the
same (in the HTTP sniffer) except missing Content-Length. The debugger fires
up and I am inside the PUT method, but I have no body for the message :
(Pdb) p self.body
''
As son as I press c connection closes and everything is ok on the iphone
side.

Next step is with Content-Length:3 .. I have the same thing but after the
debugger starts:
(Pdb) p self.body
'TES'
The body is 3 caracters legth instead of 4 ... and after I continue the
iPhone app gives me a Error (opperation cannot be completed) and HTTP
sniffer tells me that the connection status is : Resset by peer.

As I explained with a simple php script all works on the phone side and
serve side!.
The conclusion is that if in a PUT request I put a Content-length :
1: bigger or equal of the body size - the wed server expects more data until
it times out, it never gives the control to PUT method
2: 0 - the body is sent to the server but inside PUT method self.body is
empty
3. less than the size of the body : bad on the client side(connection
reseted by peer) and on the server side inside PUT self.body is truncated.

First of all if anybody has any suggestions they are more than welcome.

I have read a related problem on the zope forums which they solved by
changing the web server from paster to zserver. I don't understand this part
and I don't know how to do it (if posible in grok).

Also do you think that is posible to get the content of a PUT request in a
difrent way ?

Thank you all for reading this long post ....


Adrian Tofan
_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[Michael Haubenwallner-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Adrian Tofan wrote:
> I have read a related problem on the zope forums which they solved by
> changing the web server from paster to zserver. I don't understand this part
> and I don't know how to do it (if posible in grok).
>

I found this in http://pypi.python.org/pypi/zope.publisher

3.5.3 (2008-06-20)

Bugs fixed:

    * It turns out that some Web servers (Paste for example) do not send
the EOF character after the data has been transmitted and the read() of
the cached stream simply hangs if no expected content length has been
specified.

Could be this is a fix to the problem you described.
Grok itself uses zope.publisher 3.4.6 atm.

You could try and patch the z.p 3.4.6 egg along the bugix from 3.5.3 and
  see if that works for you.

Hth,
Michael

- --
http://blog.d2m.at
http://planetzope.org

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKHlpnl0uAvQJUKVYRAsIfAJ9tIv74IMyaPk91hNeY2jYQMpitbQCfSX/f
GwYJVlM8TvWSGpTu/W37mKw=
=8OT0
-----END PGP SIGNATURE-----

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[Martijn Faassen-2]

Michael Haubenwallner wrote:
> You could try and patch the z.p 3.4.6 egg along the bugix from 3.5.3 and
>   see if that works for you.

Another way is to try running with zope.publisher 3.5.3 by putting it in
a custom [versions] in buildout.cfg

[versions]
zope.publisher=3.5.3

I don't know whether that will work though, but it might help with
experimentation.

If you can get it working (or not) please do let us know, and we can
look into this more. Hopefully Michael's suggestion will work and we can
make a zope.publisher 3.4.7 bugfix release for this.

Regards,

Martijn

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[Souheil CHELFOUH]

I tried changing the zope.publisher version to 3.5.x already and got
very strange behavior, local utilities (and more widely, instance
grokkers) were all messed up, in a basic site (app + container)

2009/5/28 Martijn Faassen <faassen@...>:

> Michael Haubenwallner wrote:
>> You could try and patch the z.p 3.4.6 egg along the bugix from 3.5.3 and
>>   see if that works for you.
>
> Another way is to try running with zope.publisher 3.5.3 by putting it in
> a custom [versions] in buildout.cfg
>
> [versions]
> zope.publisher=3.5.3
>
> I don't know whether that will work though, but it might help with
> experimentation.
>
> If you can get it working (or not) please do let us know, and we can
> look into this more. Hopefully Michael's suggestion will work and we can
> make a zope.publisher 3.4.7 bugfix release for this.
>
> Regards,
>
> Martijn
>
> _______________________________________________
> Grok-dev mailing list
> Grok-dev@...
> http://mail.zope.org/mailman/listinfo/grok-dev
>

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[modiadi]

ok,

I will look on how they fixed the EOF thing in 3.5.3 and if&how I can
patch the 3.4.6 egg.

I will make some time for this tonight and let you all know about the
progress.

thanks


-------- Original Message --------
Subject: Re: [Grok-dev] HTTP PUT strange behavior
From: Souheil CHELFOUH <trollfot@...>
Date: Thu, May 28, 2009 4:28 am
To: Martijn Faassen <faassen@...>
Cc: grok-dev@...

I tried changing the zope.publisher version to 3.5.x already and got
very strange behavior, local utilities (and more widely, instance
grokkers) were all messed up, in a basic site (app + container)

2009/5/28 Martijn Faassen <faassen@...>:

> Michael Haubenwallner wrote:
>> You could try and patch the z.p 3.4.6 egg along the bugix from 3.5.3 and
>>   see if that works for you.
>
> Another way is to try running with zope.publisher 3.5.3 by putting it in
> a custom [versions] in buildout.cfg
>
> [versions]
> zope.publisher=3.5.3
>
> I don't know whether that will work though, but it might help with
> experimentation.
>
> If you can get it working (or not) please do let us know, and we can
> look into this more. Hopefully Michael's suggestion will work and we can
> make a zope.publisher 3.4.7 bugfix release for this.
>
> Regards,
>
> Martijn
>
> _

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[Martijn Faassen-2]

Souheil CHELFOUH wrote:
> I tried changing the zope.publisher version to 3.5.x already and got
> very strange behavior, local utilities (and more widely, instance
> grokkers) were all messed up, in a basic site (app + container)

Very strange; I don't see how it would affect either local utilities or
instance grokkers. What does "messed up" mean? Any idea of what the
cause could be?

Regards,

Martijn

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[Souheil CHELFOUH]

I've had a headache trying to find the cause.
Roughly, the Catalog and Intid utilities were not find on a
getUtility, though the instance grokker and the indexes were fired
normally at the application instanciation.
I had this problem severak time. To reproduce it, you only need to
create a simple grok site and to have a grok.Indexes. If you have a
recent version of zope.publisher, you get an error at the application
creation
The index can't be created as the getUtility fails on both ICatalog and IIntids

2009/5/28 Martijn Faassen <faassen@...>:

> Souheil CHELFOUH wrote:
>> I tried changing the zope.publisher version to 3.5.x already and got
>> very strange behavior, local utilities (and more widely, instance
>> grokkers) were all messed up, in a basic site (app + container)
>
> Very strange; I don't see how it would affect either local utilities or
> instance grokkers. What does "messed up" mean? Any idea of what the
> cause could be?
>
> Regards,
>
> Martijn
>
> _______________________________________________
> Grok-dev mailing list
> Grok-dev@...
> http://mail.zope.org/mailman/listinfo/grok-dev
>

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[Jan-Wijbrand Kolman-3]

Souheil CHELFOUH wrote:
> I've had a headache trying to find the cause.
> Roughly, the Catalog and Intid utilities were not find on a
> getUtility, though the instance grokker and the indexes were fired
> normally at the application instanciation.
> I had this problem severak time. To reproduce it, you only need to
> create a simple grok site and to have a grok.Indexes. If you have a
> recent version of zope.publisher, you get an error at the application
> creation
> The index can't be created as the getUtility fails on both ICatalog and IIntids

As similar report a while ago:

  http://permalink.gmane.org/gmane.comp.web.zope.grok.devel/7611

This does not mean I know why this is...


regards,
jw

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[Michael Haubenwallner-2]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

adi@... wrote:
> ok,
>
> I will look on how they fixed the EOF thing in 3.5.3 and if&how I can
> patch the 3.4.6 egg.
>
> I will make some time for this tonight and let you all know about the
> progress.
>

Hi, did patching fix your problem?

I am asking, because in that case i would apply the fix to the z.p 3.4
branch.

Regards
Michael

- --
http://blog.d2m.at
http://planetzope.org

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFKJ3NCl0uAvQJUKVYRAhpgAKCATiVI9QY44s1C7QX0XMs38hhfogCfXP0a
iXqZ8MHUZQ0JuJjfoKXCJPQ=
=TBND
-----END PGP SIGNATURE-----

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev

[modiadi]

Hello,

I am late with the answer, but the fix works.

I copied some changes from 3.5.3 in to 3.4.6 and it works. Here it is  
what I did :

The file is zope.publisher-3.4.6-py2.4.egg/zope/publisher/http.py  and  
the diff looks like this :


204a205
 >         self.size = size and int(size) or -1
207c208
<         self.read()
---
 >         self.read(self.size)
217,225c218,222
<         # Previous versions of Twisted did not support the ``size``  
argument
<         # See http://twistedmatrix.com/trac/ticket/1451
<         #     https://bugs.launchpad.net/zope3/+bug/98284
<         # Note, however, that we cannot pass a size of None to  
cStringIO
<         # objects, or we'll get a TypeError: an integer is required
<         if size is not None:
<             data = self.stream.readline(size)
<         else:
<             data = self.stream.readline()
---
 >         # XXX We should pass the ``size`` argument to  
self.stream.readline
 >         # but twisted.web2.wsgi.InputStream.readline() doesn't  
accept it.
 >         # See http://www.zope.org/Collectors/Zope3-dev/535 and
 >         #     http://twistedmatrix.com/trac/ticket/1451
 >         data = self.stream.readline()
233c230
<
---
 >
835c832
<         if isinstance(t, (ClassType, type)):
---
 >         if isinstance(t, (ClassType, type)):


This is all.

Thank you all for the help and please excuse me for the late fix.

Regars,
Adi





On 4 juin 09, at 09:09, Michael Haubenwallner wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> adi@... wrote:
>> ok,
>>
>> I will look on how they fixed the EOF thing in 3.5.3 and if&how I can
>> patch the 3.4.6 egg.
>>
>> I will make some time for this tonight and let you all know about the
>> progress.
>>
>
> Hi, did patching fix your problem?
>
> I am asking, because in that case i would apply the fix to the z.p 3.4
> branch.
>
> Regards
> Michael
>
> - --
> http://blog.d2m.at
> http://planetzope.org
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.6 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
>
> iD8DBQFKJ3NCl0uAvQJUKVYRAhpgAKCATiVI9QY44s1C7QX0XMs38hhfogCfXP0a
> iXqZ8MHUZQ0JuJjfoKXCJPQ=
> =TBND
> -----END PGP SIGNATURE-----
>

_______________________________________________
Grok-dev mailing list
Grok-dev@...
http://mail.zope.org/mailman/listinfo/grok-dev