Socket close

[Serra, Guido, VF-Group]

Hi All,
I ran out of sockets.

How can I close them before the next test execution?

( before the __call__() methods gets executed again )

I have a plain script recorder by TCPProxy...

Regards,
G.


# Create an HTTPRequest for each request, then replace the
# reference to the HTTPRequest with an instrumented version.
# You can access the unadorned instance using request101.__target__.
request101 = HTTPRequest(url=url0, headers=headers0)
request101 = Test(101, 'GET ping.json').wrap(request101)


class TestRunner:
  """A TestRunner instance is created for each worker thread."""

  # A method for each recorded page.
  def page1(self):
    """GET ping.json (request 101)."""
    result = request101.GET('/services/json/admin/ping.json')

    return result

  def __call__(self):
    """This method is called for every run performed by the worker thread."""
    obj = self.page1()      # GET ping.json (request 101)
    #del(obj)


def instrumentMethod(test, method_name, c=TestRunner):
  """Instrument a method with the given Test."""
  unadorned = getattr(c, method_name)
  import new
  method = new.instancemethod(test.wrap(unadorned), None, c)
  setattr(c, method_name, method)

# Replace each method with an instrumented version.
# You can call the unadorned method using self.page1.__target__().
instrumentMethod(Test(100, 'Page 1'), 'page1')

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Philip Aston-3]

The Grinder automatically closes all the connections at the end of the
__call__ method. You can close a connection explicitly with:

    from net.grinder.plugin.http import HTTPPluginControl

    HTTPPluginControl.getThreadConnection("http://my.server.com").close()

but that really shouldn't be necessary.


Perhaps you need to increase the number of file descriptors on your load
injector or the server you are testing (depending on where the error
message appears).  If you are running on Windows, you may be interested
in http://grinder.sourceforge.net/faq.html#windows-address-in-use and
http://grinder.sourceforge.net/faq.html#ten-cps

- Phil


Guido Serra wrote:

> Hi All,
> I ran out of sockets.
>
> How can I close them before the next test execution?
>
> ( before the __call__() methods gets executed again )
>
> I have a plain script recorder by TCPProxy...
>
> Regards,
> G.
>
>
> # Create an HTTPRequest for each request, then replace the
> # reference to the HTTPRequest with an instrumented version.
> # You can access the unadorned instance using request101.__target__.
> request101 = HTTPRequest(url=url0, headers=headers0)
> request101 = Test(101, 'GET ping.json').wrap(request101)
>
>
> class TestRunner:
>   """A TestRunner instance is created for each worker thread."""
>
>   # A method for each recorded page.
>   def page1(self):
>     """GET ping.json (request 101)."""
>     result = request101.GET('/services/json/admin/ping.json')
>
>     return result
>
>   def __call__(self):
>     """This method is called for every run performed by the worker
> thread."""
>     obj = self.page1()      # GET ping.json (request 101)
>     #del(obj)
>
>
> def instrumentMethod(test, method_name, c=TestRunner):
>   """Instrument a method with the given Test."""
>   unadorned = getattr(c, method_name)
>   import new
>   method = new.instancemethod(test.wrap(unadorned), None, c)
>   setattr(c, method_name, method)
>
> # Replace each method with an instrumented version.
> # You can call the unadorned method using self.page1.__target__().
> instrumentMethod(Test(100, 'Page 1'), 'page1')

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Serra, Guido, VF-Group]

Hi Phil,
I went for this solution...

 def page1(self):
    """GET ping.json (request 101)."""
    result = request101.GET('/services/json/admin/ping.json')
    connectionDefaults.close()

But still... I reach 32k open descriptor... I’m on a SUN Solaris testing server.

The I get a ...

9/14/09 12:32:03 PM (thread 68 run 234 test 101): Aborted run due to Java exception calling TestRunner
Java exception calling TestRunner
        File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py", line 39, in page1
        File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py", line 46, in __call__
Caused by: java.net.BindException: Cannot assign requested address
        at java.net.PlainSocketImpl.socketBind(Native Method)
        at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
        at java.net.Socket.bind(Socket.java:562)
        at java.net.Socket.<init>(Socket.java:364)
        at java.net.Socket.<init>(Socket.java:208)
        at HTTPClient.HTTPConnection.getSocket(HTTPConnection.java:3386)
        at HTTPClient.HTTPConnection.sendRequest(HTTPConnection.java:3082)
        at HTTPClient.HTTPConnection.handleRequest(HTTPConnection.java:2876)
        at HTTPClient.HTTPConnection.setupRequest(HTTPConnection.java:2668)
        at HTTPClient.HTTPConnection.Get(HTTPConnection.java:985)

Regards,
G.


Am 9/14/09 12:50 PM schrieb "Philip Aston" unter <philip.aston@...>:

The Grinder automatically closes all the connections at the end of the
__call__ method. You can close a connection explicitly with:

    from net.grinder.plugin.http import HTTPPluginControl

    HTTPPluginControl.getThreadConnection("http://my.server.com").close()

but that really shouldn't be necessary.


Perhaps you need to increase the number of file descriptors on your load
injector or the server you are testing (depending on where the error
message appears).  If you are running on Windows, you may be interested
in http://grinder.sourceforge.net/faq.html#windows-address-in-use and
http://grinder.sourceforge.net/faq.html#ten-cps

- Phil


Guido Serra wrote:
> Hi All,
> I ran out of sockets.
>
> How can I close them before the next test execution?
>
> ( before the __call__() methods gets executed again )
>
> I have a plain script recorder by TCPProxy...
>
> Regards,
> G.
>
>
> # Create an HTTPRequest for each request, then replace the
> # reference to the HTTPRequest with an instrumented version.
> # You can access the unadorned instance using request101.__target__.
> request101 = HTTPRequest(url=url0, headers=headers0)
> request101 = Test(101, 'GET ping.json').wrap(request101)
>
>
> class TestRunner:
>   """A TestRunner instance is created for each worker thread."""
>
>   # A method for each recorded page.
>   def page1(self):
>     """GET ping.json (request 101)."""
>     result = request101.GET('/services/json/admin/ping.json')
>
>     return result
>
>   def __call__(self):
>     """This method is called for every run performed by the worker
> thread."""
>     obj = self.page1()      # GET ping.json (request 101)
>     #del(obj)
>
>
> def instrumentMethod(test, method_name, c=TestRunner):
>   """Instrument a method with the given Test."""
>   unadorned = getattr(c, method_name)
>   import new
>   method = new.instancemethod(test.wrap(unadorned), None, c)
>   setattr(c, method_name, method)
>
> # Replace each method with an instrumented version.
> # You can call the unadorned method using self.page1.__target__().
> instrumentMethod(Test(100, 'Page 1'), 'page1')


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Serra, Guido, VF-Group]

Aha.... Nice...

  public void close() {
    // A no-op.
  }

>From the src/net/grinder/plugin/http/HTTPPluginConnectionDefaults.java

Shall we provide a fix? :-D

Cheers,
G.


Am 9/14/09 12:50 PM schrieb "Philip Aston" unter <philip.aston@...>:

The Grinder automatically closes all the connections at the end of the
__call__ method. You can close a connection explicitly with:

    from net.grinder.plugin.http import HTTPPluginControl

    HTTPPluginControl.getThreadConnection("http://my.server.com").close()

but that really shouldn't be necessary.


Perhaps you need to increase the number of file descriptors on your load
injector or the server you are testing (depending on where the error
message appears).  If you are running on Windows, you may be interested
in http://grinder.sourceforge.net/faq.html#windows-address-in-use and
http://grinder.sourceforge.net/faq.html#ten-cps

- Phil


Guido Serra wrote:
> Hi All,
> I ran out of sockets.
>
> How can I close them before the next test execution?
>
> ( before the __call__() methods gets executed again )
>
> I have a plain script recorder by TCPProxy...
>
> Regards,
> G.
>
>
> # Create an HTTPRequest for each request, then replace the
> # reference to the HTTPRequest with an instrumented version.
> # You can access the unadorned instance using request101.__target__.
> request101 = HTTPRequest(url=url0, headers=headers0)
> request101 = Test(101, 'GET ping.json').wrap(request101)
>
>
> class TestRunner:
>   """A TestRunner instance is created for each worker thread."""
>
>   # A method for each recorded page.
>   def page1(self):
>     """GET ping.json (request 101)."""
>     result = request101.GET('/services/json/admin/ping.json')
>
>     return result
>
>   def __call__(self):
>     """This method is called for every run performed by the worker
> thread."""
>     obj = self.page1()      # GET ping.json (request 101)
>     #del(obj)
>
>
> def instrumentMethod(test, method_name, c=TestRunner):
>   """Instrument a method with the given Test."""
>   unadorned = getattr(c, method_name)
>   import new
>   method = new.instancemethod(test.wrap(unadorned), None, c)
>   setattr(c, method_name, method)
>
> # Replace each method with an instrumented version.
> # You can call the unadorned method using self.page1.__target__().
> instrumentMethod(Test(100, 'Page 1'), 'page1')


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Philip Aston-3]

No fix required, look in the right place. (HTTPConnectionWrapper.java).

:-)

- Phil


Guido Serra wrote:

> Aha.... Nice...
>
>   public void close() {
>     // A no-op.
>   }
>
> >From the src/net/grinder/plugin/http/HTTPPluginConnectionDefaults.java
>
> Shall we provide a fix? :-D
>
> Cheers,
> G.
>
>
> Am 9/14/09 12:50 PM schrieb "Philip Aston" unter
> <philip.aston@...>:
>
>     The Grinder automatically closes all the connections at the end of the
>     __call__ method. You can close a connection explicitly with:
>
>         from net.grinder.plugin.http import HTTPPluginControl
>
>         HTTPPluginControl.getThreadConnection("http://my.server.com").close(
>     <http://my.server.com%22%29.close%28>)
>
>     but that really shouldn't be necessary.
>
>
>     Perhaps you need to increase the number of file descriptors on
>     your load
>     injector or the server you are testing (depending on where the error
>     message appears).  If you are running on Windows, you may be
>     interested
>     in http://grinder.sourceforge.net/faq.html#windows-address-in-use and
>     http://grinder.sourceforge.net/faq.html#ten-cps
>
>     - Phil
>
>
>     Guido Serra wrote:
>     > Hi All,
>     > I ran out of sockets.
>     >
>     > How can I close them before the next test execution?
>     >
>     > ( before the __call__() methods gets executed again )
>     >
>     > I have a plain script recorder by TCPProxy...
>     >
>     > Regards,
>     > G.
>     >
>     >
>     > # Create an HTTPRequest for each request, then replace the
>     > # reference to the HTTPRequest with an instrumented version.
>     > # You can access the unadorned instance using request101.__target__.
>     > request101 = HTTPRequest(url=url0, headers=headers0)
>     > request101 = Test(101, 'GET ping.json').wrap(request101)
>     >
>     >
>     > class TestRunner:
>     >   """A TestRunner instance is created for each worker thread."""
>     >
>     >   # A method for each recorded page.
>     >   def page1(self):
>     >     """GET ping.json (request 101)."""
>     >     result = request101.GET('/services/json/admin/ping.json')
>     >
>     >     return result
>     >
>     >   def __call__(self):
>     >     """This method is called for every run performed by the worker
>     > thread."""
>     >     obj = self.page1()      # GET ping.json (request 101)
>     >     #del(obj)
>     >
>     >
>     > def instrumentMethod(test, method_name, c=TestRunner):
>     >   """Instrument a method with the given Test."""
>     >   unadorned = getattr(c, method_name)
>     >   import new
>     >   method = new.instancemethod(test.wrap(unadorned), None, c)
>     >   setattr(c, method_name, method)
>     >
>     > # Replace each method with an instrumented version.
>     > # You can call the unadorned method using self.page1.__target__().
>     > instrumentMethod(Test(100, 'Page 1'), 'page1')
>
>
>     ------------------------------------------------------------------------------
>     Let Crystal Reports handle the reporting - Free Crystal Reports
>     2008 30-Day
>     trial. Simplify your report design, integration and deployment -
>     and focus on
>     what you do best, core application coding. Discover what's new with
>     Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>     _______________________________________________
>     grinder-use mailing list
>     grinder-use@...
>     https://lists.sourceforge.net/lists/listinfo/grinder-use
>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Philip Aston-3]

Closing the "connection defaults" doesn't close all connections (perhaps
it should).

Use the code I gave below to find the write connection for your URL and
close that. But it will make little difference, since The Grinder will
close all the connections at the end of the run.

Your test looks quite like a denial of service. You are repeatedly
opening and closing connections as fast as possible. You will have to
tune both the client and the server to achieve this. I suggest you look
at: http://grinder.sourceforge.net/faq.html#linux-address-in-use, and
translate to Solaris.

- Phil

Guido Serra wrote:

> Hi Phil,
> I went for this solution...
>
>  def page1(self):
>     """GET ping.json (request 101)."""
>     result = request101.GET('/services/json/admin/ping.json')
>     connectionDefaults.close()
>
> But still... I reach 32k open descriptor... I’m on a SUN Solaris
> testing server.
>
> The I get a ...
>
> 9/14/09 12:32:03 PM (thread 68 run 234 test 101): Aborted run due to
> Java exception calling TestRunner
> Java exception calling TestRunner
>         File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py",
> line 39, in page1
>         File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py",
> line 46, in __call__
> Caused by: java.net.BindException: Cannot assign requested address
>         at java.net.PlainSocketImpl.socketBind(Native Method)
>         at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
>         at java.net.Socket.bind(Socket.java:562)
>         at java.net.Socket.<init>(Socket.java:364)
>         at java.net.Socket.<init>(Socket.java:208)
>         at HTTPClient.HTTPConnection.getSocket(HTTPConnection.java:3386)
>         at HTTPClient.HTTPConnection.sendRequest(HTTPConnection.java:3082)
>         at
> HTTPClient.HTTPConnection.handleRequest(HTTPConnection.java:2876)
>         at
> HTTPClient.HTTPConnection.setupRequest(HTTPConnection.java:2668)
>         at HTTPClient.HTTPConnection.Get(HTTPConnection.java:985)
>
> Regards,
> G.
>
>
> Am 9/14/09 12:50 PM schrieb "Philip Aston" unter
> <philip.aston@...>:
>
>     The Grinder automatically closes all the connections at the end of the
>     __call__ method. You can close a connection explicitly with:
>
>         from net.grinder.plugin.http import HTTPPluginControl
>
>         HTTPPluginControl.getThreadConnection("http://my.server.com").close(
>     <http://my.server.com%22%29.close%28>)
>
>     but that really shouldn't be necessary.
>
>
>     Perhaps you need to increase the number of file descriptors on
>     your load
>     injector or the server you are testing (depending on where the error
>     message appears).  If you are running on Windows, you may be
>     interested
>     in http://grinder.sourceforge.net/faq.html#windows-address-in-use and
>     http://grinder.sourceforge.net/faq.html#ten-cps
>
>     - Phil
>
>
>     Guido Serra wrote:
>     > Hi All,
>     > I ran out of sockets.
>     >
>     > How can I close them before the next test execution?
>     >
>     > ( before the __call__() methods gets executed again )
>     >
>     > I have a plain script recorder by TCPProxy...
>     >
>     > Regards,
>     > G.
>     >
>     >
>     > # Create an HTTPRequest for each request, then replace the
>     > # reference to the HTTPRequest with an instrumented version.
>     > # You can access the unadorned instance using request101.__target__.
>     > request101 = HTTPRequest(url=url0, headers=headers0)
>     > request101 = Test(101, 'GET ping.json').wrap(request101)
>     >
>     >
>     > class TestRunner:
>     >   """A TestRunner instance is created for each worker thread."""
>     >
>     >   # A method for each recorded page.
>     >   def page1(self):
>     >     """GET ping.json (request 101)."""
>     >     result = request101.GET('/services/json/admin/ping.json')
>     >
>     >     return result
>     >
>     >   def __call__(self):
>     >     """This method is called for every run performed by the worker
>     > thread."""
>     >     obj = self.page1()      # GET ping.json (request 101)
>     >     #del(obj)
>     >
>     >
>     > def instrumentMethod(test, method_name, c=TestRunner):
>     >   """Instrument a method with the given Test."""
>     >   unadorned = getattr(c, method_name)
>     >   import new
>     >   method = new.instancemethod(test.wrap(unadorned), None, c)
>     >   setattr(c, method_name, method)
>     >
>     > # Replace each method with an instrumented version.
>     > # You can call the unadorned method using self.page1.__target__().
>     > instrumentMethod(Test(100, 'Page 1'), 'page1')
>
>
>     ------------------------------------------------------------------------------
>     Let Crystal Reports handle the reporting - Free Crystal Reports
>     2008 30-Day
>     trial. Simplify your report design, integration and deployment -
>     and focus on
>     what you do best, core application coding. Discover what's new with
>     Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>     _______________________________________________
>     grinder-use mailing list
>     grinder-use@...
>     https://lists.sourceforge.net/lists/listinfo/grinder-use
>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Serra, Guido, VF-Group]

Hi Phil,
I need to do a DoS, exactly because I need to measure if a “REST json call” we have in place can handle such a load.

I need java to release the socket datastructure properly. Are u missing anything on the Stream release code?

What’s a run for u ? A plain __call__() method execution of the grinder script, or the overall run?

I need to keep this with 2000tps for at least 1hour... And currently I ran out of sockets in a few seconds.

Regards,
G.


Am 9/14/09 1:34 PM schrieb "Philip Aston" unter <philip.aston@...>:

Closing the "connection defaults" doesn't close all connections (perhaps
it should).

Use the code I gave below to find the write connection for your URL and
close that. But it will make little difference, since The Grinder will
close all the connections at the end of the run.

Your test looks quite like a denial of service. You are repeatedly
opening and closing connections as fast as possible. You will have to
tune both the client and the server to achieve this. I suggest you look
at: http://grinder.sourceforge.net/faq.html#linux-address-in-use, and
translate to Solaris.

- Phil

Guido Serra wrote:
> Hi Phil,
> I went for this solution...
>
>  def page1(self):
>     """GET ping.json (request 101)."""
>     result = request101.GET('/services/json/admin/ping.json')
>     connectionDefaults.close()
>
> But still... I reach 32k open descriptor... I’m on a SUN Solaris
> testing server.
>
> The I get a ...
>
> 9/14/09 12:32:03 PM (thread 68 run 234 test 101): Aborted run due to
> Java exception calling TestRunner
> Java exception calling TestRunner
>         File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py",
> line 39, in page1
>         File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py",
> line 46, in __call__
> Caused by: java.net.BindException: Cannot assign requested address
>         at java.net.PlainSocketImpl.socketBind(Native Method)
>         at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
>         at java.net.Socket.bind(Socket.java:562)
>         at java.net.Socket.<init>(Socket.java:364)
>         at java.net.Socket.<init>(Socket.java:208)
>         at HTTPClient.HTTPConnection.getSocket(HTTPConnection.java:3386)
>         at HTTPClient.HTTPConnection.sendRequest(HTTPConnection.java:3082)
>         at
> HTTPClient.HTTPConnection.handleRequest(HTTPConnection.java:2876)
>         at
> HTTPClient.HTTPConnection.setupRequest(HTTPConnection.java:2668)
>         at HTTPClient.HTTPConnection.Get(HTTPConnection.java:985)
>
> Regards,
> G.
>
>
> Am 9/14/09 12:50 PM schrieb "Philip Aston" unter
> <philip.aston@...>:
>
>     The Grinder automatically closes all the connections at the end of the
>     __call__ method. You can close a connection explicitly with:
>
>         from net.grinder.plugin.http import HTTPPluginControl
>
>         HTTPPluginControl.getThreadConnection("http://my.server.com").close(
>     <http://my.server.com%22%29.close%28>)
>
>     but that really shouldn't be necessary.
>
>
>     Perhaps you need to increase the number of file descriptors on
>     your load
>     injector or the server you are testing (depending on where the error
>     message appears).  If you are running on Windows, you may be
>     interested
>     in http://grinder.sourceforge.net/faq.html#windows-address-in-use and
>     http://grinder.sourceforge.net/faq.html#ten-cps
>
>     - Phil
>
>
>     Guido Serra wrote:
>     > Hi All,
>     > I ran out of sockets.
>     >
>     > How can I close them before the next test execution?
>     >
>     > ( before the __call__() methods gets executed again )
>     >
>     > I have a plain script recorder by TCPProxy...
>     >
>     > Regards,
>     > G.
>     >
>     >
>     > # Create an HTTPRequest for each request, then replace the
>     > # reference to the HTTPRequest with an instrumented version.
>     > # You can access the unadorned instance using request101.__target__.
>     > request101 = HTTPRequest(url=url0, headers=headers0)
>     > request101 = Test(101, 'GET ping.json').wrap(request101)
>     >
>     >
>     > class TestRunner:
>     >   """A TestRunner instance is created for each worker thread."""
>     >
>     >   # A method for each recorded page.
>     >   def page1(self):
>     >     """GET ping.json (request 101)."""
>     >     result = request101.GET('/services/json/admin/ping.json')
>     >
>     >     return result
>     >
>     >   def __call__(self):
>     >     """This method is called for every run performed by the worker
>     > thread."""
>     >     obj = self.page1()      # GET ping.json (request 101)
>     >     #del(obj)
>     >
>     >
>     > def instrumentMethod(test, method_name, c=TestRunner):
>     >   """Instrument a method with the given Test."""
>     >   unadorned = getattr(c, method_name)
>     >   import new
>     >   method = new.instancemethod(test.wrap(unadorned), None, c)
>     >   setattr(c, method_name, method)
>     >
>     > # Replace each method with an instrumented version.
>     > # You can call the unadorned method using self.page1.__target__().
>     > instrumentMethod(Test(100, 'Page 1'), 'page1')
>
>
>     ------------------------------------------------------------------------------
>     Let Crystal Reports handle the reporting - Free Crystal Reports
>     2008 30-Day
>     trial. Simplify your report design, integration and deployment -
>     and focus on
>     what you do best, core application coding. Discover what's new with
>     Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>     _______________________________________________
>     grinder-use mailing list
>     grinder-use@...
>     https://lists.sourceforge.net/lists/listinfo/grinder-use
>


------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Tuvell_Walter]

I can vouch that the solution Phil pointed to (http://grinder.sourceforge.net/faq.html#linux-address-in-use) has worked for me when I've encountered this problem.  I'm working on Linux, and you'll have to translate to Solaris, as already mentioned.  And, since you're really pushing the limits here and "running out of sockets in a few seconds", you'll probably need to tighten down the settings even further (more ports for ip_local_port_range, and fewer seconds for tcp_fin_timeout).

 

 

---

From: Guido Serra [mailto:guido.serra@...]
Sent: Monday, September 14, 2009 7:48 AM
To: grinder-use
Subject: Re: [Grinder-use] Socket close

Hi Phil,
I need to do a DoS, exactly because I need to measure if a “REST json call” we have in place can handle such a load.

I need java to release the socket datastructure properly. Are u missing anything on the Stream release code?

What’s a run for u ? A plain __call__() method execution of the grinder script, or the overall run?

I need to keep this with 2000tps for at least 1hour... And currently I ran out of sockets in a few seconds.

Regards,
G.


Am 9/14/09 1:34 PM schrieb "Philip Aston" unter <philip.aston@...>:

Closing the "connection defaults" doesn't close all connections (perhaps
it should).

Use the code I gave below to find the write connection for your URL and
close that. But it will make little difference, since The Grinder will
close all the connections at the end of the run.

Your test looks quite like a denial of service. You are repeatedly
opening and closing connections as fast as possible. You will have to
tune both the client and the server to achieve this. I suggest you look
at: http://grinder.sourceforge.net/faq.html#linux-address-in-use, and
translate to Solaris.

- Phil

Guido Serra wrote:

> Hi Phil,
> I went for this solution...
>
>  def page1(self):
>     """GET ping.json (request 101)."""
>     result = request101.GET('/services/json/admin/ping.json')
>     connectionDefaults.close()
>
> But still... I reach 32k open descriptor... I’m on a SUN Solaris
> testing server.
>
> The I get a ...
>
> 9/14/09 12:32:03 PM (thread 68 run 234 test 101): Aborted run due to
> Java exception calling TestRunner
> Java exception calling TestRunner
>         File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py",
> line 39, in page1
>         File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py",
> line 46, in __call__
> Caused by: java.net.BindException: Cannot assign requested address
>         at java.net.PlainSocketImpl.socketBind(Native Method)
>         at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
>         at java.net.Socket.bind(Socket.java:562)
>         at java.net.Socket.<init>(Socket.java:364)
>         at java.net.Socket.<init>(Socket.java:208)
>         at HTTPClient.HTTPConnection.getSocket(HTTPConnection.java:3386)
>         at HTTPClient.HTTPConnection.sendRequest(HTTPConnection.java:3082)
>         at
> HTTPClient.HTTPConnection.handleRequest(HTTPConnection.java:2876)
>         at
> HTTPClient.HTTPConnection.setupRequest(HTTPConnection.java:2668)
>         at HTTPClient.HTTPConnection.Get(HTTPConnection.java:985)
>
> Regards,
> G.
>
>
> Am 9/14/09 12:50 PM schrieb "Philip Aston" unter
> <philip.aston@...>:
>
>     The Grinder automatically closes all the connections at the end of the
>     __call__ method. You can close a connection explicitly with:
>
>         from net.grinder.plugin.http import HTTPPluginControl
>
>         HTTPPluginControl.getThreadConnection("http://my.server.com").close(
>     <">http://my.server.com%22%29.close%28>)
>
>     but that really shouldn't be necessary.
>
>
>     Perhaps you need to increase the number of file descriptors on
>     your load
>     injector or the server you are testing (depending on where the error
>     message appears).  If you are running on Windows, you may be
>     interested
>     in http://grinder.sourceforge.net/faq.html#windows-address-in-use and
>     http://grinder.sourceforge.net/faq.html#ten-cps
>
>     - Phil
>
>
>     Guido Serra wrote:
>     > Hi All,
>     > I ran out of sockets.
>     >
>     > How can I close them before the next test execution?
>     >
>     > ( before the __call__() methods gets executed again )
>     >
>     > I have a plain script recorder by TCPProxy...
>     >
>     > Regards,
>     > G.
>     >
>     >
>     > # Create an HTTPRequest for each request, then replace the
>     > # reference to the HTTPRequest with an instrumented version.
>     > # You can access the unadorned instance using request101.__target__.
>     > request101 = HTTPRequest(url=url0, headers=headers0)
>     > request101 = Test(101, 'GET ping.json').wrap(request101)
>     >
>     >
>     > class TestRunner:
>     >   """A TestRunner instance is created for each worker thread."""
>     >
>     >   # A method for each recorded page.
>     >   def page1(self):
>     >     """GET ping.json (request 101)."""
>     >     result = request101.GET('/services/json/admin/ping.json')
>     >
>     >     return result
>     >
>     >   def __call__(self):
>     >     """This method is called for every run performed by the worker
>     > thread."""
>     >     obj = self.page1()      # GET ping.json (request 101)
>     >     #del(obj)
>     >
>     >
>     > def instrumentMethod(test, method_name, c=TestRunner):
>     >   """Instrument a method with the given Test."""
>     >   unadorned = getattr(c, method_name)
>     >   import new
>     >   method = new.instancemethod(test.wrap(unadorned), None, c)
>     >   setattr(c, method_name, method)
>     >
>     > # Replace each method with an instrumented version.
>     > # You can call the unadorned method using self.page1.__target__().
>     > instrumentMethod(Test(100, 'Page 1'), 'page1')
>
>
>     ------------------------------------------------------------------------------
>     Let Crystal Reports handle the reporting - Free Crystal Reports
>     2008 30-Day
>     trial. Simplify your report design, integration and deployment -
>     and focus on
>     what you do best, core application coding. Discover what's new with
>     Crystal Reports now.  http://p.sf.net/sfu/bobj-july
>     _______________________________________________
>     grinder-use mailing list
>     grinder-use@...
>     https://lists.sourceforge.net/lists/listinfo/grinder-use
>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use

[Philip Aston-3]

Guido Serra wrote:
> Hi Phil,
> I need to do a DoS, exactly because I need to measure if a “REST json
> call” we have in place can handle such a load.
>
> I need java to release the socket datastructure properly. Are u
> missing anything on the Stream release code?

I think you'll find Java *is* closing the socket correctly, but the OS
(on both client and server) will not fully release socket resources for
a minute or two due to the FIN wait time out.
>
> What’s a run for u ? A plain __call__() method execution of the
> grinder script, or the overall run?

The former.

>
> I need to keep this with 2000tps for at least 1hour... And currently I
> ran out of sockets in a few seconds.

You will need to tune the OS TCP stacks aggressively to achieve this.

Or you could do this:

    def __call__(self):
      for x in xrange(0, 1000): # 1000 requests per connection
        obj = self.page1()
     
- Phil

> Regards,
> G.
>
>
> Am 9/14/09 1:34 PM schrieb "Philip Aston" unter <philip.aston@...>:
>
>     Closing the "connection defaults" doesn't close all connections
>     (perhaps
>     it should).
>
>     Use the code I gave below to find the write connection for your
>     URL and
>     close that. But it will make little difference, since The Grinder will
>     close all the connections at the end of the run.
>
>     Your test looks quite like a denial of service. You are repeatedly
>     opening and closing connections as fast as possible. You will have to
>     tune both the client and the server to achieve this. I suggest you
>     look
>     at: http://grinder.sourceforge.net/faq.html#linux-address-in-use, and
>     translate to Solaris.
>
>     - Phil
>
>     Guido Serra wrote:
>     > Hi Phil,
>     > I went for this solution...
>     >
>     >  def page1(self):
>     >     """GET ping.json (request 101)."""
>     >     result = request101.GET('/services/json/admin/ping.json')
>     >     connectionDefaults.close()
>     >
>     > But still... I reach 32k open descriptor... I’m on a SUN Solaris
>     > testing server.
>     >
>     > The I get a ...
>     >
>     > 9/14/09 12:32:03 PM (thread 68 run 234 test 101): Aborted run due to
>     > Java exception calling TestRunner
>     > Java exception calling TestRunner
>     >         File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py",
>     > line 39, in page1
>     >         File "/opt/SP/apps/projects/ipcomms.gf2/crius/src/ping.py",
>     > line 46, in __call__
>     > Caused by: java.net.BindException: Cannot assign requested address
>     >         at java.net.PlainSocketImpl.socketBind(Native Method)
>     >         at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
>     >         at java.net.Socket.bind(Socket.java:562)
>     >         at java.net.Socket.<init>(Socket.java:364)
>     >         at java.net.Socket.<init>(Socket.java:208)
>     >         at
>     HTTPClient.HTTPConnection.getSocket(HTTPConnection.java:3386)
>     >         at
>     HTTPClient.HTTPConnection.sendRequest(HTTPConnection.java:3082)
>     >         at
>     > HTTPClient.HTTPConnection.handleRequest(HTTPConnection.java:2876)
>     >         at
>     > HTTPClient.HTTPConnection.setupRequest(HTTPConnection.java:2668)
>     >         at HTTPClient.HTTPConnection.Get(HTTPConnection.java:985)
>     >
>     > Regards,
>     > G.
>     >
>     >
>     > Am 9/14/09 12:50 PM schrieb "Philip Aston" unter
>     > <philip.aston@...>:
>     >
>     >     The Grinder automatically closes all the connections at the
>     end of the
>     >     __call__ method. You can close a connection explicitly with:
>     >
>     >         from net.grinder.plugin.http import HTTPPluginControl
>     >
>     >         HTTPPluginControl.getThreadConnection("http://my.server.com").close
>     <http://my.server.com%22%29.close>(
>     >     <http://my.server.com%22%29.close%28>
>     <http://my.server.com%22%29.close%28%3E>)
>     >
>     >     but that really shouldn't be necessary.
>     >
>     >
>     >     Perhaps you need to increase the number of file descriptors on
>     >     your load
>     >     injector or the server you are testing (depending on where
>     the error
>     >     message appears).  If you are running on Windows, you may be
>     >     interested
>     >     in
>     http://grinder.sourceforge.net/faq.html#windows-address-in-use and
>     >     http://grinder.sourceforge.net/faq.html#ten-cps
>     >
>     >     - Phil
>     >
>     >
>     >     Guido Serra wrote:
>     >     > Hi All,
>     >     > I ran out of sockets.
>     >     >
>     >     > How can I close them before the next test execution?
>     >     >
>     >     > ( before the __call__() methods gets executed again )
>     >     >
>     >     > I have a plain script recorder by TCPProxy...
>     >     >
>     >     > Regards,
>     >     > G.
>     >     >
>     >     >
>     >     > # Create an HTTPRequest for each request, then replace the
>     >     > # reference to the HTTPRequest with an instrumented version.
>     >     > # You can access the unadorned instance using
>     request101.__target__.
>     >     > request101 = HTTPRequest(url=url0, headers=headers0)
>     >     > request101 = Test(101, 'GET ping.json').wrap(request101)
>     >     >
>     >     >
>     >     > class TestRunner:
>     >     >   """A TestRunner instance is created for each worker
>     thread."""
>     >     >
>     >     >   # A method for each recorded page.
>     >     >   def page1(self):
>     >     >     """GET ping.json (request 101)."""
>     >     >     result = request101.GET('/services/json/admin/ping.json')
>     >     >
>     >     >     return result
>     >     >
>     >     >   def __call__(self):
>     >     >     """This method is called for every run performed by the
>     worker
>     >     > thread."""
>     >     >     obj = self.page1()      # GET ping.json (request 101)
>     >     >     #del(obj)
>     >     >
>     >     >
>     >     > def instrumentMethod(test, method_name, c=TestRunner):
>     >     >   """Instrument a method with the given Test."""
>     >     >   unadorned = getattr(c, method_name)
>     >     >   import new
>     >     >   method = new.instancemethod(test.wrap(unadorned), None, c)
>     >     >   setattr(c, method_name, method)
>     >     >
>     >     > # Replace each method with an instrumented version.
>     >     > # You can call the unadorned method using
>     self.page1.__target__().
>     >     > instrumentMethod(Test(100, 'Page 1'), 'page1')
>     >
>     >
>

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
grinder-use mailing list
grinder-use@...
https://lists.sourceforge.net/lists/listinfo/grinder-use