[ANN] Experimental testing plugin

[Peter Ledbrook-2]

Hi everyone,

I have been working on some improvements to the Grails testing
framework recently which will go into Grails 1.1. Some of the support
classes though will work with Grails 1.0.x, so I have packaged them up
as a plugin. You can install it using:

  grails install-plugin testing

Before you rush to download it, please be warned that it is nowhere
near a complete implementation yet, particularly
ControllerUnitTestCase. However, I would like to solicit feedback
early on so that we can really nail the problems that people are
having in testing. And if you can supply patches, great!

So, what lurks in the plugin? I'm planning to get some documentation
up soon, but for now I'll cover some common use cases. Before that,
the current set of classes are designed to be used in unit tests. As a
general trend, we want to encourage people to write unit tests in
preference to integration tests. We also want to make sure that unit
tests can be run from within an IDE, i.e. there should be no
requirement on a running Grails instance.

For the first example I'll show you how to test domain constraints.
Domain classes often lack logic, and so they don't get tested.
However, plenty of errors can creep in to the constraints so it's
worth validating them.

  class MyDomain {
      String name
      Integer age

      static constraints = {
          name(nullable: false, blank: false)
          age(nullable: false, min: 10, max: 100)
      }
  }

  class MyDomainUnitTests extends GrailsUnitTestCase {
      void testConstraints() {
          // Mock the validate() method.
          registerMetaClass(MyDomain)
          MockUtils.prepareForConstraintsTests(MyDomain)

          // Test that a fresh new domain instance fails validation on
          // the "nullable: false" constraints.
          def testInstance = new MyDomain()
          def errors = testInstance.validate()
          assertEquals 2, errors.size()
          assertEquals "nullable", errors["name"]
          assertEquals "nullable", errors["age"]

          // Test the other constraints
          testInstance = new MyDomain(name: "  ", age: 5)
          errors = testInstance.validate()
          assertEquals 2, errors.size()
          assertEquals "blank", errors["name"]
          assertEquals "min", errors["age"]
      }
}

The main things to note here are:

1. We sub-class GrailsUnitTestCase
2. "registerMetaClass()" and "MockUtils.prepare...()" add the
validate() method to the domain class
3. We create instances of the domain class, call validate(), and check
whether any errors were found

On (2), the two lines will be replaced by a method on
GrailsUnitTestCase in the near future. On (3), the validate() method
returns a map of validation errors. Note that we check for the name of
the constraint, not the i18n error code associated with the
constraint.

That's it for domain constraints. For other unit tests, such as for
services and controllers, GrailsUnitTestCase provides the method
"mockDomain(Class, List)":

  void testMethod() {
      mockDomain(MyDomain, [
              new MyDomain(name: "John Smith", age: 35),
              new MyDomain(name: "Alice Smith", age: 64),
              new MyDomain(name: "Irene Pane", age: 22),
              new MyDomain(name: "Patrick Rose", age: 45) ])

      def testService = new MyService()
      testService.doSomething()
      ...
  }

The method injects working versions of the dynamic methods and
properties that normally go with domain classes, in particular the
dynamic finders. Where appropriate, these injected methods/properties
use the given list of domain instances as a source of data. For
example, if MyService.doSomething called a dynamic finder like this:

  MyDomain.findByNameLike("%Smith")

the mock property would return a list containing the "John Smith" and
"Alice Smith" MyDomain instances in that order. Another useful method
is "mockFor()" which returns an object that you can use pretty much
like the Groovy MockFor class:

  def mockControl = mockFor(MyDomain)
  mockControl.demand.save(1..1) {-> return true}
  mockControl.demand.static.findByName(1..1) { name -> return [] }

The best thing about this method is that it works seemlessly with
"mockDomain()", i.e. you can readily override the methods provided by
"mockDomain()" via the mock object returned by "mockFor()".

Finally, there is a ControllerUnitTestCase, but it is in the very
early stages of development. I recommend you only use it if you're
willing to patch it up with the functionality you need. It will
automatically inject all the normal controller properties and methods,
but not much else. However, one nice feature I have implemented
already is the ability to set the body of the request to some XML
(either a string or builder markup), particularly useful for REST
controllers based on XML. I need to add support for JSON too.

That's it for now. I would certainly give GrailsUnitTestCase a go
because I have already found it much easier to write Grails unit tests
than I used to. If you want to raise issues or provide patches, please
add them to the main Grails JIRA, setting the fix version to 1.1 and
assigning them to me (username "pledbrook").

Cheers,

Peter

--
Software Engineer
G2One, Inc.
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Robert Fletcher]

That's fantastic news. We've got some hand rolled stuff for mocking up
the Grails 'magic' on controllers & taglibs in unit tests which has
been helpful but is quite ad-hoc. Domain classes & services we tend to
test using integration tests as we want to ensure the persistence
layer is working correctly as much as anything. I'll look forward to
giving this a try.

Cheers,
Rob

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jay Guidos]

Sounds great Peter!

I can already see the use of this, as I have had to crib together
working domain classes to get service unit tests to run in the past, and
the mockDomain closure is a more complete and comprehensive version of
what I already had started.  I am going on holidays but I will download
and try your plugin when I get back.

Well done,

Jay Guidos

-----Original Message-----
From: Peter Ledbrook [mailto:peter@...]
Sent: Friday, July 18, 2008 8:43 AM
To: user@...
Subject: [grails-user] [ANN] Experimental testing plugin

Hi everyone,

I have been working on some improvements to the Grails testing
framework recently which will go into Grails 1.1. Some of the support
classes though will work with Grails 1.0.x, so I have packaged them up
as a plugin. You can install it using:

  grails install-plugin testing

Before you rush to download it, please be warned that it is nowhere
near a complete implementation yet, particularly
ControllerUnitTestCase. However, I would like to solicit feedback
early on so that we can really nail the problems that people are
having in testing. And if you can supply patches, great!

So, what lurks in the plugin? I'm planning to get some documentation
up soon, but for now I'll cover some common use cases. Before that,
the current set of classes are designed to be used in unit tests. As a
general trend, we want to encourage people to write unit tests in
preference to integration tests. We also want to make sure that unit
tests can be run from within an IDE, i.e. there should be no
requirement on a running Grails instance.

For the first example I'll show you how to test domain constraints.
Domain classes often lack logic, and so they don't get tested.
However, plenty of errors can creep in to the constraints so it's
worth validating them.

  class MyDomain {
      String name
      Integer age

      static constraints = {
          name(nullable: false, blank: false)
          age(nullable: false, min: 10, max: 100)
      }
  }

  class MyDomainUnitTests extends GrailsUnitTestCase {
      void testConstraints() {
          // Mock the validate() method.
          registerMetaClass(MyDomain)
          MockUtils.prepareForConstraintsTests(MyDomain)

          // Test that a fresh new domain instance fails validation on
          // the "nullable: false" constraints.
          def testInstance = new MyDomain()
          def errors = testInstance.validate()
          assertEquals 2, errors.size()
          assertEquals "nullable", errors["name"]
          assertEquals "nullable", errors["age"]

          // Test the other constraints
          testInstance = new MyDomain(name: "  ", age: 5)
          errors = testInstance.validate()
          assertEquals 2, errors.size()
          assertEquals "blank", errors["name"]
          assertEquals "min", errors["age"]
      }
}

The main things to note here are:

1. We sub-class GrailsUnitTestCase
2. "registerMetaClass()" and "MockUtils.prepare...()" add the
validate() method to the domain class
3. We create instances of the domain class, call validate(), and check
whether any errors were found

On (2), the two lines will be replaced by a method on
GrailsUnitTestCase in the near future. On (3), the validate() method
returns a map of validation errors. Note that we check for the name of
the constraint, not the i18n error code associated with the
constraint.

That's it for domain constraints. For other unit tests, such as for
services and controllers, GrailsUnitTestCase provides the method
"mockDomain(Class, List)":

  void testMethod() {
      mockDomain(MyDomain, [
              new MyDomain(name: "John Smith", age: 35),
              new MyDomain(name: "Alice Smith", age: 64),
              new MyDomain(name: "Irene Pane", age: 22),
              new MyDomain(name: "Patrick Rose", age: 45) ])

      def testService = new MyService()
      testService.doSomething()
      ...
  }

The method injects working versions of the dynamic methods and
properties that normally go with domain classes, in particular the
dynamic finders. Where appropriate, these injected methods/properties
use the given list of domain instances as a source of data. For
example, if MyService.doSomething called a dynamic finder like this:

  MyDomain.findByNameLike("%Smith")

the mock property would return a list containing the "John Smith" and
"Alice Smith" MyDomain instances in that order. Another useful method
is "mockFor()" which returns an object that you can use pretty much
like the Groovy MockFor class:

  def mockControl = mockFor(MyDomain)
  mockControl.demand.save(1..1) {-> return true}
  mockControl.demand.static.findByName(1..1) { name -> return [] }

The best thing about this method is that it works seemlessly with
"mockDomain()", i.e. you can readily override the methods provided by
"mockDomain()" via the mock object returned by "mockFor()".

Finally, there is a ControllerUnitTestCase, but it is in the very
early stages of development. I recommend you only use it if you're
willing to patch it up with the functionality you need. It will
automatically inject all the normal controller properties and methods,
but not much else. However, one nice feature I have implemented
already is the ability to set the body of the request to some XML
(either a string or builder markup), particularly useful for REST
controllers based on XML. I need to add support for JSON too.

That's it for now. I would certainly give GrailsUnitTestCase a go
because I have already found it much easier to write Grails unit tests
than I used to. If you want to raise issues or provide patches, please
add them to the main Grails JIRA, setting the fix version to 1.1 and
assigning them to me (username "pledbrook").

Cheers,

Peter

--
Software Engineer
G2One, Inc.
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Miller, Johnny]

Peter,
 
Thanks lots for that. We will definitely have a look at it and let you know how we get on.
 
Johnny

________________________________

From: Peter Ledbrook [mailto:peter@...]
Sent: Fri 18/07/2008 15:42
To: user@...
Subject: [grails-user] [ANN] Experimental testing plugin



Hi everyone,

I have been working on some improvements to the Grails testing
framework recently which will go into Grails 1.1. Some of the support
classes though will work with Grails 1.0.x, so I have packaged them up
as a plugin. You can install it using:

  grails install-plugin testing

Before you rush to download it, please be warned that it is nowhere
near a complete implementation yet, particularly
ControllerUnitTestCase. However, I would like to solicit feedback
early on so that we can really nail the problems that people are
having in testing. And if you can supply patches, great!

So, what lurks in the plugin? I'm planning to get some documentation
up soon, but for now I'll cover some common use cases. Before that,
the current set of classes are designed to be used in unit tests. As a
general trend, we want to encourage people to write unit tests in
preference to integration tests. We also want to make sure that unit
tests can be run from within an IDE, i.e. there should be no
requirement on a running Grails instance.

For the first example I'll show you how to test domain constraints.
Domain classes often lack logic, and so they don't get tested.
However, plenty of errors can creep in to the constraints so it's
worth validating them.

  class MyDomain {
      String name
      Integer age

      static constraints = {
          name(nullable: false, blank: false)
          age(nullable: false, min: 10, max: 100)
      }
  }

  class MyDomainUnitTests extends GrailsUnitTestCase {
      void testConstraints() {
          // Mock the validate() method.
          registerMetaClass(MyDomain)
          MockUtils.prepareForConstraintsTests(MyDomain)

          // Test that a fresh new domain instance fails validation on
          // the "nullable: false" constraints.
          def testInstance = new MyDomain()
          def errors = testInstance.validate()
          assertEquals 2, errors.size()
          assertEquals "nullable", errors["name"]
          assertEquals "nullable", errors["age"]

          // Test the other constraints
          testInstance = new MyDomain(name: "  ", age: 5)
          errors = testInstance.validate()
          assertEquals 2, errors.size()
          assertEquals "blank", errors["name"]
          assertEquals "min", errors["age"]
      }
}

The main things to note here are:

1. We sub-class GrailsUnitTestCase
2. "registerMetaClass()" and "MockUtils.prepare...()" add the
validate() method to the domain class
3. We create instances of the domain class, call validate(), and check
whether any errors were found

On (2), the two lines will be replaced by a method on
GrailsUnitTestCase in the near future. On (3), the validate() method
returns a map of validation errors. Note that we check for the name of
the constraint, not the i18n error code associated with the
constraint.

That's it for domain constraints. For other unit tests, such as for
services and controllers, GrailsUnitTestCase provides the method
"mockDomain(Class, List)":

  void testMethod() {
      mockDomain(MyDomain, [
              new MyDomain(name: "John Smith", age: 35),
              new MyDomain(name: "Alice Smith", age: 64),
              new MyDomain(name: "Irene Pane", age: 22),
              new MyDomain(name: "Patrick Rose", age: 45) ])

      def testService = new MyService()
      testService.doSomething()
      ...
  }

The method injects working versions of the dynamic methods and
properties that normally go with domain classes, in particular the
dynamic finders. Where appropriate, these injected methods/properties
use the given list of domain instances as a source of data. For
example, if MyService.doSomething called a dynamic finder like this:

  MyDomain.findByNameLike("%Smith")

the mock property would return a list containing the "John Smith" and
"Alice Smith" MyDomain instances in that order. Another useful method
is "mockFor()" which returns an object that you can use pretty much
like the Groovy MockFor class:

  def mockControl = mockFor(MyDomain)
  mockControl.demand.save(1..1) {-> return true}
  mockControl.demand.static.findByName(1..1) { name -> return [] }

The best thing about this method is that it works seemlessly with
"mockDomain()", i.e. you can readily override the methods provided by
"mockDomain()" via the mock object returned by "mockFor()".

Finally, there is a ControllerUnitTestCase, but it is in the very
early stages of development. I recommend you only use it if you're
willing to patch it up with the functionality you need. It will
automatically inject all the normal controller properties and methods,
but not much else. However, one nice feature I have implemented
already is the ability to set the body of the request to some XML
(either a string or builder markup), particularly useful for REST
controllers based on XML. I need to add support for JSON too.

That's it for now. I would certainly give GrailsUnitTestCase a go
because I have already found it much easier to write Grails unit tests
than I used to. If you want to raise issues or provide patches, please
add them to the main Grails JIRA, setting the fix version to 1.1 and
assigning them to me (username "pledbrook").

Cheers,

Peter

--
Software Engineer
G2One, Inc.
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email







-----------------------------------------
Information in this email including any attachments may be
privileged, confidential and is intended exclusively for the
addressee. The views expressed may not be official policy, but the
personal views of the originator. If you have received it in error,
please notify the sender by return e-mail and delete it from your
system. You should not reproduce, distribute, store, retransmit,
use or disclose its contents to anyone.

Please note we reserve the right to monitor all e-mail
communication through our internal and external networks.

SKY and the SKY marks are trade marks of British Sky Broadcasting
Group plc and are used under licence. British Sky Broadcasting
Limited (Registration No. 2906991), Sky Interactive Limited
(Registration No. 3554332), Sky-In-Home Service Limited
(Registration No. 2067075) and Sky Subscribers Services Limited
(Registration No. 2340150) are direct or indirect subsidiaries of
British Sky Broadcasting Group plc (Registration No. 2247735). All
of the companies mentioned in this paragraph are incorporated in
England and Wales and share the same registered office at Grant
Way, Isleworth, Middlesex TW7 5QD.  


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Robert Fletcher]

I'm planning to start having a play with this new plugin this week.
One simple thing that occurred to me; currently 'grails create-*'
creates an integration test as well as whatever artefact. Can the
plugin override this to make creation of a unit test the default? With
improved support for Grails 'magic' in unit tests it makes for a much
better default choice IMO.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Peter Ledbrook-2]

> I'm planning to start having a play with this new plugin this week.
> One simple thing that occurred to me; currently 'grails create-*'
> creates an integration test as well as whatever artefact. Can the
> plugin override this to make creation of a unit test the default? With
> improved support for Grails 'magic' in unit tests it makes for a much
> better default choice IMO.

I don't think this is possible with a plugin without providing its own
script. Definitely worth doing for 1.1 though. Could you raise an
issue please?

Thanks,

Peter

--
Software Engineer
G2One, Inc.
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Robert Fletcher]

Done: http://jira.codehaus.org/browse/GRAILS-3286

Cheers, Rob

On Tue, Jul 29, 2008 at 9:27 AM, Peter Ledbrook <peter@...> wrote:
>
> I don't think this is possible with a plugin without providing its own
> script. Definitely worth doing for 1.1 though. Could you raise an
> issue please?
>
> Thanks,
>
> Peter
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Robert Fletcher]

Peter,

A couple of us have been playing with the testing plugin this
afternoon. We've converted a couple of our controller unit tests that
were using our own equivalent of what goes on in
ControllerUnitTestCase.setUp. The plugin is really nice so far. I'm
especially impressed with the mockDomain behaviour - that trims down a
lot of really ugly code.

The only flaw I've spotted so far is that the mock logger attached to
the controller is not handling the 2 arg form of the various logging
methods, e.g. error(String, Throwable).

Cheers,
Rob

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Peter Ledbrook-2]

> The only flaw I've spotted so far is that the mock logger attached to
> the controller is not handling the 2 arg form of the various logging
> methods, e.g. error(String, Throwable).

Oops - very good point! That should be easy enough to change.

Thanks,

Peter

--
Software Engineer
G2One, Inc.
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Robert Fletcher]

Another quick one... mocked domain classes don't have a getAll(ids) method.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Peter Ledbrook-2]

> Another quick one... mocked domain classes don't have a getAll(ids) method.

Wow, I didn't even know there was a getAll() method :) Thanks for the feedback!

Cheers,

Peter

--
Software Engineer
G2One, Inc.
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email