test isolation, grok droppings

[Reinout van Rees]

A couple of things together got me a test error yesterday when coding with
jkolman.  I'll mention it here in case someone else gets bitten.

a) I added a default allow_teardown=True option to z3c.testsetup when running
zcml layer tests.  zope.app.testing's default is not to take any risks and to
run zcml layers in a separate python process.  A brute-force way of saying "I
don't trust the component architecture to tear itself down completely".  In
practice this teardown is no problem.  The separate process means the profiler
and code coverage lose part of their data: 20% coverage for a 98% covered app
is the result, for instance.

b) The only possible problem with not running in a separate process is when
something changes modules/classes (once imported) at runtime.

c) Grokking modules and classes leaves droppings.  Normally this isn't a
problem.

d) Long-standing grok problem: having a view in one module subclass from a
view in another one.  Grok cannot reliably see that the superclass has a
grok.template(), so it requires you to re-specify it.  At runtime, there's
actually not a problem as regular python inheritance picks up the template
just fine.

e) Solution for (d): just add a bare render() method.  Grok doesn't complain
at grok time about a missing template or render method.  And when actually
rendering the view, the template takes precedence.


Combine a till e.  Use a second test layer so that grokking happens a second
time.  Grok groks the module with the bare-render-hack a second time and sees,
from the first run, that there's a template dropping.  ERROR: both a template
and a render method!


I probably still missed some requirements for this bug to surface (like
probably alphabetic order of the module names).


Weird one :-)


Reinout

--
Reinout van Rees - reinout@... - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets"

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Jonathan Ballet-5]

Hi Reinout,

On Thu, Oct 1, 2009 at 3:24 PM, Reinout van Rees <reinout@...> wrote:
[...]
> I probably still missed some requirements for this bug to surface (like
> probably alphabetic order of the module names).

We found some weird bugs with the new release of z3c.testsetup too,
but I didn't took the time to actually try to isolate the problem.
In most cases, everything is OK, but launching our big tests suite
throw some ForbiddenAttribute errors and other things we didn't have
before with <0.5 (and launching those failing tests one by one doesn't
show the problem).

I hope to have more time to see what's happening, but currently, we
stick we our internal release (0.4 + my patch which have been
integrated in 0.5).

Cheers,

 Jonathan
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Reinout van Rees]

On 2009-10-01, Jonathan Ballet <jonathan.ballet@...> wrote:

> Hi Reinout,
>
> On Thu, Oct 1, 2009 at 3:24 PM, Reinout van Rees <reinout@...> wrote:
> [...]
>> I probably still missed some requirements for this bug to surface (like
>> probably alphabetic order of the module names).
>
> We found some weird bugs with the new release of z3c.testsetup too,
> but I didn't took the time to actually try to isolate the problem.
> In most cases, everything is OK, but launching our big tests suite
> throw some ForbiddenAttribute errors and other things we didn't have
> before with <0.5 (and launching those failing tests one by one doesn't
> show the problem).
>
> I hope to have more time to see what's happening, but currently, we
> stick we our internal release (0.4 + my patch which have been
> integrated in 0.5).

One really quick way of verifying if it has to do with the teardown: just add
an 'allow_teardown=False' to your register_all_tests() call.


Reinout

--
Reinout van Rees - reinout@... - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets"

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Jonathan Ballet-5]

On Wed, Oct 7, 2009 at 1:26 PM, Reinout van Rees <reinout@...> wrote:
> One really quick way of verifying if it has to do with the teardown: just add
> an 'allow_teardown=False' to your register_all_tests() call.

I did that, and effectively, using 'allow_teardown=False' make my
tests pass again.
So, it looks like this is definitively a problem with those tear downs...

Cheers,

 Jonathan
_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Reinout van Rees]

On 2009-10-08, Jonathan Ballet <jonathan.ballet@...> wrote:
> On Wed, Oct 7, 2009 at 1:26 PM, Reinout van Rees <reinout@...> wrote:
>> One really quick way of verifying if it has to do with the teardown: just add
>> an 'allow_teardown=False' to your register_all_tests() call.
>
> I did that, and effectively, using 'allow_teardown=False' make my
> tests pass again.
> So, it looks like this is definitively a problem with those tear downs...

I'm leaning towards setting allow_teardown's to its (zope.app.testing's
default) default of False again.  At least the brute-force separate process
hides the incomplete test teardowns (bugs?) that way.  Less surprise that way,
I sadly guess.

Ideally, the test's teardowns would be better, of course :-)


Reinout

--
Reinout van Rees - reinout@... - http://reinout.vanrees.org
Software developer at http://www.thehealthagency.com
"Military engineers build missiles. Civil engineers build targets"

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev

[Jan-Wijbrand Kolman-3]

Reinout van Rees wrote:
> I'm leaning towards setting allow_teardown's to its (zope.app.testing's
> default) default of False again.

I would agree here.

The IReRaiseException adapter registration that Grok __used__ to make,
that suddenly would influence tests behaviour, leading to confusion.

To me this seems somewhat similar, in that z3c.testsetup will by default
reverse default testing behaviour. And that can be very surprising to a
developer.

+1

regards,
jw

_______________________________________________
Grok-dev mailing list
Grok-dev@...
https://mail.zope.org/mailman/listinfo/grok-dev