testing behaviour or testing code?

[Pat Maddox]

On 9/3/07, Peter Marklund <peter_marklund@...> wrote:

> > There's a very useful guideline in TDD that says "test YOUR code, not
> > everyone elses." The validation library we're testing here is
> > ActiveRecord's. It's already tested (we hope!).
>
> Personally, I don't have the courage to assume Rails code is always
> working. I know from experience it doesn't always work although it is
> quite solid in general. The Rails code has been tested but not in
> conjunction with my particular apps. I also want to test my
> assumptions of how the Rails API works, maybe it doesn't work as I
> think. Having tests/specs that cover Rails interaction with my app,
> which higher level tests of course naturally do (system/integration
> tests), gives me much more courage to upgrade Rails as well.
>
> Peter

That's a good point.  Having specs in place that demonstrate how you
expect the code to behave will alert when a newer version of Rails
behaves a bit differently.  Granted, in the validates_presence_of
example that probably won't be an issue, but you get the idea.  I
think it was Kevin Clark who said it's a good idea to learn Ruby by
writing specs...then whenever you upgrade Ruby or install new
libraries, your spec suite will make it clear when your assumptions
about the language need to change.

Pat
_______________________________________________
rspec-users mailing list
rspec-users@...
http://rubyforge.org/mailman/listinfo/rspec-users

[Scott Taylor-6]

On Sep 3, 2007, at 3:48 AM, Pat Maddox wrote:

> On 9/3/07, Peter Marklund <peter_marklund@...> wrote:
>>> There's a very useful guideline in TDD that says "test YOUR code,  
>>> not
>>> everyone elses." The validation library we're testing here is
>>> ActiveRecord's. It's already tested (we hope!).
>>
>> Personally, I don't have the courage to assume Rails code is always
>> working. I know from experience it doesn't always work although it is
>> quite solid in general.

I think there are also a lot of leaky abstractions when it comes to  
rails code.  It's tempting to think that rails is just doing a bunch  
of stuff automatically for you (and it is) - but there are edge  
cases, and unless you know *exactly* what rails is doing under the  
cover, testing the behaviour seems to be a good idea.  I've already  
run into one bug in the last week (when doing something rather  
dynamic in a model class) which I wouldn't have expected.

Scott

_______________________________________________
rspec-users mailing list
rspec-users@...
http://rubyforge.org/mailman/listinfo/rspec-users

[David Chelimsky-2]

On 9/3/07, Peter Marklund <peter_marklund@...> wrote:
> > There's a very useful guideline in TDD that says "test YOUR code, not
> > everyone elses." The validation library we're testing here is
> > ActiveRecord's. It's already tested (we hope!).
>
> Personally, I don't have the courage to assume Rails code is always
> working.

The school of thought that says "test your code" addresses this issue
as well - you can have examples that specifically test assumptions in
an API - but then they should be separated from your other examples
(as they are not testing your code). Check out JUnit Recipes by J.B.
Rainsberger.

> I know from experience it doesn't always work although it is
> quite solid in general. The Rails code has been tested but not in
> conjunction with my particular apps. I also want to test my
> assumptions of how the Rails API works, maybe it doesn't work as I
> think.

Again - JB calls these "learning tests."

> Having tests/specs that cover Rails interaction with my app,
> which higher level tests of course naturally do (system/integration
> tests), gives me much more courage to upgrade Rails as well.

Agreed. And Story Runner is the perfect place for these.

Cheers,
David

>
> Peter
>
> _______________________________________________
> rspec-users mailing list
> rspec-users@...
> http://rubyforge.org/mailman/listinfo/rspec-users
>
_______________________________________________
rspec-users mailing list
rspec-users@...
http://rubyforge.org/mailman/listinfo/rspec-users

[Jay Levitt-7]

On 9/2/2007 11:49 PM, David Chelimsky wrote:

> On 9/2/07, Jay Levitt <lists-rspec@...> wrote:
>> On 9/2/2007 12:43 PM, David Chelimsky wrote:
>>> The problem we face is that AR promises huge productivity gains for
>>> the non TDD-er, and challenges the thinking of the die-hard TDD-er.
>>>
>>> I've gone back and forth about whether it's OK to test validations like this:
>>>
>>> it "should validate_presence_of digits" do
>>>   PhoneNumber.expects(:validates_presence_of).with(:digits)
>>>   load "#{RAILS_ROOT}/app/models/phone_number.rb"
>>> end
>>>
>>> On the one hand, it looks immediately like we're testing
>>> implementation. On the other, we're not really - we're mocking a call
>>> to an API. The confusion is that the API is represented in the same
>>> object as the one we're testing (at least its class object). I haven't
>>> really done this in anger yet, but I'm starting to think it's the
>>> right way to go - especially now that we have Story Runner to cover
>>> things end to end. WDYT of this approach?
>> Personally, I don't much like it.  It feels too much like:
>>
>> it "should validate_presence_of digits" do
>>    my_model.line(7).should_read "validates_presence_of :digits"
>> end
> A couple of things to consider:
>
> There's a very useful guideline in TDD that says "test YOUR code, not
> everyone elses." The validation library we're testing here is
> ActiveRecord's. It's already tested (we hope!).

Right... and I'm not testing that ActiveRecord's validation works.  I'm
testing that my model works as I expect it to work.

For instance, in your example, you just verify that you call
validates_presence_of with fieldname :digits.  You're not verifying that
that's the right thing to do, or that it behaves the way you expect it to.

Also, I think this conflicts with "test behavior, not implementation".
All I care about is the behavior of the model; I don't care if it calls
validates_presence_of, or if it calls acts_as_phone_number.


> Also - there's a difference between the behaviour of a system and the
> behaviour of an object. The system's job is to validate that the phone
> number is all digits. So it makes sense to have examples like that in
> high level examples using Story Runner, rails integration tests, or an
> in-browser suite like Selenium or Watir.

Ah, but (as Pat pointed out) in Rails, validations are, in fact, the job
of the model.  They may be done with validates_* "declarations", or with
custom code, or with plugins.

> This model object's job is to make sure the input gets validated, not
> to actually validate it. If the model made a more OO-feeling call out
> to a message library - something like this:
>
> class PhoneNumber
>   def validators
>     @validators ||= []
>   end
>
>   def add_validator (validator)
>     validators << validator
>   end
>
>   def validate(input)
>     validators.each {|v| v.validate (input)}
>   end
> end
>
> Then submitting mock validators via add_validator and setting mock
> expectations that they get called would be totally par for the course.

Yeah, and I guess I still haven't swallowed that part of mocking -
because, again, it's brittle and tied to implementation.  I have no
problem mocking out ActiveRecord, because that's a major part of any
Rails app and it's a given that you'll be using it in a certain way.
Ditto for any other major library.  But validations are so simplistic
that you might write a given validation in five different ways, and
specifying -which- of those five ways the code should use just feels wrong.

And even for AR, as someone pointed out - what if I want to use .new or
.build instead of .create, or .update_attributes instead of the setter
function?

The ideal answer for that is to build a more sophisticated AR mock that
lets you write expectations that work in any of those cases.  I want to
know that User.register.should do_something_that_creates_a_new_record,
not that it explicitly called .create.

And interestingly, in the case of .update_attributes vs. direct
assignment, it seems to me that the proper way to "test the behavior,
not the implementation" is to check the value of the field after the
fact - which of course apparently conflicts with "test behavior, not
state".  But, when the behavior IS to set a certain state, I feel like
it's OK.

Jay

_______________________________________________
rspec-users mailing list
rspec-users@...
http://rubyforge.org/mailman/listinfo/rspec-users

[Wincent Colaiuta]

El 7/9/2007, a las 5:36, Jay Levitt escribió:

>> There's a very useful guideline in TDD that says "test YOUR code, not
>> everyone elses." The validation library we're testing here is
>> ActiveRecord's. It's already tested (we hope!).
>
> Right... and I'm not testing that ActiveRecord's validation works.  
> I'm
> testing that my model works as I expect it to work.
>
> For instance, in your example, you just verify that you call
> validates_presence_of with fieldname :digits.  You're not verifying  
> that
> that's the right thing to do, or that it behaves the way you expect  
> it to.
>
> Also, I think this conflicts with "test behavior, not implementation".
> All I care about is the behavior of the model; I don't care if it  
> calls
> validates_presence_of, or if it calls acts_as_phone_number.

Very true that you shouldn't be testing ActiveRecord's validation  
(Rails' own unit tests are there for that).

But if you want to do truly *driven* BDD then you will have to test  
something; in other words, *before* you go ahead and add this line to  
your model:

   validates_presence_of :foo

You need to write a failing spec for it first. Otherwise, why would  
you write it? Doing BDD in its purest form you shouldn't be writing  
*any* line of code without your specs driving it. This means the  
familiar "write failing spec, write code, confirm working spec" cycle.

So the question is, what is the best kind of spec to write to *drive*  
the writing of your "validates_presence_of" lines? For some  
validations it's quite easy. For others it is less straightforward.  
There are probably multiple valid (or valid-ish) answers, but it's  
sometimes difficult to know which one is best.

Cheers,
Wincent

_______________________________________________
rspec-users mailing list
rspec-users@...
http://rubyforge.org/mailman/listinfo/rspec-users

[David Richards-3]

On Sep 7, 2007, at 8:24 AM, Wincent Colaiuta wrote:

> El 7/9/2007, a las 5:36, Jay Levitt escribió:
>
>>> There's a very useful guideline in TDD that says "test YOUR code,  
>>> not
>>> everyone elses." The validation library we're testing here is
>>> ActiveRecord's. It's already tested (we hope!).
>>
>> Right... and I'm not testing that ActiveRecord's validation works.
>> I'm
>> testing that my model works as I expect it to work.
>>
>> For instance, in your example, you just verify that you call
>> validates_presence_of with fieldname :digits.  You're not verifying
>> that
>> that's the right thing to do, or that it behaves the way you expect
>> it to.
>>
>> Also, I think this conflicts with "test behavior, not  
>> implementation".
>> All I care about is the behavior of the model; I don't care if it
>> calls
>> validates_presence_of, or if it calls acts_as_phone_number.
>
> Very true that you shouldn't be testing ActiveRecord's validation
> (Rails' own unit tests are there for that).
>
> But if you want to do truly *driven* BDD then you will have to test
> something; in other words, *before* you go ahead and add this line to
> your model:
>
>    validates_presence_of :foo
>
> You need to write a failing spec for it first. Otherwise, why would
> you write it? Doing BDD in its purest form you shouldn't be writing
> *any* line of code without your specs driving it. This means the
> familiar "write failing spec, write code, confirm working spec" cycle.
>
> So the question is, what is the best kind of spec to write to *drive*
> the writing of your "validates_presence_of" lines? For some
> validations it's quite easy. For others it is less straightforward.
> There are probably multiple valid (or valid-ish) answers, but it's
> sometimes difficult to know which one is best.
>

I've kind of adopted this plain-vanilla approach:

http://snippets.dzone.com/posts/show/4508

You'll see I'm a little fat, especially with my polymorphic  
association.  Maybe they can be improved, but it's been the right  
balance of testing before I write code for me--I gain confidence and  
momentum, and my later iterations are constrained enough to keep me  
from stepping out of bounds.  I'm sure my ideas will evolve over time.


> Cheers,
> Wincent
>
> _______________________________________________
> rspec-users mailing list
> rspec-users@...
> http://rubyforge.org/mailman/listinfo/rspec-users

_______________________________________________
rspec-users mailing list
rspec-users@...
http://rubyforge.org/mailman/listinfo/rspec-users

[Jay Levitt-7]

Wincent Colaiuta wrote:

> El 7/9/2007, a las 5:36, Jay Levitt escribió:
>
> Very true that you shouldn't be testing ActiveRecord's validation  
> (Rails' own unit tests are there for that).
>
> But if you want to do truly *driven* BDD then you will have to test  
> something; in other words, *before* you go ahead and add this line to  
> your model:
>
>    validates_presence_of :foo
>
> You need to write a failing spec for it first. Otherwise, why would  
> you write it? Doing BDD in its purest form you shouldn't be writing  
> *any* line of code without your specs driving it. This means the  
> familiar "write failing spec, write code, confirm working spec" cycle.
>
> So the question is, what is the best kind of spec to write to *drive*  
> the writing of your "validates_presence_of" lines? For some  
> validations it's quite easy. For others it is less straightforward.  
> There are probably multiple valid (or valid-ish) answers, but it's  
> sometimes difficult to know which one is best.

Well put!  To me, if the spec I write is:

   Model.expects(:validates_presence_of).with(:digits)

then I haven't written a spec at all - I've written the code I plan to
write, and spelled it differently!  The English version of that spec is:

Model
   - should call validates_presence_of with parameter :digits

That's just specifying what a line of my code should *say*, not how
Model should *behave*.

I really like Wincent's approach - test that valid input yields a valid
response and that invalid input yields an invalid response.

Jay

_______________________________________________
rspec-users mailing list
rspec-users@...
http://rubyforge.org/mailman/listinfo/rspec-users