Are you really using JUnit during your development?

[J. B. Rainsberger]

Cédric Beust ♔ wrote:

> In my opinion, the slowness of database tests is vastly exaggerated.
>
> Don't over-estimate the time taken to contact a localhost database until you
> actually start watching the clock tick while your tests run.

I'm sure you meant this as general advice, because I've certainly
measured the speed of database client tests in the course of my work.

 > And in my
> experience, it takes a lot of databases, tables and complex queries to get
> there.

Perhaps Moore's Law has changed the balance, but when you consider my
goal is 100 tests/second, you can be sure the database client tests
don't cut the mustard. Even without the arbitrary speed goal, there is a
good reason to avoid testing against the database.

It's just too easy when testing against the database to write more tests
than is necessary against the database. Rather than 20-30 such tests,
often we end up with hundreds, and that's not just idiot practitioners
doing it. I've seen otherwise intelligent, thoughtful people fall into
this trap. This is one of the reasons I advise against it.

> Nowadays, databases and networks are extremely fast, so don't optimize
> prematurely. Test the "real code" and wait until the tests start taking a
> long time before deciding on a course of action (and when such a thing
> happens, I will usually choose not to use mock objects and split my tests
> further into "slow" and "fast" groups).

I agree with you in general (and I'm pretty sure you knew that). I am
talking about years of experience doing this, and not weeks, so I
strongly doubt that I have optimized prematurely (and I'm pretty sure
you haven't aimed that statement at me in particular).

> Like Elliotte, I like testing real objects, but I don't hesitate to use
> interfaces whenever it makes sense.

I prefer testing with real objects, if they run entirely in memory and
if they don't make my tests overly complex. Strangely enough, though,
real collaborators tend to make my tests overly complex. They introduce
irrelevant details in my tests and make it difficult to pinpoint
problems. It's about test execution speed, but not exclusively so.
--
J. B. (Joe) Rainsberger :: http://www.jbrains.info
Your guide to software craftsmanship
JUnit Recipes: Practical Methods for Programmer Testing
2005 Gordon Pask Award for contribution Agile Software Practice



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[J. B. Rainsberger]

Elliotte Harold wrote:

> J. B. Rainsberger wrote:
>
>  > I gather your test suites run quickly enough to get enough feedback to
>  > keep you happy. Mine don't, so I want faster feedback, and more stuff
>  > running in memory makes that happen.
>
> Sometimes you need to do that, but when that happens I tend to split the
> test suite into a full version and a compact version. The full version
> can be run automatically and repeatedly on a build server. If it takes
> an hour or even 24 hours to run, it's no big deal, as long as the
> compact version runs quickly.
>
> I also sometimes reorganize test suites so I run just the subset that
> seems most likely to touch the code I'm working on. That can also reduce
> build times to manageable levels.

I prefer to know my code works when I integrate it, and not at some
point after that. If I'm checking in code four times per day or more, I
don't want to wait 6 hours to find out that my 9:45 AM check-in didn't
work, then have to back away three check-ins to figure out what
happened. That way lies integration hell.

>  > A question: when you test business logic that touches a relational
>  > database, do you just put up with how slow those tests are? If not, what
>  > do you do? HSQLDB? Something else?
>
> I haven't done a lot of RDB testing. My slowest tests mostly involve
> network connections and web servers. For that, I either put up with the
> slowness or subset the tests as described above. I suppose I could build
> a web server into my test suite, but honestly that's never been the
> simplest thing that could possibly work for me. :-) If it ever becomes a
> major problem, I might consider it.

That's good to know. You've never, say, used Jetty for that purpose? I
haven't, but I hear it works quite well, filling the same role that
HSQLDB would do in RDB testing.
--
J. B. (Joe) Rainsberger :: http://www.jbrains.info
Your guide to software craftsmanship
JUnit Recipes: Practical Methods for Programmer Testing
2005 Gordon Pask Award for contribution Agile Software Practice



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[J. B. Rainsberger]

Robert Martin wrote:

>  > Not disputing this point (as I have only been using Rails now for a
>  > week or
>  > total and so far I love the testing capabilities that I have seenl)
>  > but I
>  > was disappointed that the Pragmatic Series book Agile Web
>  > Development with
>  > Rails has no coverage of applying TDD with Rails.
>  >
>  > One could argue that this was not the point of the book, that it is
>  > about
>  > learning the Rails framework but I just expected to see some
>  > mention and
>  > demonstration of it by either Dave Thomas of Mike Clark.
>
> I was disappointed too. Mike's chapter was very good, but it was
> almost pointedly devoid of TDD. The test-later approach that the
> chapter took left the example relatively untested.
>
> Nevertheless I am very pleased with the testing orientation of Rails,
> and am looking forward to seeing that very active and energetic
> community continue to embrace the disciplines of TDD.

I am finding it difficult to do TDD with Rails, mostly because I'm
expecting to use the same dependency injection techniques there as I use
with Java, and that doesn't seem to fly. For this, I have two questions:

  * Does anyone else have this or a similar problem?
  * Would someone point me to a place to discuss those problems?

Thanks.
--
J. B. (Joe) Rainsberger :: http://www.jbrains.info
Your guide to software craftsmanship
JUnit Recipes: Practical Methods for Programmer Testing
2005 Gordon Pask Award for contribution Agile Software Practice



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[J. B. Rainsberger]

Robert Martin wrote:

>  > > 1. There are no instances in which TDD is impossible.
>  > > 2. There are no instances in which TDD is not advantageous.
>  > > 3. The impediments to TDD are human, not technical.
>  > > 4. The alternative to TDD is code that is not repeatably testable.
>  > >
>  > > Therefore I consider TDD to be a minimum professional standard.
>  > > Developers who do not use TDD fall below that standard.
>  >
>  > I can't quite agree with that, but I would be thrilled to wake up one
>  > morning to find out that it's true. If that became /the/ minimum
>  > professional standard (whatever that would mean), I wouldn't complain.
>
> I'm not quite sure I know what you are disagreeing and agreeing with.

Yes, I was unclear.

I think you wrote, essentially, that "good code" is impossible without
TDD. That's the thing with which I don't quite agree. Let me check the
thread to be sure...

<pause />

It turns out I was wrong, so let me double back.

I wonder about the notion of TDD being a minimum professional standard,
only because it's about process, and not results. I would agree with low
defect-rate, well-designed, easily-maintained code being a minimum
professional standard, and TDD being an excellent way to achieve that.
It is my preferred way to achieve that. I am not prepared, however, to
require it.

I hope that clarifies the matter.
--
J. B. (Joe) Rainsberger :: http://www.jbrains.info
Your guide to software craftsmanship
JUnit Recipes: Practical Methods for Programmer Testing
2005 Gordon Pask Award for contribution Agile Software Practice



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Cédric Beust ♔]

On 8/19/06, J. B. Rainsberger <jbrains@...> wrote:

>
>
> > Like Elliotte, I like testing real objects, but I don't hesitate to use
> > interfaces whenever it makes sense.
>
> I prefer testing with real objects, if they run entirely in memory and
> if they don't make my tests overly complex. Strangely enough, though,
> real collaborators tend to make my tests overly complex. They introduce
> irrelevant details in my tests and make it difficult to pinpoint
> problems. It's about test execution speed, but not exclusively so.

Just to clarify my position above: the "but" was in opposition to Elliote's
point (which is to stay away from interfaces), not to testing real objects.
Obviously, testing real objects and using interfaces are not mutually
exclusive.

To summarize:

   - I tend to use real objects as much as possible (avoiding mocks,
   using real DB connections, etc...).
   - I use interfaces whenever it makes sense (if they make either my
   code or my tests easier to develop, read and maintain).
   - I don't let the assumed execution speed of a test stop me from
   writing that test in the first place.  I write the test, run it, and if it
   takes longer than it "should", then I move it into the "functional" or
   "slow" group.

--
Cédric
http://testng.org


[Non-text portions of this message have been removed]



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Elliotte Harold]

J. B. Rainsberger wrote:

> Perhaps Moore's Law has changed the balance, but when you consider my
> goal is 100 tests/second, you can be sure the database client tests
> don't cut the mustard. Even without the arbitrary speed goal, there is a
> good reason to avoid testing against the database.

I don't think the rate matters so much as the absolute time. In
particular, I'm writing an article right now which hypothesizes that the
magic number is 90 seconds. Any test suite that runs in 90 seconds or
less is fast enough, whether that's 9000 tests or 90. Look for details
on The Cafes in a week or two. I will be curious to see if that 90
second figure matches other people's experiences or not.

--
Elliotte Rusty Harold  elharo@...
Java I/O 2nd Edition Just Published!
http://www.cafeaulait.org/books/javaio2/
http://www.amazon.com/exec/obidos/ISBN=0596527500/ref=nosim/cafeaulaitA/


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Cédric Beust ♔]

On 8/22/06, Elliotte Harold <elharo@...> wrote:

>
> J. B. Rainsberger wrote:
>
> > Perhaps Moore's Law has changed the balance, but when you consider my
> > goal is 100 tests/second, you can be sure the database client tests
> > don't cut the mustard. Even without the arbitrary speed goal, there is a
> > good reason to avoid testing against the database.
>
> I don't think the rate matters so much as the absolute time. In
> particular, I'm writing an article right now which hypothesizes that the
> magic number is 90 seconds. Any test suite that runs in 90 seconds or
> less is fast enough, whether that's 9000 tests or 90. Look for details
> on The Cafes in a week or two. I will be curious to see if that 90
> second figure matches other people's experiences or not.

I also prefer to put emphasis on time more than the number of tests, but I
can easily tolerate a few minutes.  I usually use this time to review my
change with a diff, make sure I'm not checking in more files than needed,
etc...

Still, as the software grows in complexity, the time to run unit tests is
bound to go up, so at some point, you need a way to be able to run subsets
of those (i.e. only run database unit tests if you only modified
database-related code).

--
Cédric
http://testng.org


[Non-text portions of this message have been removed]



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Andrew McDonagh-3]

I've found once I give in to Rail's 'convention over configuration' and
accept I need a db (even though I shouldn't) it helped a lot.

The second biggest hurdle was deciding to not unit test sometimes, as
the difference between a Rails unit test and a Rails Functional test can
be very little, so sometimes I've just written acceptance tests

Andrew

J. B. Rainsberger wrote:

>
> Robert Martin wrote:
>
> > > Not disputing this point (as I have only been using Rails now for a
> > > week or
> > > total and so far I love the testing capabilities that I have seenl)
> > > but I
> > > was disappointed that the Pragmatic Series book Agile Web
> > > Development with
> > > Rails has no coverage of applying TDD with Rails.
> > >
> > > One could argue that this was not the point of the book, that it is
> > > about
> > > learning the Rails framework but I just expected to see some
> > > mention and
> > > demonstration of it by either Dave Thomas of Mike Clark.
> >
> > I was disappointed too. Mike's chapter was very good, but it was
> > almost pointedly devoid of TDD. The test-later approach that the
> > chapter took left the example relatively untested.
> >
> > Nevertheless I am very pleased with the testing orientation of Rails,
> > and am looking forward to seeing that very active and energetic
> > community continue to embrace the disciplines of TDD.
>
> I am finding it difficult to do TDD with Rails, mostly because I'm
> expecting to use the same dependency injection techniques there as I use
> with Java, and that doesn't seem to fly. For this, I have two questions:
>
> * Does anyone else have this or a similar problem?
> * Would someone point me to a place to discuss those problems?
>
> Thanks.
> --
> J. B. (Joe) Rainsberger :: http://www.jbrains.info 
> <http://www.jbrains.info>
> Your guide to software craftsmanship
> JUnit Recipes: Practical Methods for Programmer Testing
> 2005 Gordon Pask Award for contribution Agile Software Practice
>
>
> ------------------------------------------------------------------------
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.405 / Virus Database: 268.11.4/424 - Release Date: 21/08/2006
>  

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[dsaff]

Elliotte Harold wrote:

> J. B. Rainsberger wrote:
>
>  
>> Perhaps Moore's Law has changed the balance, but when you consider my
>> goal is 100 tests/second, you can be sure the database client tests
>> don't cut the mustard. Even without the arbitrary speed goal, there is a
>> good reason to avoid testing against the database.
>>    
>
> I don't think the rate matters so much as the absolute time. In
> particular, I'm writing an article right now which hypothesizes that the
> magic number is 90 seconds. Any test suite that runs in 90 seconds or
> less is fast enough, whether that's 9000 tests or 90. Look for details
> on The Cafes in a week or two. I will be curious to see if that 90
> second figure matches other people's experiences or not.
>  

Personally, I find that after 30 seconds, my brain has moved on from the
task.  Of course, "fast enough" depends a lot on when the tests are
run--as a driver for each new feature, for regressions before each
check-in, continuously on every compile, etc.

    David Saff


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Simon Chappell]

On 8/22/06, David Saff <saff@...> wrote:

> Personally, I find that after 30 seconds, my brain has moved on from the
> task.  Of course, "fast enough" depends a lot on when the tests are
> run--as a driver for each new feature, for regressions before each
> check-in, continuously on every compile, etc.

When working on individual objects in Eclipse, I use the JUnit runner
and run the individual test class after each small change.
Periodically, and always before check in, I run the full build and
test suite to ensure that I haven't broken anything else. Time-wise, I
can tolerate a periodic build of less than two minutes ... about the
amount of time necessary to run to the bathroom and get some more tea.
:-)

Simon

--
www.simonpeter.org


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[dsaff]

Simon Chappell wrote:
> Time-wise, I
> can tolerate a periodic build of less than two minutes ... about the
> amount of time necessary to run to the bathroom and get some more tea.
> :-)
>  

Always glad to find a fellow practitioner of tea-driven development.  :-)

    David Saff


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Simon Chappell]

On 8/22/06, David Saff <saff@...> wrote:
> Simon Chappell wrote:
> > Time-wise, I
> > can tolerate a periodic build of less than two minutes ... about the
> > amount of time necessary to run to the bathroom and get some more tea.
> > :-)
> >
>
> Always glad to find a fellow practitioner of tea-driven development.  :-)

Quite so. Anything else would be uncivilised! :-)

Technical details: Tetley, Black Pekoe, brewed about 5 minutes, served
with milk but no sugar.

Simon

--
www.simonpeter.org


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Shane Mingins-2]

On 8/23/06, Andrew McDonagh <yahoogroups@...> wrote:
>
>
> The second biggest hurdle was deciding to not unit test sometimes, as
> the difference between a Rails unit test and a Rails Functional test can
> be very little, so sometimes I've just written acceptance tests
>







This is something that our Rails developer has suggested to me (although in
a slighlty different vein) and I am in the middle of the testing chapter (by
Mike Clark) so that I can get some sort of sense of this is actually true or
not.  I was originally using the first version of the (Pragmatic) Rails book
but have switched to the second version which does differ.

(JB) Is the TDD list a good place to discuss the TDD aspects/challenges with
Rails???  I would be interested in following any discussions.

Cheers
Shane


[Non-text portions of this message have been removed]



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Andrew McDonagh-3]

Just to emphasize I'm not saying 'no unit tests', only that that
sometimes the thing you are unit testing is easier tested as a
functional/acceptance test.  Its largely down the the little amount of
code you need to write to do something.


Moving this to TDD list is good for me



Shane Mingins wrote:

>
> On 8/23/06, Andrew McDonagh <yahoogroups@...
> <mailto:yahoogroups%40andrewmcdonagh.f2s.com>> wrote:
> >
> >
> > The second biggest hurdle was deciding to not unit test sometimes, as
> > the difference between a Rails unit test and a Rails Functional test can
> > be very little, so sometimes I've just written acceptance tests
> >
>
> This is something that our Rails developer has suggested to me
> (although in
> a slighlty different vein) and I am in the middle of the testing
> chapter (by
> Mike Clark) so that I can get some sort of sense of this is actually
> true or
> not. I was originally using the first version of the (Pragmatic) Rails
> book
> but have switched to the second version which does differ.
>

>
> (JB) Is the TDD list a good place to discuss the TDD
> aspects/challenges with
> Rails??? I would be interested in following any discussions.
>
> Cheers
> Shane
>
> [Non-text portions of this message have been removed]
>
>
> ------------------------------------------------------------------------
>
> No virus found in this incoming message.
> Checked by AVG Free Edition.
> Version: 7.1.405 / Virus Database: 268.11.4/424 - Release Date: 21/08/2006
>  

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Michael Feathers]

David Saff wrote:

>Elliotte Harold wrote:
>  
>
>>J. B. Rainsberger wrote:
>>
>>  
>>    
>>
>>>Perhaps Moore's Law has changed the balance, but when you consider my
>>>goal is 100 tests/second, you can be sure the database client tests
>>>don't cut the mustard. Even without the arbitrary speed goal, there is a
>>>good reason to avoid testing against the database.
>>>    
>>>      
>>>
>>I don't think the rate matters so much as the absolute time. In
>>particular, I'm writing an article right now which hypothesizes that the
>>magic number is 90 seconds. Any test suite that runs in 90 seconds or
>>less is fast enough, whether that's 9000 tests or 90. Look for details
>>on The Cafes in a week or two. I will be curious to see if that 90
>>second figure matches other people's experiences or not.
>>  
>>    
>>
>
>Personally, I find that after 30 seconds, my brain has moved on from the
>task.  Of course, "fast enough" depends a lot on when the tests are
>run--as a driver for each new feature, for regressions before each
>check-in, continuously on every compile, etc.
>
>  
>

I agree.  Personally, I think that 90 seconds is ridiculously long.  You
might as well be waiting for a C++ build in code with bad dependency
management at that point.  Sure, running all the tests may take that
long, but when I'm working in an area of code, I start running subsets
long before that.

Michael Feathers
www.objectmentor.com






 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Cédric Beust ♔]

On 8/22/06, Michael Feathers <mfeathers@...> wrote:
>
>
> I agree.  Personally, I think that 90 seconds is ridiculously long.  You
> might as well be waiting for a C++ build in code with bad dependency
> management at that point.  Sure, running all the tests may take that
> long, but when I'm working in an area of code, I start running subsets
> long before that.


We (well, maybe just I) might be talking about different things.

I usually only run tests before checking in, not every single time I save a
file.  While I'm coding, I usually never use the "normal" build procedure:
I stay within Eclipse and use the IDE to compile and run everything.  I
don't really care if the tests are broken at this point because it's
inevitable, and as long as I'm not checking in, I'm not breaking anything.

Once I think I have something consistent and I feel ready to check in, I run
the tests.  And this is when 90 seconds or even a few minutes doesn't really
feel that long, because this is when I go back to the shell to run the
"real" build (the build that the other team members will run) and I start
doing some clean up, such as diff'ing the files I'm about to check in,
making sure they are well commented, etc...

--
Cédric
http://testng.org


[Non-text portions of this message have been removed]



 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Simon Chappell]

On 8/23/06, Cédric Beust ♔ <cbeust@...> wrote:

> On 8/22/06, Michael Feathers <mfeathers@...> wrote:
> >
> >
> > I agree.  Personally, I think that 90 seconds is ridiculously long.  You
> > might as well be waiting for a C++ build in code with bad dependency
> > management at that point.  Sure, running all the tests may take that
> > long, but when I'm working in an area of code, I start running subsets
> > long before that.
>
>
> We (well, maybe just I) might be talking about different things.
>
> I usually only run tests before checking in, not every single time I save a
> file.  While I'm coding, I usually never use the "normal" build procedure:
> I stay within Eclipse and use the IDE to compile and run everything.  I
> don't really care if the tests are broken at this point because it's
> inevitable, and as long as I'm not checking in, I'm not breaking anything.
>
> Once I think I have something consistent and I feel ready to check in, I run
> the tests.  And this is when 90 seconds or even a few minutes doesn't really
> feel that long, because this is when I go back to the shell to run the
> "real" build (the build that the other team members will run) and I start
> doing some clean up, such as diff'ing the files I'm about to check in,
> making sure they are well commented, etc...

I think that I need to sit down. I've come over all faint. Cédric and
I agree on something. :-)

Simon

--
www.simonpeter.org


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[Michael Feathers]

Cédric Beust ♔ wrote:

>On 8/22/06, Michael Feathers <mfeathers@...> wrote:
>  
>
>>I agree.  Personally, I think that 90 seconds is ridiculously long.  You
>>might as well be waiting for a C++ build in code with bad dependency
>>management at that point.  Sure, running all the tests may take that
>>long, but when I'm working in an area of code, I start running subsets
>>long before that.
>>    
>>
>
>
>We (well, maybe just I) might be talking about different things.
>
>I usually only run tests before checking in, not every single time I save a
>file.  While I'm coding, I usually never use the "normal" build procedure:
>I stay within Eclipse and use the IDE to compile and run everything.  I
>don't really care if the tests are broken at this point because it's
>inevitable, and as long as I'm not checking in, I'm not breaking anything.
>
>Once I think I have something consistent and I feel ready to check in, I run
>the tests.  And this is when 90 seconds or even a few minutes doesn't really
>feel that long, because this is when I go back to the shell to run the
>"real" build (the build that the other team members will run) and I start
>doing some clean up, such as diff'ing the files I'm about to check in,
>making sure they are well commented, etc...
>
>  
>

Yes, it's just a different practice. I run the tests frequently to make
sure that everything else is working while I work on one particular
thing. Handy, particularly when refactoring.

Michael Feathers
www.objectmnentor.com




 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[J. B. Rainsberger]

Elliotte Harold wrote:

>  > Perhaps Moore's Law has changed the balance, but when you consider my
>  > goal is 100 tests/second, you can be sure the database client tests
>  > don't cut the mustard. Even without the arbitrary speed goal, there is a
>  > good reason to avoid testing against the database.
>
> I don't think the rate matters so much as the absolute time. In
> particular, I'm writing an article right now which hypothesizes that the
> magic number is 90 seconds. Any test suite that runs in 90 seconds or
> less is fast enough, whether that's 9000 tests or 90. Look for details
> on The Cafes in a week or two. I will be curious to see if that 90
> second figure matches other people's experiences or not.

For a group of tests that I want to run on every change, 90 seconds is
out of the question. Anything more than 10 seconds is too much.

For a group of tests that I want to run at least before I commit
changes, I can go as much as 5 minutes. It's likely that a 5-minute
suite fits comfortably in a 10-minute build (including check out,
compile, all that).

As for the rate, it does matter: if 20% of my tests take 45 seconds and
the rest (all in memory) take 5 seconds, then I will defer improving the
slow tests, even though there's already a fair amount of design inertia
there. In general, if my tests run at slower than 10 tests/second, I
will ask "Why" and consider doing something about it. To me, it's no
different than catching a production execution speed problem early.
--
J. B. (Joe) Rainsberger :: http://www.jbrains.info
Your guide to software craftsmanship
JUnit Recipes: Practical Methods for Programmer Testing
2005 Gordon Pask Award for contribution Agile Software Practice


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 

[J. B. Rainsberger]

Simon Chappell wrote:

>  > I usually only run tests before checking in, not every single time I
> save a
>  > file.

Really? Do you check in that frequently, or do you run your tests that
rarely?
--
J. B. (Joe) Rainsberger :: http://www.jbrains.info
Your guide to software craftsmanship
JUnit Recipes: Practical Methods for Programmer Testing
2005 Gordon Pask Award for contribution Agile Software Practice


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/junit/

<*> To unsubscribe from this group, send an email to:
    junit-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/