[Boost.Test] C++ Unit Tests with Boost.Test, Part 1

[Richard-185]

[Please do not mail me a copy of your followup]

<http://legalizeadulthood.wordpress.com/2009/07/04/c-unit-tests-with-boost-test-part-1/>

In this post, I will describe how to start making unit tests in C++
with Boost.Test.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
      <http://www.xmission.com/~legalize/book/download/index.html>

      Legalize Adulthood! <http://legalizeadulthood.wordpress.com>

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[John Phillips-3]

Richard wrote:
> [Please do not mail me a copy of your followup]
>
> <http://legalizeadulthood.wordpress.com/2009/07/04/c-unit-tests-with-boost-test-part-1/>
>
> In this post, I will describe how to start making unit tests in C++
> with Boost.Test.


   I glanced across them, and at first glance they look like a nice set
of tutorials. He has 5 parts posted.

                        John

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Johan Nilsson-4]

[This is actually mainly comments to Part 2]

Richard wrote:
> [Please do not mail me a copy of your followup]
>
> <http://legalizeadulthood.wordpress.com/2009/07/04/c-unit-tests-with-boost-test-part-1/>
>
> In this post, I will describe how to start making unit tests in C++
> with Boost.Test.

I've only glanced through the parts, so I might have missed something. Also
please note that my comments are very much from a TDD point of view.

In Part 2, you recommend using the CHECK macros by default. Being a
long-time Boost.Test user (as well as a long-time TDD:er) I strongly believe
that by default one should always use the REQUIRE variety of the Boost.Test
macros, for a couple of reasons:

- They most closely correspond to xUnit assertions. TDD:ers are used to
having their test cases aborted upon an assertion failure.

- When you have multiple verifications per test[*], one of the early
verifications are quite commonly some kind of precondition/guard statement
which could lead to memory corruption (or simply useless further testing) if
the test continues. I know you mention this as a special case in your
article, but if you use REQUIRE by default you will always be safe.

People will probably object to the above, saying that the want to test
multiple things in the same test, and want to see the result of all "related
verifications". Sorry, but I just won't buy that, because:

- You really don't want to manually inspect a lot of numbers/results when
TDD:ing. Seeing the name of the failing test case should be enough most of
the time.

- If you really need to see e.g. "x out of y" verifications fail together
with their output, from a _single_ test case, this is a clear indicator that
you should split that into multiple (more specific) test cases.

Other than this specific thing, it looks like a good introduction to testing
with Boost.Test.

Just my 0.02EUR.

/ Johan

[*] As you normally should attempt to stay out of.


_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Richard-185]

[Please do not mail me a copy of your followup]

boost-users@... spake the secret code
<h2s7c1$gik$1@...> thusly:

>In Part 2, you recommend using the CHECK macros by default. Being a
>long-time Boost.Test user (as well as a long-time TDD:er) I strongly believe
>that by default one should always use the REQUIRE variety of the Boost.Test
>macros, for a couple of reasons [...]

I'll see about getting that updated soon.

>People will probably object to the above, saying that the want to test
>multiple things in the same test, and want to see the result of all "related
>verifications". [...]

If you read through the whole series, you'll see that each test case
tests one thing and one thing only.  I link to xUnit Test Patterns
book online, which has quite a bit to say about test smells.

There's so much to cover on this subject and the posts were already
getting a bit lengthy (originally I thought it would be 1 blog post!).

Thanks for the feedback.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
      <http://www.xmission.com/~legalize/book/download/index.html>

      Legalize Adulthood! <http://legalizeadulthood.wordpress.com>

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Gennadiy Rozental-2]

Johan Nilsson <r.johan.nilsson <at> gmail.com> writes:

>
> [This is actually mainly comments to Part 2]
>
> Richard wrote:
> > [Please do not mail me a copy of your followup]
> >
> >
<http://legalizeadulthood.wordpress.com/2009/07/04/c-unit-tests-with-boost-test-part-1/>

> >
> > In this post, I will describe how to start making unit tests in C++
> > with Boost.Test.
>
> I've only glanced through the parts, so I might have missed something. Also
> please note that my comments are very much from a TDD point of view.
>
> In Part 2, you recommend using the CHECK macros by default. Being a
> long-time Boost.Test user (as well as a long-time TDD:er) I strongly believe
> that by default one should always use the REQUIRE variety of the Boost.Test
> macros, for a couple of reasons:

This only effectively implements single assertion per test case policy, which is
what you as hardcore TDDer probably doing anyway. For all us other folks CHECK
should be default level. REQUIRE should be used if you actually require one
assertion to pass before you proceed to next one.

Gennadiy

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Gennadiy Rozental-2]

Richard <legalize+jeeves <at> mail.xmission.com> writes:

> There's so much to cover on this subject and the posts were already
> getting a bit lengthy (originally I thought it would be 1 blog post!).
>
> Thanks for the feedback.

And you only touched the surface ;)

These are very nice. I definitely need to collect references to the online
tutorials somewhere inside Boost.Test docs.

Couple notes:

Didn't you mean to include single header variant of UTF in first tutorial?

You probably should also mention BOOST_FIXTURE_TEST_SUITE, since it's frequently
a better way to manage fixtures.

I'll look a bit more later and post more comments if I have any.

Regards,

Gennadiy

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Johan Nilsson-4]

Gennadiy Rozental wrote:

> Johan Nilsson <r.johan.nilsson <at> gmail.com> writes:
>
>>
>> [This is actually mainly comments to Part 2]
>>
>> Richard wrote:
>>> [Please do not mail me a copy of your followup]
>>>
>>>
> <http://legalizeadulthood.wordpress.com/2009/07/04/c-unit-tests-with-boost-test-part-1/>
>>>
>>> In this post, I will describe how to start making unit tests in C++
>>> with Boost.Test.
>>
>> I've only glanced through the parts, so I might have missed
>> something. Also please note that my comments are very much from a
>> TDD point of view.
>>
>> In Part 2, you recommend using the CHECK macros by default. Being a
>> long-time Boost.Test user (as well as a long-time TDD:er) I strongly
>> believe that by default one should always use the REQUIRE variety of
>> the Boost.Test macros, for a couple of reasons:
>
> This only effectively implements single assertion per test case
> policy, which is what you as hardcore TDDer probably doing anyway.
> For all us other folks CHECK should be default level. REQUIRE should
> be used if you actually require one assertion to pass before you
> proceed to next one.

Hmm, I wouldn't describe myself as "hard-core" TDD:er but, yes, having a
single assertion per test case is definitely something I aim for. Multiple
assertions are ok when verifying different aspects of the same logical
state, but even then such a thing can be split into multiple test cases (or
fixed by adding a custom assert).

As for REQUIRE vs CHECK there's another point of view IMHO; either the SUT
works or it does not. Why (in automated testing) bother checking whether
something "half-works"? If you are into "manually" verifying stuff by
looking at and interpreting large amounts of output on the screen IMHO
you're not really doing automated unit/developer testing.

(Ok, so maybe I'm a "medium-core" TDD:er then :)

I still can appreciate that Boost.Test can be used in many testing scenarios
and strategies, of which one TDD is only one application. The series of blog
posts though seems to be mainly written as an introduction on how to do
test-first development with Boost.Test, even though the title starts out
with "C++ Unit Tests ..." - which could mean many different things. Hence my
original comments.

Regards,
Johan


_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Gennadiy Rozental-2]

Johan Nilsson <r.johan.nilsson <at> gmail.com> writes:

> As for REQUIRE vs CHECK there's another point of view IMHO; either the SUT
> works or it does not. Why (in automated testing) bother checking whether
> something "half-works"? If you are into "manually" verifying stuff by
> looking at and interpreting large amounts of output on the screen IMHO
> you're not really doing automated unit/developer testing.

I am not gonna argue virtues of of using single or multiple asserts in test
cases. My only point is that you can't have it both ways. Either you have single
assertion per test case policy, or you agree people to use CHECK to validate
independent assertions within the same test case. REQUIRE still needs to be used
in case of dependent assertions.

Gennadiy

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Richard-185]

[Please do not mail me a copy of your followup]

boost-users@... spake the secret code
<loom.20090707T083334-298@...> thusly:

>Richard <legalize+jeeves <at> mail.xmission.com> writes:
>
>> There's so much to cover on this subject and the posts were already
>> getting a bit lengthy (originally I thought it would be 1 blog post!).
>>
>> Thanks for the feedback.
>
>And you only touched the surface ;)

Yes, there's quite a bit inside Boost.Test (thanks, BTW).  I have had
a hard time getting bootstrapped into writing tests using it, though.

My first unit tests were in NUnit and I found it very easy to become
productive right away.  At my last employer, we evaluated a bunch of
test frameworks for C++ and ended up picking Boost.Test as our
official test framework for C++.  I had been using it for a while,
mostly with BOOST_AUTO_TEST_CASE, but recently I wanted to learn more
about fixtures to extract common setup/teardown code.  I had to work
with the documentation for a while in order to figure out how to do
this.

I don't know exactly what it is about the documentation, but its
difficult for me to find stuff in there quickly.  The easier it is to
get started with a library, the more people use it.  I don't know why,
but Boost.Test feels hard to get started.  Once I got it figured out,
I decided that some tutorials would help others.

>These are very nice. I definitely need to collect references to the online
>tutorials somewhere inside Boost.Test docs.

Feel free.  I think the permalinks should be good for a long time now
that my blog is on wordpress.com.

>Didn't you mean to include single header variant of UTF in first tutorial?

I didn't want to do that because I wanted the first tutorial to cover
setting up a whole project and the way I've been doing that is to have
the dedicated main go into a single source file.  I know there are
other ways to have a "standalone single .cpp test project", but I
don't feel those cases are typical.  I always end up having one test
file per SUT (i.e. production class) with all the test cases for that
SUT in that source file.  I always end up with an EXE project and a
single main source file and one source file for each class I want to
test.

>You probably should also mention BOOST_FIXTURE_TEST_SUITE, since it's frequently
>a better way to manage fixtures.

That's covered in part 5 after I motivate its use with some
test-driven GUI development.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
      <http://www.xmission.com/~legalize/book/download/index.html>

      Legalize Adulthood! <http://legalizeadulthood.wordpress.com>

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Gennadiy Rozental-2]

Richard <legalize+jeeves <at> mail.xmission.com> writes:

[...]

> I don't know why, but Boost.Test feels hard to get started.  

Can't fix something you can't even pinpoint ;) I've tried to make it friendly
for the first time users. Could it be that you came from C#, Java experience and
thus have some different expectations?

> Once I got it figured out, I decided that some tutorials would help others.

Yes. Tutorials are definitely a good media to promote the library.

Maybe some more to come? to cover some more advanced topics?

Gennadiy

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users

[Richard-185]

[Please do not mail me a copy of your followup]

boost-users@... spake the secret code
<loom.20090708T072855-599@...> thusly:

>Richard <legalize+jeeves <at> mail.xmission.com> writes:
>
>[...]
>
>> I don't know why, but Boost.Test feels hard to get started.  
>
>Can't fix something you can't even pinpoint ;)

Yeah, I know.  If I have specific suggestions for improving the docs,
you'll hear about them.  I know its not beceause I started with NUnit.

>Yes. Tutorials are definitely a good media to promote the library.
>
>Maybe some more to come? to cover some more advanced topics?

I think what's needed next (as far as my blog is concerned) is showing
how to use a C++ mocking framework, particularly for COM objects.
--
"The Direct3D Graphics Pipeline" -- DirectX 9 draft available for download
      <http://www.xmission.com/~legalize/book/download/index.html>

      Legalize Adulthood! <http://legalizeadulthood.wordpress.com>

_______________________________________________
Boost-users mailing list
Boost-users@...
http://lists.boost.org/mailman/listinfo.cgi/boost-users