Can we discuss opinions on some various unit test frameworks for C++

[markpetronic]

We are using Scrum and developing a client-server application where the
client is embedded VxWorks in C++ and the server is Java. We are doing
well using JUnit on the server. But, we are struggling on the client
side. The client needs to be portable to run on embedded VxWorks and
Linux. We are using an older version of VxWorks (for reasons beyond our
control) and it does not support STL very well. However, on the Linux
side, we can do pretty much anything with more modern libraries and
compliers. So, we are developing on Linux (a portable kernel) that will
also run on VxWorks. Once we start doing the VxWorks builds, we would
like our unit tests to be able to run in that environment as well as on
the Linux builds. However, VxWorks is the more restrictive environment
so the team is using CxxTest because it does not use STL and other is
fairly lightweight using Perl to help setup the test wrappers. But the
problem is, we are having lots of trouble making it work easily and
integrate it into Eclipse.
What I am thinking of doing is recommend we try another framework
(GoogleTest, CppTest, etc) and forget about being able to run the unit
tests under the VxWorks build environment. Instead, the unit testing can
be done against the Linux build. Sure, there could be some
environment-specific issues that we could miss but I think the overall
productivity gain of a better framework would offset that minor
disadvantage. Finally, we are moving towards a Linux client anyway so
the future could likely mean the VxWorks version is not longer enhanced
and just remains a legacy application.
Long explanation, I know, but wanted to try to make it clear. So, my
basic questions are:
1. Any have similar issues with CxxTest or use this framework and why
did you select it?2. What are you preferences on various unit test
frameworks for use with C++?3. Any suggestions on what we should do?4.
Any recommendations regarding easier to use or more preferred frameworks
supporting mock objects in C+?
Thanks,Mark

[Phlip]

markpetronic wrote:

> What I am thinking of doing is recommend we try another framework
> (GoogleTest, CppTest, etc)

Try UnitTest++. It is lean, yet supports all the streaming & Abstract Test
patterns that modern testers hanker for.

> 4. Any recommendations regarding easier to use or more preferred
> frameworks supporting mock objects in C+?

If you architect from scratch, try to avoid mocks.

--
   Not Mike Feathers

[James Martin-7]

I recently switched from Boost Test to CppUTest and have been really
impressed so far. The rationale was to break dependencies with Boost (which
can be a little overkill at times). CppUTest is lightweight, does memory
leak detection, compiles easily on nix/osx/windows and has some nice helper
scripts/features bundled in for TDDers.

http://cpputest.org.

I'd advise grabbing SVN HEAD from sourceforge to try out the latest stuff,
including ready-to-go example .project files for Eclipse.



Thanks,
James.

[Hubert Matthews]

I now use GoogleTest.  I used to use a simple homegrown framework that
I used when teaching C++.  The key factor for me with a test framework
is that the tests must be self registering.  Otherwise you risk
accidentally not running tests.

Other reasons that I moved to GoogleTest are:
- XML output in JUnit format that integrates nicely with Hudson for
continuous integration
- the ability to disable certain tests
- setup and teardown
- terminating and non-terminating assertions

Things I'd like to change or work out how to do with GT:
- get IDE-friendly error messages so that I can jump to the offending line

GoogleMock integrates well with GoogleTest.  Mocking in C++ requires
abstract base classes with pure virtual functions.  You may need to
insert some of these.  They are often a good thing but excessive
mocking is probably an indication of too fine-grained testing.  You
might also like to consider using templates instead of inheritance to
substitute types.  Function pointers are another trick along with
function objects (functors).

On Linux I have also used loader substitution of functions (the --wrap
option of ld) and LD_PRELOAD for mocking functions.

--
Hubert Matthews         http://www.oxyware.com/
Software Consultant     hubert@...

[Kaleb Pederson-2]

On Monday 26 October 2009 06:21:04 pm James Martin wrote:

> I recently switched from Boost Test to CppUTest and have been really
> impressed so far. The rationale was to break dependencies with Boost (which
> can be a little overkill at times). CppUTest is lightweight, does memory
> leak detection, compiles easily on nix/osx/windows and has some nice helper
> scripts/features bundled in for TDDers.
>
> http://cpputest.org.
>
> I'd advise grabbing SVN HEAD from sourceforge to try out the latest stuff,
> including ready-to-go example .project files for Eclipse.

I've spent most of the last few years working with Qt (qt.nokia.com), a cross platform GUI toolkit.  Although I say GUI, it's modular can be used for non-GUI applications as well.  It includes a unit test library called QTestLib that works quite well (http://doc.trolltech.com/4.5/qtestlib-manual.html)

The main features that it includes that I like are:

* Basic GUI Testing (drive combo boxes, input fields, mouse clicks, etc.)
* Benchmarking
* Data-driven testing (run multiple data sets through a single test)

It does have some weaknesses in the test management area.  If you have hundreds of tests, it's a pain to select which tests to run.  I wrote a wrapper around QTestLib that allows me to specify the test class or classes, using a regex if desired, and then the tests to run.  Lastly, it's xml output option was useful for integrating with a CI-server so I could get feedback about the unit tests.

I've used QTestlib for testing C projects that I've worked on as well.

I've been tempted to play with googletest (http://code.google.com/p/googletest/). Does anybody have any experience with it?

Thanks.

--
Kaleb Pederson

Twitter - http://twitter.com/kalebpederson
Blog - http://kalebpederson.com