Re: Measuring integration test code coverage for a Maven plugin (fwd)

[dfabulich]

Should have cc:ed surefire-dev also.

Here's the coverage report.

http://docs.codehaus.org/download/attachments/35422245/surefire-clover-2007-12-10.zip

-Dan

---------- Forwarded message ----------
Date: Mon, 10 Dec 2007 00:02:50 -0800 (Pacific Standard Time)
From: Dan Fabulich <dan@...>
To: Maven Developers List <dev@...>
Subject: Re: Measuring integration test code coverage for a Maven plugin

Raphaël Piéroni wrote:

> My guess is that you should deploy your instrumented plugin in a test
> repository.

That's basically what I wound up doing.

For the record, I started by specifying a special local repository in my
setting directory.

Then I tried just using maven-clover-plugin 3.6, calling instrumentInternal
directly (which you're not supposed to do because you risk getting cloverized
stuff in your production output, though that's exactly what I wanted to do in
my case).

That didn't quite work because instrumentInternal includes a function
redirectArtifact() that changes the output jar to be blah-clover.jar; once I
commented that out and added clover as a dependency to all of my code, I was
able to run my integration tests and get a total view of my integration test
coverage.

For the record, Surefire's unit tests covered 45% of Surefire; with the added
integration tests we've covered 79.4% (87 classes, 3,323 / 4,187 elements).  It
also highlighted some key areas where we could add more tests, which was
exactly what I wanted.

-Dan

2007/12/7, Dan Fabulich <dan@...>:

>
> I recently added a bunch of integration tests for Surefire.  These
> integration tests automatically fork a separate Maven process to run real
> Maven builds, like the Maven core integration tests do.
>
> This naturally led me to wonder: Does Surefire (now) have reasonable code
> coverage?  Specifically, which lines in Surefire were covered by unit
> tests, which by integration tests, and which weren't covered at all?
>
> I know there's a variety of handy code coverage tools that work with
> Maven, allowing you to instrument classes for code coverage and run your
> unit tests against the instrumented classes.
>
> The catch in this case is that I need to somehow convince Maven to use the
> instrumented version of my plugin, and not the regular "real" version of
> the plugin, when I go to run my integration tests.  The clover plugin, for
> example, doesn't seem to want to let me do that.
>
> [On the other hand, maybe I should just use an instrumenting JVM
> instead...?  Java 1.5's new java.lang.instrument would probably do the
> trick, but I'm not aware of any code coverage tool that works with
> j.l.instrument, and anyway I'd have to fix SUREFIRE-179 just to get it to
> work... :-)]
>
> Has anybody ever done this before?  More generally, I don't think I've
> ever seen an example of anyone using Maven to run multi-process
> integration tests (e.g. cargo tests) and also measuring code coverage on
> those integration tests.  Has anyone seen a good example of this that I
> could reuse?
>
> -Dan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>
>