Consider promoting emma as top level plugin for coverage

View: New views
7 Messages — Rating Filter:   Alert me  

Consider promoting emma as top level plugin for coverage

by Julien HENRY :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

A project of my organisation is running JUnit testing of complicated computing algorithms. Running mvn clean test takes less than 2 minutes. Running mvn cobertura:cobertura takes nearly 25 minutes. Running mvn emma:emma takes less than 3 minutes.

Of course running sonar:sonar takes a long time when using cobertura and is much better when using emma.

I raised this issue on cobertura mailing list with a simple test case to reproduce but after looking at the cobertura code, I don't think they can improve things easily. According to me there is a big design issue in Cobertura, with too much overhead for instrumented code.
Emma has a much smaller overhead (this is partly due to the fact that emma doesn't "count" line coverage) so I think you may consider using it as a default code coverage plugin, or at least include it in the distribution, after fixing SONARPLUGINS-270 of course ;)

Regards,

Julien





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Re: Consider promoting emma as top level plugin for coverage

by Olivier Gaudin-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello Julien,

and SONARPLUGINS-52 as well ! Then it could be considered.

Olivier


On Tue, Oct 27, 2009 at 4:25 AM, Julien HENRY <henryju@...> wrote:
Hi,

A project of my organisation is running JUnit testing of complicated computing algorithms. Running mvn clean test takes less than 2 minutes. Running mvn cobertura:cobertura takes nearly 25 minutes. Running mvn emma:emma takes less than 3 minutes.

Of course running sonar:sonar takes a long time when using cobertura and is much better when using emma.

I raised this issue on cobertura mailing list with a simple test case to reproduce but after looking at the cobertura code, I don't think they can improve things easily. According to me there is a big design issue in Cobertura, with too much overhead for instrumented code.
Emma has a much smaller overhead (this is partly due to the fact that emma doesn't "count" line coverage) so I think you may consider using it as a default code coverage plugin, or at least include it in the distribution, after fixing SONARPLUGINS-270 of course ;)

Regards,

Julien





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email




Re : [sonar-user] Consider promoting emma as top level plugin for coverage

by Julien HENRY :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Are you sure the hit per line is available with emma? I don't see this information in generated HTML report.

Also, is this information so important?

Julien


>
>De : Olivier Gaudin <gaudol@...>
>À : user@...
>Envoyé le : Mar 27 Octobre 2009, 12 h 23 min 15 s
>Objet : Re: [sonar-user] Consider promoting emma as top level plugin for  coverage
>
>Hello Julien,
>
>and SONARPLUGINS-52 as well ! Then it could be considered.
>
>Olivier
>
>
>
>On Tue, Oct 27, 2009 at 4:25 AM, Julien HENRY <henryju@...> wrote:
>
>>>Hi,
>>
>>>>A project of my organisation is running JUnit testing of complicated computing algorithms. Running mvn clean test takes less than 2 minutes. Running mvn cobertura:cobertura takes nearly 25 minutes. Running mvn emma:emma takes less than 3 minutes.
>>
>>>>Of course running sonar:sonar takes a long time when using cobertura and is much better when using emma.
>>
>>>>I raised this issue on cobertura mailing list with a simple test case to reproduce but after looking at the cobertura code, I don't think they can improve things easily. According to me there is a big design issue in Cobertura, with too much overhead for instrumented code.
>>>>
>>Emma has a much smaller overhead (this is partly due to the fact that emma doesn't "count" line coverage) so I think you may consider using it as a default code coverage plugin, or at least include it in the distribution, after fixing SONARPLUGINS-270 of course ;)
>>
>>>>Regards,
>>
>>>>Julien
>>
>>
>>
>>
>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe from this list, please visit:
>>
>>>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Re: Re : [sonar-user] Consider promoting emma as top level plugin for coverage

by Freddy Mallet :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Henry,

Emma is definitely a good code coverage engine but it has two major flaws for a good integration into Sonar :
  • The XML report file doesn't contain hit per line information and so line covered can't be displayed by Sonar in the Source viewer
  • Emma doesn't manage branch coverage whereas Clover and Cobertura do.
regards,
Freddy

On Tue, Oct 27, 2009 at 9:04 AM, Julien HENRY <henryju@...> wrote:
Are you sure the hit per line is available with emma? I don't see this information in generated HTML report.

Also, is this information so important?

Julien


>
>De : Olivier Gaudin <gaudol@...>
>À : user@...
>Envoyé le : Mar 27 Octobre 2009, 12 h 23 min 15 s
>Objet : Re: [sonar-user] Consider promoting emma as top level plugin for  coverage
>
>Hello Julien,
>
>and SONARPLUGINS-52 as well ! Then it could be considered.
>
>Olivier
>
>
>
>On Tue, Oct 27, 2009 at 4:25 AM, Julien HENRY <henryju@...> wrote:
>
>>>Hi,
>>
>>>>A project of my organisation is running JUnit testing of complicated computing algorithms. Running mvn clean test takes less than 2 minutes. Running mvn cobertura:cobertura takes nearly 25 minutes. Running mvn emma:emma takes less than 3 minutes.
>>
>>>>Of course running sonar:sonar takes a long time when using cobertura and is much better when using emma.
>>
>>>>I raised this issue on cobertura mailing list with a simple test case to reproduce but after looking at the cobertura code, I don't think they can improve things easily. According to me there is a big design issue in Cobertura, with too much overhead for instrumented code.
>>>>
>>Emma has a much smaller overhead (this is partly due to the fact that emma doesn't "count" line coverage) so I think you may consider using it as a default code coverage plugin, or at least include it in the distribution, after fixing SONARPLUGINS-270 of course ;)
>>
>>>>Regards,
>>
>>>>Julien
>>
>>
>>
>>
>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe from this list, please visit:
>>
>>>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email




Re : [sonar-user] Re : [sonar-user] Consider promoting emma as top level plugin for coverage

by Julien HENRY :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Freddy,

>    * The XML report file doesn't contain hit per line information
and so line covered can't be displayed by Sonar in the Source viewer

Are you sure this is only the XML report that is missing hit per line? According to me Emma simply doesn't count the hit per line. I would be very happy to be wrong because in this case it would be quite simple to add this information to the XML.
I read that hit per line is available in HTML report but it seems it is wrong (at least with my own testing). Do you have an example of Emma HTML report with hit per line?

Regards,

Julien

>
>De : Freddy Mallet <freddy.mallet@...>
>À : user@...
>Envoyé le : Mer 28 Octobre 2009, 13 h 51 min 11 s
>Objet : Re: [sonar-user] Re : [sonar-user] Consider promoting emma as top  level plugin for coverage
>
>Hi Henry,
>
>Emma is definitely a good code coverage engine but it has two major flaws for a good integration into Sonar :
>
>    * The XML report file doesn't contain hit per line information and so line covered can't be displayed by Sonar in the Source viewer
>
>    * Emma doesn't manage branch coverage whereas Clover and Cobertura do.
>
>regards,
>Freddy
>
>
>On Tue, Oct 27, 2009 at 9:04 AM, Julien HENRY <henryju@...> wrote:
>
>>>
>>Are you sure the hit per line is available with emma? I don't see this information in generated HTML report.
>>
>>>>Also, is this information so important?
>>
>>>>Julien
>>
>>
>>>>>
>>>>>De : Olivier Gaudin <gaudol@...>
>>>>>À : user@...
>>>>>Envoyé le : Mar 27 Octobre 2009, 12 h 23 min 15 s
>>>>>Objet : Re: [sonar-user] Consider promoting emma as top level plugin for  coverage
>>
>>>
>>>>>Hello Julien,
>>>>>
>>>>>and SONARPLUGINS-52 as well ! Then it could be considered.
>>>>>
>>>>>Olivier
>>>>>
>>>>>
>>>>>
>>>>>On Tue, Oct 27, 2009 at 4:25 AM, Julien HENRY <henryju@...> wrote:
>>>>>
>>>>>>>Hi,
>>>>>>
>>>>>>>>A project of my organisation is running JUnit testing of complicated computing algorithms. Running mvn clean test takes less than 2 minutes. Running mvn cobertura:cobertura takes nearly 25 minutes. Running mvn emma:emma takes less than 3 minutes.
>>>>
>>
>>>>
>>>>>>>>Of course running sonar:sonar takes a long time when using cobertura and is much better when using emma.
>>>>>>
>>>>>>>>I raised this issue on cobertura mailing list with a simple test case to reproduce but after looking at the cobertura code, I don't think they can improve things easily. According to me there is a big design issue in Cobertura, with too much overhead for instrumented code.
>>>>
>>
>>>>>>
>>>>>>Emma has a much smaller overhead (this is partly due to the fact that emma doesn't "count" line coverage) so I think you may consider using it as a default code coverage plugin, or at least include it in the distribution, after fixing SONARPLUGINS-270 of course ;)
>>>>
>>
>>>>
>>>>>>>>Regards,
>>>>>>
>>>>>>>>Julien
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>---------------------------------------------------------------------
>>>>>>>>To unsubscribe from this list, please visit:
>>>>>>
>>>>>>>>    http://xircles.codehaus.org/manage_email
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>
>>
>>
>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe from this list, please visit:
>>
>>>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Re: Re : [sonar-user] Consider promoting emma as top level plugin for coverage

by rpelisse :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi guys,

It's no secret I'm big fan of Emma and greatly dislike Cobertura.

"Emma doesn't manage branch coverage whereas Clover and Cobertura do." => euh, isn't it the otherway ???

(however if you have such tremendous performance differences, this may not be right).

"The XML report file doesn't contain hit per line information and so line covered can't be displayed by Sonar in the Source viewer" => well, this could be a nice contribution to Emma... Anybody volunteering ? :)


2009/10/28 Freddy Mallet <freddy.mallet@...>
Hi Henry,

Emma is definitely a good code coverage engine but it has two major flaws for a good integration into Sonar :
  • The XML report file doesn't contain hit per line information and so line covered can't be displayed by Sonar in the Source viewer
  • Emma doesn't manage branch coverage whereas Clover and Cobertura do.
regards,
Freddy


On Tue, Oct 27, 2009 at 9:04 AM, Julien HENRY <henryju@...> wrote:
Are you sure the hit per line is available with emma? I don't see this information in generated HTML report.

Also, is this information so important?

Julien


>
>De : Olivier Gaudin <gaudol@...>
>À : user@...
>Envoyé le : Mar 27 Octobre 2009, 12 h 23 min 15 s
>Objet : Re: [sonar-user] Consider promoting emma as top level plugin for  coverage
>
>Hello Julien,
>
>and SONARPLUGINS-52 as well ! Then it could be considered.
>
>Olivier
>
>
>
>On Tue, Oct 27, 2009 at 4:25 AM, Julien HENRY <henryju@...> wrote:
>
>>>Hi,
>>
>>>>A project of my organisation is running JUnit testing of complicated computing algorithms. Running mvn clean test takes less than 2 minutes. Running mvn cobertura:cobertura takes nearly 25 minutes. Running mvn emma:emma takes less than 3 minutes.
>>
>>>>Of course running sonar:sonar takes a long time when using cobertura and is much better when using emma.
>>
>>>>I raised this issue on cobertura mailing list with a simple test case to reproduce but after looking at the cobertura code, I don't think they can improve things easily. According to me there is a big design issue in Cobertura, with too much overhead for instrumented code.
>>>>
>>Emma has a much smaller overhead (this is partly due to the fact that emma doesn't "count" line coverage) so I think you may consider using it as a default code coverage plugin, or at least include it in the distribution, after fixing SONARPLUGINS-270 of course ;)
>>
>>>>Regards,
>>
>>>>Julien
>>
>>
>>
>>
>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe from this list, please visit:
>>
>>>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email






--
Romain PELISSE,
"The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it" -- Terry Pratchett
http://belaran.eu/

Re: Re : [sonar-user] Consider promoting emma as top level plugin for coverage

by Freddy Mallet :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Romain,

On Wed, Oct 28, 2009 at 10:55 AM, Romain Pelisse <belaran@...> wrote:
Hi guys,

It's no secret I'm big fan of Emma and greatly dislike Cobertura.   

Only passionate guys can change the world ;-)
 

"Emma doesn't manage branch coverage whereas Clover and Cobertura do." => euh, isn't it the otherway ???

I do confirm that Emma doesn't manage branch coverage but only a non-standard block coverage concept.

cheers,
Freddy
 
(however if you have such tremendous performance differences, this may not be right).

"The XML report file doesn't contain hit per line information and so line covered can't be displayed by Sonar in the Source viewer" => well, this could be a nice contribution to Emma... Anybody volunteering ? :)  


2009/10/28 Freddy Mallet <freddy.mallet@...>

Hi Henry,

Emma is definitely a good code coverage engine but it has two major flaws for a good integration into Sonar :
  • The XML report file doesn't contain hit per line information and so line covered can't be displayed by Sonar in the Source viewer
  • Emma doesn't manage branch coverage whereas Clover and Cobertura do.
regards,
Freddy


On Tue, Oct 27, 2009 at 9:04 AM, Julien HENRY <henryju@...> wrote:
Are you sure the hit per line is available with emma? I don't see this information in generated HTML report.

Also, is this information so important?

Julien


>
>De : Olivier Gaudin <gaudol@...>
>À : user@...
>Envoyé le : Mar 27 Octobre 2009, 12 h 23 min 15 s
>Objet : Re: [sonar-user] Consider promoting emma as top level plugin for  coverage
>
>Hello Julien,
>
>and SONARPLUGINS-52 as well ! Then it could be considered.
>
>Olivier
>
>
>
>On Tue, Oct 27, 2009 at 4:25 AM, Julien HENRY <henryju@...> wrote:
>
>>>Hi,
>>
>>>>A project of my organisation is running JUnit testing of complicated computing algorithms. Running mvn clean test takes less than 2 minutes. Running mvn cobertura:cobertura takes nearly 25 minutes. Running mvn emma:emma takes less than 3 minutes.
>>
>>>>Of course running sonar:sonar takes a long time when using cobertura and is much better when using emma.
>>
>>>>I raised this issue on cobertura mailing list with a simple test case to reproduce but after looking at the cobertura code, I don't think they can improve things easily. According to me there is a big design issue in Cobertura, with too much overhead for instrumented code.
>>>>
>>Emma has a much smaller overhead (this is partly due to the fact that emma doesn't "count" line coverage) so I think you may consider using it as a default code coverage plugin, or at least include it in the distribution, after fixing SONARPLUGINS-270 of course ;)
>>
>>>>Regards,
>>
>>>>Julien
>>
>>
>>
>>
>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe from this list, please visit:
>>
>>>>    http://xircles.codehaus.org/manage_email
>>
>>
>>
>




---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email






--
Romain PELISSE,
"The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it" -- Terry Pratchett
http://belaran.eu/