100% coverage with instrumenting the already instrumented jars

[rashmi ramesh]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Hi,

I have observed that if i instrument the same jars repeatedly the code coverage shows 100%.

ie say for the 1st time i instrument A.jar  and i generate the report and delete the cobertura.ser file for this run.

Then i instrument the same A.jar which was already instrumented the first time and when i generate the report i get the coverage as 100%. Shouldnt the coverage be the same every time you instrument if you are using the same tests?

I am using TestNG for my tests and using cobertura 1.9.1.

Also Is there a way that coberura instruments jar files which have the compiled class files instead of .java files?

Thanks !

Rashmi.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel

[Jake Cobb-2]

Hello,

First, Cobertura only instruments at the bytecode level (i.e. class files) not at the source level (i.e. java files).

When Cobertura instruments, it stores a lot of static information in the cobertura.ser file.  It also adds a marker interface (HasBeenInstrumented) to the classes and skips any that already have this interface.  When it gathers coverage, it intends to be accumulating counts for pre-existing objects created during the instrumentation. However, it will create them on the fly if they don't exist.

So, what I believe you are seeing is this:

1. You instrument the jar and create a cobertura.ser that has containers for all the hit counts (so coverage % is hits > 0  over total number of hit count containers).
2. You destroy the cobertura.ser and "re-instrument", but all the previously instrumented classes are skipped because they have the marker interface.
3. You run your tests.  Everything that gets hit has a container created on the fly.  Since the only containers that exist are those created due to test coverage, you get 100% coverage.

Possible solutions:

1. Any time you delete cobertura.ser, you should also instrument uninstrumented classes so the full info is generated.
2. After initial instrumentation, make a copy of the cobertura.ser file before it has any hit counts.  Overwrite cobertura.ser with this copy to reset your hit counts.

-Jake Cobb

2009/8/25 rashmi ramesh <silverqueen_2000_2000@...>

Hi,

I have observed that if i instrument the same jars repeatedly the code coverage shows 100%.

ie say for the 1st time i instrument A.jar  and i generate the report and delete the cobertura.ser file for this run.

Then i instrument the same A.jar which was already instrumented the first time and when i generate the report i get the coverage as 100%. Shouldnt the coverage be the same every time you instrument if you are using the same tests?

I am using TestNG for my tests and using cobertura 1.9.1.

Also Is there a way that coberura instruments jar files which have the compiled class files instead of .java files?

Thanks !

Rashmi.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel

[rashmi ramesh]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Hi Jake,

Thanks a lot for your explanation. You have clearly answered the 100% coverage.

When you say that Cobertura instruments the class files and not the .java files, im wondering as to how the maven cobertura plugin works since i believe that this plugin looks for .java files. Does it compile the java files and then instrument? Is this the behaviour with the maven-cobertura plugin ?

Regards,

Rashmi.

---

From: Jake Cobb <jake.cobb@...>
To: rashmi ramesh <silverqueen_2000_2000@...>
Cc: cobertura-devel@...
Sent: Tuesday, August 25, 2009 10:52:39 AM
Subject: Re: [Cobertura-devel] 100% coverage with instrumenting the already instrumented jars

Hello,

First, Cobertura only instruments at the bytecode level (i.e. class files) not at the source level (i.e. java files).

When Cobertura instruments, it stores a lot of static information in the cobertura.ser file.  It also adds a marker interface (HasBeenInstrumented) to the classes and skips any that already have this interface.  When it gathers coverage, it intends to be accumulating counts for pre-existing objects created during the instrumentation. However, it will create them on the fly if they don't exist.

So, what I believe you are seeing is this:

1. You instrument the jar and create a cobertura.ser that has containers for all the hit counts (so coverage % is hits > 0  over total number of hit count containers).
2. You destroy the cobertura.ser and "re-instrument", but all the previously instrumented classes are skipped because they have the marker interface.
3. You run your tests.  Everything that gets hit has a container created on the fly.  Since the only containers that exist are those created due to test coverage, you get 100% coverage.

Possible solutions:

1. Any time you delete cobertura.ser, you should also instrument uninstrumented classes so the full info is generated.
2. After initial instrumentation, make a copy of the cobertura.ser file before it has any hit counts.  Overwrite cobertura.ser with this copy to reset your hit counts.

-Jake Cobb

2009/8/25 rashmi ramesh <silverqueen_2000_2000@...>

Hi,

I have observed that if i instrument the same jars repeatedly the code coverage shows 100%.

ie say for the 1st time i instrument A.jar  and i generate the report and delete the cobertura.ser file for this run.

Then i instrument the same A.jar which was already instrumented the first time and when i generate the report i get the coverage as 100%. Shouldnt the coverage be the same every time you instrument if you are using the same tests?

I am using TestNG for my tests and using cobertura 1.9.1.

Also Is there a way that coberura instruments jar files which have the compiled class files instead of .java files?

Thanks !

Rashmi.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel

[Jake Cobb-2]

I haven't used the Maven plugin so I can't comment on its behavior.  Compiling and then instrumenting is what I would suspect is happening.  The .java files are also needed for generating the HTML report, just not for the instrumentation itself.

-Jake

On Tue, Aug 25, 2009 at 2:13 PM, rashmi ramesh <silverqueen_2000_2000@...> wrote:

Hi Jake,

Thanks a lot for your explanation. You have clearly answered the 100% coverage.

When you say that Cobertura instruments the class files and not the .java files, im wondering as to how the maven cobertura plugin works since i believe that this plugin looks for .java files. Does it compile the java files and then instrument? Is this the behaviour with the maven-cobertura plugin ?

Regards,

Rashmi.

---

From: Jake Cobb <jake.cobb@...>
To: rashmi ramesh <silverqueen_2000_2000@...>
Cc: cobertura-devel@...
Sent: Tuesday, August 25, 2009 10:52:39 AM
Subject: Re: [Cobertura-devel] 100% coverage with instrumenting the already instrumented jars

Hello,

First, Cobertura only instruments at the bytecode level (i.e. class files) not at the source level (i.e. java files).

When Cobertura instruments, it stores a lot of static information in the cobertura.ser file.  It also adds a marker interface (HasBeenInstrumented) to the classes and skips any that already have this interface.  When it gathers coverage, it intends to be accumulating counts for pre-existing objects created during the instrumentation. However, it will create them on the fly if they don't exist.

So, what I believe you are seeing is this:

1. You instrument the jar and create a cobertura.ser that has containers for all the hit counts (so coverage % is hits > 0  over total number of hit count containers).
2. You destroy the cobertura.ser and "re-instrument", but all the previously instrumented classes are skipped because they have the marker interface.
3. You run your tests.  Everything that gets hit has a container created on the fly.  Since the only containers that exist are those created due to test coverage, you get 100% coverage.

Possible solutions:

1. Any time you delete cobertura.ser, you should also instrument uninstrumented classes so the full info is generated.
2. After initial instrumentation, make a copy of the cobertura.ser file before it has any hit counts.  Overwrite cobertura.ser with this copy to reset your hit counts.

-Jake Cobb

2009/8/25 rashmi ramesh <silverqueen_2000_2000@...>

Hi,

I have observed that if i instrument the same jars repeatedly the code coverage shows 100%.

ie say for the 1st time i instrument A.jar  and i generate the report and delete the cobertura.ser file for this run.

Then i instrument the same A.jar which was already instrumented the first time and when i generate the report i get the coverage as 100%. Shouldnt the coverage be the same every time you instrument if you are using the same tests?

I am using TestNG for my tests and using cobertura 1.9.1.

Also Is there a way that coberura instruments jar files which have the compiled class files instead of .java files?

Thanks !

Rashmi.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel

[rashmi ramesh]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Hi Jake,

Will the hit count be reset if i resuse the cobertura.ser file ?

ie 

1. Instrument the jar and save the cobertura.ser

2. Instrument the same jar and use the .ser from 1. At this point will the hit count be reset or is it incremental.

My concern is that say for the 1st run, line 2 was hit and for the 2nd run line 2 was not hit since a test failed. 

Then how will this data appear ? Is there an extra tag i can specify to reset the hit counts?

Regards,

Rashmi.

---

From: Jake Cobb <jake.cobb@...>
To: rashmi ramesh <silverqueen_2000_2000@...>
Cc: cobertura-devel@...
Sent: Tuesday, August 25, 2009 11:27:46 AM
Subject: Re: [Cobertura-devel] 100% coverage with instrumenting the already instrumented jars

I haven't used the Maven plugin so I can't comment on its behavior.  Compiling and then instrumenting is what I would suspect is happening.  The .java files are also needed for generating the HTML report, just not for the instrumentation itself.

-Jake

On Tue, Aug 25, 2009 at 2:13 PM, rashmi ramesh <silverqueen_2000_2000@...> wrote:

Hi Jake,

Thanks a lot for your explanation. You have clearly answered the 100% coverage.

When you say that Cobertura instruments the class files and not the .java files, im wondering as to how the maven cobertura plugin works since i believe that this plugin looks for .java files. Does it compile the java files and then instrument? Is this the behaviour with the maven-cobertura plugin ?

Regards,

Rashmi.

---

From: Jake Cobb <jake.cobb@...>
To: rashmi ramesh <silverqueen_2000_2000@...>
Cc: cobertura-devel@...
Sent: Tuesday, August 25, 2009 10:52:39 AM
Subject: Re: [Cobertura-devel] 100% coverage with instrumenting the already instrumented jars

Hello,

First, Cobertura only instruments at the bytecode level (i.e. class files) not at the source level (i.e. java files).

When Cobertura instruments, it stores a lot of static information in the cobertura.ser file.  It also adds a marker interface (HasBeenInstrumented) to the classes and skips any that already have this interface.  When it gathers coverage, it intends to be accumulating counts for pre-existing objects created during the instrumentation. However, it will create them on the fly if they don't exist.

So, what I believe you are seeing is this:

1. You instrument the jar and create a cobertura.ser that has containers for all the hit counts (so coverage % is hits > 0  over total number of hit count containers).
2. You destroy the cobertura.ser and "re-instrument", but all the previously instrumented classes are skipped because they have the marker interface.
3. You run your tests.  Everything that gets hit has a container created on the fly.  Since the only containers that exist are those created due to test coverage, you get 100% coverage.

Possible solutions:

1. Any time you delete cobertura.ser, you should also instrument uninstrumented classes so the full info is generated.
2. After initial instrumentation, make a copy of the cobertura.ser file before it has any hit counts.  Overwrite cobertura.ser with this copy to reset your hit counts.

-Jake Cobb

2009/8/25 rashmi ramesh <silverqueen_2000_2000@...>

Hi,

I have observed that if i instrument the same jars repeatedly the code coverage shows 100%.

ie say for the 1st time i instrument A.jar  and i generate the report and delete the cobertura.ser file for this run.

Then i instrument the same A.jar which was already instrumented the first time and when i generate the report i get the coverage as 100%. Shouldnt the coverage be the same every time you instrument if you are using the same tests?

I am using TestNG for my tests and using cobertura 1.9.1.

Also Is there a way that coberura instruments jar files which have the compiled class files instead of .java files?

Thanks !

Rashmi.

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel

------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now.  http://p.sf.net/sfu/bobj-july
_______________________________________________
Cobertura-devel mailing list
Cobertura-devel@...
https://lists.sourceforge.net/lists/listinfo/cobertura-devel