[rvm-core] New DaCapo regressions

[Steve Blackburn]

Hi all,

I just wanted to give you all a heads-up that we now have new  
performance regressions running for the DaCapo suite every 12 hours  
(we will have sanity data available soon too).   Follow the bottom  
right links on the DaCapo web page (www.dacapobench.org).

This is an incremental improvement over the regressions which Robin  
maintained for a long time (gratitude to Robin for doing the hard work  
of getting the first one working and maintaining it for so long!).

Part of the importance of this email is that these results (as Robin's  
did) give us a heads up as to how Jikes RVM is a) performing against  
other VMs, and b) performing on candidate benchmarks for the next  
DaCapo release as those benchmarks emerge and evolve.

--Steve

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers]

Steve Blackburn wrote:

> Hi all,
>
> I just wanted to give you all a heads-up that we now have new  
> performance regressions running for the DaCapo suite every 12 hours  
> (we will have sanity data available soon too).   Follow the bottom  
> right links on the DaCapo web page (www.dacapobench.org).
>
> This is an incremental improvement over the regressions which Robin  
> maintained for a long time (gratitude to Robin for doing the hard work  
> of getting the first one working and maintaining it for so long!).
>
> Part of the importance of this email is that these results (as Robin's  
> did) give us a heads up as to how Jikes RVM is a) performing against  
> other VMs, and b) performing on candidate benchmarks for the next  
> DaCapo release as those benchmarks emerge and evolve.
>
> --Steve
>  

Hi Steve,

some of the tests appear to have failed to run on the Jikes RVM
(specifically chart, fop and sunflow), are there trackers for the issues?

Thanks,
Ian

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

Hi Ian,

I've been pretty frantic just trying to get the perf regressions  
presentable, and have not had a chance to look at the sanity  
regressions.   The chart regression is the well known Classpath gtk  
one, I suspect (since I did not want to give any of the VMs  
dispensations---they are just running out of the box... ...or not).

Actually this is not quite true---I had to explicitly set -
X:processors=all for us to get competitive performance.  I've lost  
track of the status of that.   I wonder why all is not the default  
(rather than 1).

Once I have the sanity regressions on line (in a few days), it should  
be easy to investigate and get something in the trackers.

--Steve


On 30/07/2008, at 7:53 PM, Ian Rogers wrote:

> Steve Blackburn wrote:
>> Hi all,
>>
>> I just wanted to give you all a heads-up that we now have new
>> performance regressions running for the DaCapo suite every 12 hours
>> (we will have sanity data available soon too).   Follow the bottom
>> right links on the DaCapo web page (www.dacapobench.org).
>>
>> This is an incremental improvement over the regressions which Robin
>> maintained for a long time (gratitude to Robin for doing the hard  
>> work
>> of getting the first one working and maintaining it for so long!).
>>
>> Part of the importance of this email is that these results (as  
>> Robin's
>> did) give us a heads up as to how Jikes RVM is a) performing against
>> other VMs, and b) performing on candidate benchmarks for the next
>> DaCapo release as those benchmarks emerge and evolve.
>>
>> --Steve
>>
>
> Hi Steve,
>
> some of the tests appear to have failed to run on the Jikes RVM
> (specifically chart, fop and sunflow), are there trackers for the  
> issues?
>
> Thanks,
> Ian
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

For what it is worth, we now (as of a few minutes ago) have "warmup"  
curves too.  These show you how quickly the various VMs warm up over  
10 iterations of a given benchmark.

--Steve

On 30/07/2008, at 8:09 PM, Steve Blackburn wrote:

> Hi Ian,
>
> I've been pretty frantic just trying to get the perf regressions
> presentable, and have not had a chance to look at the sanity
> regressions.   The chart regression is the well known Classpath gtk
> one, I suspect (since I did not want to give any of the VMs
> dispensations---they are just running out of the box... ...or not).
>
> Actually this is not quite true---I had to explicitly set -
> X:processors=all for us to get competitive performance.  I've lost
> track of the status of that.   I wonder why all is not the default
> (rather than 1).
>
> Once I have the sanity regressions on line (in a few days), it should
> be easy to investigate and get something in the trackers.
>
> --Steve
>
>
> On 30/07/2008, at 7:53 PM, Ian Rogers wrote:
>
>> Steve Blackburn wrote:
>>> Hi all,
>>>
>>> I just wanted to give you all a heads-up that we now have new
>>> performance regressions running for the DaCapo suite every 12 hours
>>> (we will have sanity data available soon too).   Follow the bottom
>>> right links on the DaCapo web page (www.dacapobench.org).
>>>
>>> This is an incremental improvement over the regressions which Robin
>>> maintained for a long time (gratitude to Robin for doing the hard
>>> work
>>> of getting the first one working and maintaining it for so long!).
>>>
>>> Part of the importance of this email is that these results (as
>>> Robin's
>>> did) give us a heads up as to how Jikes RVM is a) performing against
>>> other VMs, and b) performing on candidate benchmarks for the next
>>> DaCapo release as those benchmarks emerge and evolve.
>>>
>>> --Steve
>>>
>>
>> Hi Steve,
>>
>> some of the tests appear to have failed to run on the Jikes RVM
>> (specifically chart, fop and sunflow), are there trackers for the
>> issues?
>>
>> Thanks,
>> Ian
>>
>> -------------------------------------------------------------------------
>> This SF.Net email is sponsored by the Moblin Your Move Developer's
>> challenge
>> Build the coolest Linux based applications with Moblin SDK & win
>> great prizes
>> Grand prize is a trip for two to an Open Source event anywhere in
>> the world
>> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers]

Steve Blackburn wrote:
> For what it is worth, we now (as of a few minutes ago) have "warmup"  
> curves too.  These show you how quickly the various VMs warm up over  
> 10 iterations of a given benchmark.
>
> --Steve
>  
The new graphs are really great. I wonder if it makes sense to run fewer
performance tests now and shift the focus of the regression machines
onto running sanity and stress tests. These configurations test more
internal assertions of the RVM and can potentially help us spot bugs
better. Could we use the results from DaCapo to replace the current
performance tests?

Thanks,
Ian

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

On 01/08/2008, at 12:21 AM, Ian Rogers wrote:
> I wonder if it makes sense to run fewer
> performance tests now and shift the focus of the regression machines
> onto running sanity and stress tests.

I agree we're not doing the Jikes RVM sanity tests frequently enough  
right now.

Fortunately I've just purchased a new quad core machine which will be  
dedicated to Jikes RVM sanity tests, starting in about a week.  Once  
we have that up and running we can reshuffle and re-prioritize a  
number of our ANU-based Jikes RVM regression set ups, including  
possibly introducing tests on another distro.   I'm struggling with  
quite a few different concerns at once, so apols for this not  
happening more quickly, but I have already started the administrative  
side, such as gettin new names registered within our domain for our  
machines.

Cheers,

--Steve

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[David P Grove]

> I just wanted to give you all a heads-up that we now have new
> performance regressions running for the DaCapo suite every 12 hours
> (we will have sanity data available soon too).   Follow the bottom
> right links on the DaCapo web page (www.dacapobench.org).
>
> This is an incremental improvement over the regressions which Robin
> maintained for a long time (gratitude to Robin for doing the hard work
> of getting the first one working and maintaining it for so long!).
>
> Part of the importance of this email is that these results (as Robin's
> did) give us a heads up as to how Jikes RVM is a) performing against
> other VMs, and b) performing on candidate benchmarks for the next
> DaCapo release as those benchmarks emerge and evolve.

It would be nice to have the same summary graph (geomean across all
benchmarks) that the old regressions did.

--dave


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[David P Grove]

Ian wrote:
> The new graphs are really great. I wonder if it makes sense to run fewer
> performance tests now and shift the focus of the regression machines
> onto running sanity and stress tests. These configurations test more
> internal assertions of the RVM and can potentially help us spot bugs
> better. Could we use the results from DaCapo to replace the current
> performance tests?

We still need jbb and jvm98 (jvm2008...) results in addition to DaCapo.
However, I agree that as the new hardware comes on line, we should think a
little about how best to deploy our cycles to get best coverage.

--dave


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

Sigh.  I did not intend to open a can of worms.   Let me clarify.

What the DaCapo regressions are not:

a. they are not using Jikes RVM resources
b. they are not part of the Jikes RVM project
c. they are not intended to help debug performance regressions within  
Jikes RVM
        - they do not provide information on revision numbers
        - they do not give performance break downs in terms of GC & mutator
        - they do not give large heap, small heap regressions
        - they only cover DaCapo
d. they are not a substitute for our own comprehensive regression  
testing


What the DaCapo regressions are:

a. A tool for the DaCapo benchmark developers (right now, that's me)
        - to characterize individual benchmarks across a spectrum of VMs
        - to identify the affect of changes to the benchmarks (crucial while  
developing the bms, as I am actively doing now)
        - to identify regressions (ie if I've broken the suite)
b. An opportunity for VM developers to compare across VMs
c. An opportunity for VM developers to anticipate how their VM will  
perform with the next DaCapo release

So yes, there are problems with our (Jikes RVM) regressions right now,  
(IMHO mostly related to our reporting).  I don't think this issue has  
much to do with the DaCapo regressions.

As to Dave's request re geomeans, yes, I hope to have something like  
that some time soon.  However, in the revamp I tried hard to deliver  
statistically meaningful graphs.   Producing meaningful geomeans is a  
little difficult since (particularly with the dacapo head) not all VMs  
run all benchmarks, so you end up comparing means across different  
sets, which is not particularly meaningful.

--Steve

On 05/08/2008, at 7:06 AM, David P Grove wrote:

>
> Ian wrote:
>> The new graphs are really great. I wonder if it makes sense to run  
>> fewer
>> performance tests now and shift the focus of the regression machines
>> onto running sanity and stress tests. These configurations test more
>> internal assertions of the RVM and can potentially help us spot bugs
>> better. Could we use the results from DaCapo to replace the current
>> performance tests?
>
> We still need jbb and jvm98 (jvm2008...) results in addition to  
> DaCapo.
> However, I agree that as the new hardware comes on line, we should  
> think a
> little about how best to deploy our cycles to get best coverage.
>
> --dave
>
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[gnu_andrew]

2008/7/30 Steve Blackburn <steve.blackburn@...>:

> Hi all,
>
> I just wanted to give you all a heads-up that we now have new
> performance regressions running for the DaCapo suite every 12 hours
> (we will have sanity data available soon too).   Follow the bottom
> right links on the DaCapo web page (www.dacapobench.org).
>
> This is an incremental improvement over the regressions which Robin
> maintained for a long time (gratitude to Robin for doing the hard work
> of getting the first one working and maintaining it for so long!).
>
> Part of the importance of this email is that these results (as Robin's
> did) give us a heads up as to how Jikes RVM is a) performing against
> other VMs, and b) performing on candidate benchmarks for the next
> DaCapo release as those benchmarks emerge and evolve.
>
> --Steve
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

Would you be interested in adding IcedTea/OpenJDK to the set of compared VMs?
This is becoming the main VM shipped with GNU/Linux distributions.
I'd be happy to help getting it setup.
--
Andrew :-)

Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net

PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA  7927 142C 2591 94EF D9D8

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

Hi Andrew,

On 27/10/2008, at 9:55 PM, Andrew John Hughes wrote:
> 2008/7/30 Steve Blackburn <steve.blackburn@...>:
>> I just wanted to give you all a heads-up that we now have new
>> performance regressions running for the DaCapo suite every 12 hours
> Would you be interested in adding IcedTea/OpenJDK to the set of  
> compared VMs?
> This is becoming the main VM shipped with GNU/Linux distributions.
> I'd be happy to help getting it setup.


This sounds good.   Let's follow up off-line.

Apols for the delay, I am traveling.

--Steve

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

It looks like the dacapobench results have stalled.

Regards,
Ian

2008/7/30 Steve Blackburn <steve.blackburn@...>:

> Hi all,
>
> I just wanted to give you all a heads-up that we now have new
> performance regressions running for the DaCapo suite every 12 hours
> (we will have sanity data available soon too).   Follow the bottom
> right links on the DaCapo web page (www.dacapobench.org).
>
> This is an incremental improvement over the regressions which Robin
> maintained for a long time (gratitude to Robin for doing the hard work
> of getting the first one working and maintaining it for so long!).
>
> Part of the importance of this email is that these results (as Robin's
> did) give us a heads up as to how Jikes RVM is a) performing against
> other VMs, and b) performing on candidate benchmarks for the next
> DaCapo release as those benchmarks emerge and evolve.
>
> --Steve
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

On 11/02/2009, at 3:08 AM, Ian Rogers wrote:
> It looks like the dacapobench results have stalled.

I haven't had time to look into it yet, but it coincides exactly with  
the change in classpath, so I'm guessing that it somehow relates to  
that.

I would like to back out of the classpath shift until:

a) we have it working on AIX
b) we have the luindex regression figured out
c) the dust has settled on the PNT commit

In the past we have a) discussed and b) tested things before bumping  
classpath versions.   This particular change was especially  
inopportune given the magnitude of the PNT commit and the issues we're  
still chasing on that.

Each of the steps a), b) & c) above depend on different people on the  
core team having available time to tackle them.   Leaving the  
classpath change in the trunk implicitly places pressure on various  
members of the core team to fix bugs immediately that they had not  
anticipated.   I don't think that is reasonable under the  
circumstances.  It is not a good idea to break from our prior approach  
of discussion and testing prior to classpath version changes.

I propose that we back out, resolve the major residual issues with  
PNT, and concurrently iron out the classpath bugs (either in a branch  
or privately).

--Steve

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[gnu_andrew]

2009/2/10 Steve Blackburn <Steve.Blackburn@...>:

> On 11/02/2009, at 3:08 AM, Ian Rogers wrote:
>> It looks like the dacapobench results have stalled.
>
> I haven't had time to look into it yet, but it coincides exactly with
> the change in classpath, so I'm guessing that it somehow relates to
> that.
>
> I would like to back out of the classpath shift until:
>
> a) we have it working on AIX
> b) we have the luindex regression figured out
> c) the dust has settled on the PNT commit
>
> In the past we have a) discussed and b) tested things before bumping
> classpath versions.   This particular change was especially
> inopportune given the magnitude of the PNT commit and the issues we're
> still chasing on that.
>
> Each of the steps a), b) & c) above depend on different people on the
> core team having available time to tackle them.   Leaving the
> classpath change in the trunk implicitly places pressure on various
> members of the core team to fix bugs immediately that they had not
> anticipated.   I don't think that is reasonable under the
> circumstances.  It is not a good idea to break from our prior approach
> of discussion and testing prior to classpath version changes.
>
> I propose that we back out, resolve the major residual issues with
> PNT, and concurrently iron out the classpath bugs (either in a branch
> or privately).
>
> --Steve
>
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and code to
> build responsive, highly engaging applications that combine the power of local
> resources and data with the reach of the web. Download the Adobe AIR SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

I agree the move to 0.98 was quick; I hadn't expected JikesRVM to have
switched so quickly (basically during the time I was at FOSDEM it
seems).  However, the issues with Classpath should have been spotted
earlier.  The release has been in development for nearly a year and
there has hardly been a flurry of patches in that time.  Indeed, many
of the patches are already being applied to JikesRVM (see the
difference between web/cvs patches).  The reason the issues weren't
anticipated is that there doesn't seem to be much testing against CVS.
 It would be much easier to track down the regression with lusearch if
there wasn't a year worth of patches inbetween, and, if this is a bug,
we'd obviously have preferred to catch it _before_ the release.
--
Andrew :-)

Free Java Software Engineer
Red Hat, Inc. (http://www.redhat.com)

Support Free Java!
Contribute to GNU Classpath and the OpenJDK
http://www.gnu.org/software/classpath
http://openjdk.java.net

PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
Fingerprint: F8EF F1EA 401E 2E60 15FA  7927 142C 2591 94EF D9D8

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

2009/2/10 Andrew John Hughes <gnu_andrew@...>:

> 2009/2/10 Steve Blackburn <Steve.Blackburn@...>:
>> On 11/02/2009, at 3:08 AM, Ian Rogers wrote:
>>> It looks like the dacapobench results have stalled.
>>
>> I haven't had time to look into it yet, but it coincides exactly with
>> the change in classpath, so I'm guessing that it somehow relates to
>> that.
>>
>> I would like to back out of the classpath shift until:
>>
>> a) we have it working on AIX
>> b) we have the luindex regression figured out
>> c) the dust has settled on the PNT commit
>>
>> In the past we have a) discussed and b) tested things before bumping
>> classpath versions.   This particular change was especially
>> inopportune given the magnitude of the PNT commit and the issues we're
>> still chasing on that.
>>
>> Each of the steps a), b) & c) above depend on different people on the
>> core team having available time to tackle them.   Leaving the
>> classpath change in the trunk implicitly places pressure on various
>> members of the core team to fix bugs immediately that they had not
>> anticipated.   I don't think that is reasonable under the
>> circumstances.  It is not a good idea to break from our prior approach
>> of discussion and testing prior to classpath version changes.
>>
>> I propose that we back out, resolve the major residual issues with
>> PNT, and concurrently iron out the classpath bugs (either in a branch
>> or privately).
>>
>> --Steve

A big problem is that PNT has broken Harmony and has a number of
issues we are just finding out about. Breaking Harmony is
unacceptable, we have lost the ability to run SPEC JVM 2008. With a
lot of fixing going on, it doesn't make sense to be applying the
changes to an out-of-date version of Classpath, it just increases
work. The AIX regressions appear trivial and I'm waiting for Dave's
follow up post to my comments on JIRA. The luindex regressions can be
fixed by increasing the timeout we give to luindex. I'm sure we'll
identify the performance regression soon and can patch Classpath. I
agree with Andrew that performance profiling the Classpath CVS should
have been more of a priority and we could have avoided the problem.
I've posted on JIRA how to collect performance data, it should be
easier to determine what's going on when that has been collected.

Ian

> I agree the move to 0.98 was quick; I hadn't expected JikesRVM to have
> switched so quickly (basically during the time I was at FOSDEM it
> seems).  However, the issues with Classpath should have been spotted
> earlier.  The release has been in development for nearly a year and
> there has hardly been a flurry of patches in that time.  Indeed, many
> of the patches are already being applied to JikesRVM (see the
> difference between web/cvs patches).  The reason the issues weren't
> anticipated is that there doesn't seem to be much testing against CVS.
>  It would be much easier to track down the regression with lusearch if
> there wasn't a year worth of patches inbetween, and, if this is a bug,
> we'd obviously have preferred to catch it _before_ the release.
> --
> Andrew :-)
>
> Free Java Software Engineer
> Red Hat, Inc. (http://www.redhat.com)
>
> Support Free Java!
> Contribute to GNU Classpath and the OpenJDK
> http://www.gnu.org/software/classpath
> http://openjdk.java.net
>
> PGP Key: 94EFD9D8 (http://subkeys.pgp.net)
> Fingerprint: F8EF F1EA 401E 2E60 15FA  7927 142C 2591 94EF D9D8
>
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and code to
> build responsive, highly engaging applications that combine the power of local
> resources and data with the reach of the web. Download the Adobe AIR SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

Hi Andrew,

Two separate issues here, both important.  I'd like not to confuse  
them.  One is immediate, one is strategic.

a) keeping Jikes RVM healthy:

On 11/02/2009, at 7:44 AM, Andrew John Hughes wrote:
> I agree the move to 0.98 was quick; I hadn't expected JikesRVM to have
> switched so quickly (basically during the time I was at FOSDEM it
> seems).

In the past we've discussed and tested beforehand, so no unpleasant  
surprises when the commit goes through.   I vote we temporarily back  
out while the problems are resolved.   I hope that's not a problem for  
anyone.


b) keeping Classpath healthy:

>  However, the issues with Classpath should have been spotted
> earlier.

I imagine we all agree with this.

The question is how to most effectively deal with it.   I think we  
used to actively track the Classpath head.   Perhaps we should resume  
that.   We're not short on IA32 regression testing hardware, though  
we're tight on PPC.  Any thoughts?

--Steve

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[David P Grove]

In my opinion there are too many things broken right now for any forward development to be happening.

Either we back out some of the easy to revert changes (eg classpath 0.98, possibly the native thread optimizations) or we put a total code freeze on for anything except bug fixes until some level of stability is restored.

Ian, you are underestimating how much time it typically takes for me to get a new version of GNU Classpath working on AIX. An easy upgrade takes a day....some of the more annoying ones in the past have taken a week or more to get straightened out. Turnaround time is a serious issue in debugging this. Each attempt to build classpath takes 60-90 minutes on AIX due to the very slow configure step and this isn't a week where I have a lot of spare cycles.

--dave

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

2009/2/11 David P Grove <groved@...>:

> In my opinion there are too many things broken right now for any forward
> development to be happening.
>
> Either we back out some of the easy to revert changes (eg classpath 0.98,
> possibly the native thread optimizations) or we put a total code freeze on
> for anything except bug fixes until some level of stability is restored.
>
> Ian, you are underestimating how much time it typically takes for me to get
> a new version of GNU Classpath working on AIX. An easy upgrade takes a
> day....some of the more annoying ones in the past have taken a week or more
> to get straightened out. Turnaround time is a serious issue in debugging
> this. Each attempt to build classpath takes 60-90 minutes on AIX due to the
> very slow configure step and this isn't a week where I have a lot of spare
> cycles.
>
> --dave

I won't fix Harmony on Classpath 0.97, fixing the class library is
tricky enough without worrying about the code being stale. Dave, if
you can handle the AIX issues it seems the machines aren't sane. I'd
suggest we just take the performance hit with Classpath 0.98 and bump
the timeouts. Most of the regressions I see relate to native threads
in particular RVM-784 that Fil is looking into. I think the priority
has to be to get RVM-784 closed and Fil has a hopeful fix in r15419.

Ian

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Daniel Frampton]

I have backed out classpath 0.98.

We really have to give native threads some time to settle in terms of
both stability and performance. Work on classpath 0.98 plus harmony
can go ahead on a branch if it needs to happen right now.

Cheers,
Daniel.

On Wed, Feb 11, 2009 at 11:25 PM, Ian Rogers
<ian.rogers@...> wrote:

> 2009/2/11 David P Grove <groved@...>:
>> In my opinion there are too many things broken right now for any forward
>> development to be happening.
>>
>> Either we back out some of the easy to revert changes (eg classpath 0.98,
>> possibly the native thread optimizations) or we put a total code freeze on
>> for anything except bug fixes until some level of stability is restored.
>>
>> Ian, you are underestimating how much time it typically takes for me to get
>> a new version of GNU Classpath working on AIX. An easy upgrade takes a
>> day....some of the more annoying ones in the past have taken a week or more
>> to get straightened out. Turnaround time is a serious issue in debugging
>> this. Each attempt to build classpath takes 60-90 minutes on AIX due to the
>> very slow configure step and this isn't a week where I have a lot of spare
>> cycles.
>>
>> --dave
>
> I won't fix Harmony on Classpath 0.97, fixing the class library is
> tricky enough without worrying about the code being stale. Dave, if
> you can handle the AIX issues it seems the machines aren't sane. I'd
> suggest we just take the performance hit with Classpath 0.98 and bump
> the timeouts. Most of the regressions I see relate to native threads
> in particular RVM-784 that Fil is looking into. I think the priority
> has to be to get RVM-784 closed and Fil has a hopeful fix in r15419.
>
> Ian
>
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills and code to
> build responsive, highly engaging applications that combine the power of local
> resources and data with the reach of the web. Download the Adobe AIR SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Filip Pizlo-2]

Ian,

I wonder, have you had luck reproducing RVM-784?  Daniel and I were  
able to reproduce it, but only just barely, after hours of running.  I  
wonder if you've been able to get it to die more reliably.

Hopefully, r15419 takes care of it.  But given our inability to  
reliably reproduce the failure in the first place, I consider this to  
really be a blind fix.

-F




On Feb 11, 2009, at 23:25 , Ian Rogers wrote:

> 2009/2/11 David P Grove <groved@...>:
>> In my opinion there are too many things broken right now for any  
>> forward
>> development to be happening.
>>
>> Either we back out some of the easy to revert changes (eg classpath  
>> 0.98,
>> possibly the native thread optimizations) or we put a total code  
>> freeze on
>> for anything except bug fixes until some level of stability is  
>> restored.
>>
>> Ian, you are underestimating how much time it typically takes for  
>> me to get
>> a new version of GNU Classpath working on AIX. An easy upgrade  
>> takes a
>> day....some of the more annoying ones in the past have taken a week  
>> or more
>> to get straightened out. Turnaround time is a serious issue in  
>> debugging
>> this. Each attempt to build classpath takes 60-90 minutes on AIX  
>> due to the
>> very slow configure step and this isn't a week where I have a lot  
>> of spare
>> cycles.
>>
>> --dave
>
> I won't fix Harmony on Classpath 0.97, fixing the class library is
> tricky enough without worrying about the code being stale. Dave, if
> you can handle the AIX issues it seems the machines aren't sane. I'd
> suggest we just take the performance hit with Classpath 0.98 and bump
> the timeouts. Most of the regressions I see relate to native threads
> in particular RVM-784 that Fil is looking into. I think the priority
> has to be to get RVM-784 closed and Fil has a hopeful fix in r15419.
>
> Ian
>
> ------------------------------------------------------------------------------
> Create and Deploy Rich Internet Apps outside the browser with  
> Adobe(R)AIR(TM)
> software. With Adobe AIR, Ajax developers can use existing skills  
> and code to
> build responsive, highly engaging applications that combine the  
> power of local
> resources and data with the reach of the web. Download the Adobe AIR  
> SDK and
> Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

------------------------------------------------------------------------------
Create and Deploy Rich Internet Apps outside the browser with Adobe(R)AIR(TM)
software. With Adobe AIR, Ajax developers can use existing skills and code to
build responsive, highly engaging applications that combine the power of local
resources and data with the reach of the web. Download the Adobe AIR SDK and
Ajax docs to start building applications today-http://p.sf.net/sfu/adobe-com
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core