« Return to Thread: [rvm-research] Looking for Sources of Performance Variation
Re: [rvm-research] Looking for Sources of Performance Variation
by Steve Blackburn
::
Rate this Message:
Hi Christian,
There are some very well known sources of performance variation for
managed runtimes, such as Jikes RVM. However, it sounds like you
have accounted for these. It is a little hard to tell though. It
might help if you can provide the following information:
- The *exact* command line used for a specific benchmark
- The *exact* results produced by one of your runs (ideally a log of
the 60 results you report below)
- The *exact* hardware you are running on (you say it is dual core, do
you mean Core 2 Duo?).
You should not see any significant variation if you turn the AOS
off. I do this fairly routinely. Nonetheless, I'd normally take 10
measurements even with the AOS off. WIth the AOS on, I'd be inclined
to take 20 measurements. I don't think you need to time the 10th
iteration. Take a look at the warm-up curves in the right column of
this page (http://dacapo.anu.edu.au/regression/perf/2006-10-MR2.html)
and you'll see that steady state is reached earlier than that. I
typically use the 4th iteration.
Cheers,
--steve
On 07/05/2009, at 10:13 PM, sunai@... wrote:
> This is a troubleshooting question. I am trying to run the dacapo
> benchmarks with an older revision (14775) of Jikes, using the
> ''perf'' test-run (I adapted it to run only the dacapo benchmarks),
> but the measurements turn out to be very unstable. E. g. for dacapo-
> fop running the 9 warum-up + 1 timed iterations for 6 executions
> mostly gives me results within a limited rage (barring 3% of
> variation), but quite a number of measurements (about one fifth) are
> very far off (+>10%). I have attempted a small baseline compiler
> modification to safe some control flow profiling (edge counters);
> when I run the patched VM with this code, all measurements are
> catapulted into the higher ballpark.
>
> I have switched off the AOS recompilation, which apparently also
> ensures that there is no invocation threshold-based recompilation. I
> am running Ubuntu 7.04 in single user mode on a 2-core Intel
> machine. The configuration is standard (profiled production build
> with classpath, dacapo 2006/10). I have not manually started
> additional system services (not even an Xvfb server to run dacapo-
> chart) and, as stated above, encounter the phenomenon on the out-of-
> the-box RVM.
>
> Right now, I am at a loss what might cause these fluctuations, so I
> am interested in any advice/ideas.
>
> Christian Sinschek,
> Technische Universität Darmstadt
> --
> Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit
> allen: http://www.gmx.net/de/go/multimessenger01
>
> ------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances!
> Your
> production scanning environment may not be a perfect world - but
> thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW
> KODAK i700
> Series Scanner you'll get full speed at 300 dpi even with all image
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> Jikesrvm-researchers mailing list
> Jikesrvm-researchers@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers
------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
Jikesrvm-researchers mailing list
Jikesrvm-researchers@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers
There are some very well known sources of performance variation for
managed runtimes, such as Jikes RVM. However, it sounds like you
have accounted for these. It is a little hard to tell though. It
might help if you can provide the following information:
- The *exact* command line used for a specific benchmark
- The *exact* results produced by one of your runs (ideally a log of
the 60 results you report below)
- The *exact* hardware you are running on (you say it is dual core, do
you mean Core 2 Duo?).
You should not see any significant variation if you turn the AOS
off. I do this fairly routinely. Nonetheless, I'd normally take 10
measurements even with the AOS off. WIth the AOS on, I'd be inclined
to take 20 measurements. I don't think you need to time the 10th
iteration. Take a look at the warm-up curves in the right column of
this page (http://dacapo.anu.edu.au/regression/perf/2006-10-MR2.html)
and you'll see that steady state is reached earlier than that. I
typically use the 4th iteration.
Cheers,
--steve
On 07/05/2009, at 10:13 PM, sunai@... wrote:
> This is a troubleshooting question. I am trying to run the dacapo
> benchmarks with an older revision (14775) of Jikes, using the
> ''perf'' test-run (I adapted it to run only the dacapo benchmarks),
> but the measurements turn out to be very unstable. E. g. for dacapo-
> fop running the 9 warum-up + 1 timed iterations for 6 executions
> mostly gives me results within a limited rage (barring 3% of
> variation), but quite a number of measurements (about one fifth) are
> very far off (+>10%). I have attempted a small baseline compiler
> modification to safe some control flow profiling (edge counters);
> when I run the patched VM with this code, all measurements are
> catapulted into the higher ballpark.
>
> I have switched off the AOS recompilation, which apparently also
> ensures that there is no invocation threshold-based recompilation. I
> am running Ubuntu 7.04 in single user mode on a 2-core Intel
> machine. The configuration is standard (profiled production build
> with classpath, dacapo 2006/10). I have not manually started
> additional system services (not even an Xvfb server to run dacapo-
> chart) and, as stated above, encounter the phenomenon on the out-of-
> the-box RVM.
>
> Right now, I am at a loss what might cause these fluctuations, so I
> am interested in any advice/ideas.
>
> Christian Sinschek,
> Technische Universität Darmstadt
> --
> Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit
> allen: http://www.gmx.net/de/go/multimessenger01
>
> ------------------------------------------------------------------------------
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances!
> Your
> production scanning environment may not be a perfect world - but
> thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW
> KODAK i700
> Series Scanner you'll get full speed at 300 dpi even with all image
> processing features enabled. http://p.sf.net/sfu/kodak-com
> _______________________________________________
> Jikesrvm-researchers mailing list
> Jikesrvm-researchers@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers
------------------------------------------------------------------------------
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
_______________________________________________
Jikesrvm-researchers mailing list
Jikesrvm-researchers@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-researchers
« Return to Thread: [rvm-research] Looking for Sources of Performance Variation
| Free embeddable forum powered by Nabble | Forum Help |
