[rvm-core] Nightly sanity tests for native thread branch
|
View:
New views
20 Messages
—
Rating Filter:
Alert me
|
| < Prev | 1 - 2 | Next > |
 |
[rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers
::
Rate this Message:
Hi,
with the push to get the native thread work into the trunk, could we switch to more frequent regression tests of the branch? Currently we get a regression test result every other day, which slows progress. I'd be happy to see other regressions scaled back for there to be enough time on the regression test machines. If we could also start testing on PPC 32 and 64 it would be great (I expect them to break, but at least we can know and track this). Thanks, Ian ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by David P Grove
::
Rate this Message:
"Ian Rogers" <rogers.email@...> wrote on 01/15/2009 03:47:08 PM: |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
I'm not sure what's been changed, talking to Daniel on IM he said we're running native thread regressions every 24hours now. The native thread branch is currently failing with what looks like a problem applying Robin's VMAccessController patch. I hope to fix this soon, but as the native thread regression has started it won't be in the next set of results.
Thanks, Ian 2009/1/16 David P Grove <groved@...>
------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
2009/1/16 Ian Rogers <ian.rogers@...> I'm not sure what's been changed, talking to Daniel on IM he said we're running native thread regressions every 24hours now. The native thread branch is currently failing with what looks like a problem applying Robin's VMAccessController patch. I hope to fix this soon, but as the native thread regression has started it won't be in the next set of results. I put fixes in but as we haven't had a native thread regression test for 3 days I can't see the effect. I'm not sure why we've not had a native thread regression test for so long. Ian
------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Daniel Frampton
::
Rate this Message:
It was running several---I assume there is some error that is causing
timeouts/hangs. I have killed them all off, cleared the crontab and started a new run manually. Cheers, Daniel. On Mon, Jan 19, 2009 at 8:16 PM, Ian Rogers <ian.rogers@...> wrote: > 2009/1/16 Ian Rogers <ian.rogers@...> >> >> I'm not sure what's been changed, talking to Daniel on IM he said we're >> running native thread regressions every 24hours now. The native thread >> branch is currently failing with what looks like a problem applying Robin's >> VMAccessController patch. I hope to fix this soon, but as the native thread >> regression has started it won't be in the next set of results. >> >> Thanks, >> Ian > > I put fixes in but as we haven't had a native thread regression test for 3 > days I can't see the effect. I'm not sure why we've not had a native thread > regression test for so long. > > Ian > > >> >> 2009/1/16 David P Grove <groved@...> >>> >>> "Ian Rogers" <rogers.email@...> wrote on 01/15/2009 03:47:08 PM: >>> I think the easiest thing would be to suspend the quarantine runs and use >>> those slots/machines. >>> >>> --dave >>> >>> > >>> > with the push to get the native thread work into the trunk, could we >>> > switch to more frequent regression tests of the branch? Currently we >>> > get a regression test result every other day, which slows progress. >>> > I'd be happy to see other regressions scaled back for there to be >>> > enough time on the regression test machines. If we could also start >>> > testing on PPC 32 and 64 it would be great (I expect them to break, >>> > but at least we can know and track this). >>> >>> >>> ------------------------------------------------------------------------------ >>> This SF.net email is sponsored by: >>> SourcForge Community >>> SourceForge wants to tell your story. >>> http://p.sf.net/sfu/sf-spreadtheword >>> _______________________________________________ >>> Jikesrvm-core mailing list >>> Jikesrvm-core@... >>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core >>> >> > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > _______________________________________________ > Jikesrvm-core mailing list > Jikesrvm-core@... > https://lists.sourceforge.net/lists/listinfo/jikesrvm-core > > ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Filip Pizlo-2
::
Rate this Message:
I second this.
-F On Jan 15, 2009, at 3:47 PM, Ian Rogers wrote: > Hi, > > with the push to get the native thread work into the trunk, could we > switch to more frequent regression tests of the branch? Currently we > get a regression test result every other day, which slows progress. > I'd be happy to see other regressions scaled back for there to be > enough time on the regression test machines. If we could also start > testing on PPC 32 and 64 it would be great (I expect them to break, > but at least we can know and track this). > > Thanks, > Ian ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
Some tests (e.g. DaCapo antlr with a development build) are freezing. Below is the thread dump:
====== Begin Thread Accounting Dump ====== threadBySlot: 0:none, 1:none, 2:2(1), 3:3(2), 4:4(5), 5:5(2), 6:6(2), 7:7(2), 8: 8(5), 9:9(5), 10:10(2), 11:11(2), 12:12(1) aboutToTerminate: freeSlots: 0:1 threads: 0:12(1), 1:2(1), 2:3(2), 3:4(5), 4:5(2), 5:6(2), 6:7(2), 7:8(5), 8:9(5) , 9:10(2), 10:11(2) ====== End Thread Accounting Dump ====== Timer ticks = 689 (0x00000000000002b1) 12-main-1-RUNNABLE acquireCount for my monitor: 0x093fec90 yieldpoints taken: 3562 (0x0000000000000dea) yieldpoints taken fully: 3562 (0x0000000000000dea) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) (stack trace will follow if thread is not lost...) 2-daemon-1-RUNNABLE acquireCount for my monitor: 75 yieldpoints taken: 0 (0x0000000000000000) yieldpoints taken fully: 0 (0x0000000000000000) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) (stack trace will follow if thread is not lost...) 3-daemon-collector-2-RUNNABLE acquireCount for my monitor: 75 yieldpoints taken: 43 (0x000000000000002b) yieldpoints taken fully: 0 (0x0000000000000000) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x7001cd20, 0x646ed5d7] Lorg/jikesrvm/mm/mminterface/Handshake; parkCollectorThread()V at line 118 at [0x7001cd60, 0x64ce8c58] Lorg/jikesrvm/mm/mminterface/CollectorThread; run()V at line 332 at [0x7001cd98, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 4-daemon-5-RUNNABLE acquireCount for my monitor: 334 yieldpoints taken: 0 (0x0000000000000000) yieldpoints taken fully: 0 (0x0000000000000000) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x7003a064, 0x64ce8c58] Lorg/jikesrvm/mm/mminterface/ConcurrentCollectorThread; run()V at line 129 at [0x7003a09c, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 5-daemon-2-RUNNABLE acquireCount for my monitor: 2134 yieldpoints taken: 576 (0x0000000000000240) yieldpoints taken fully: 576 (0x0000000000000240) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x714eafc0, 0x64ce8c58] Lorg/jikesrvm/scheduler/FinalizerThread; run()V at line 72 at [0x714eaff8, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 6-daemon-2-WAITING acquireCount for my monitor: 5156 yieldpoints taken: 1643 (0x000000000000066b) yieldpoints taken fully: 1515 (0x00000000000005eb) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x71560f24, 0x64830173] Lorg/jikesrvm/scheduler/RVMThread; waitImpl(Ljava/lang/Object;ZJ)V at line 2139 at [0x71560f68, 0x646faa2a] Lorg/jikesrvm/scheduler/RVMThread; wait(Ljava/lang/Object;)V at line 2188 at [0x71560f68, 0x646faa2a] Ljava/lang/Object; wait()V at line 66 at [0x71560f68, 0x646faa2a] Lorg/jikesrvm/adaptive/util/BlockingPriorityQueue; deleteMin()Ljava/lang/Object; at line 80 at [0x71560fc0, 0x64ce8c58] Lorg/jikesrvm/adaptive/controller/ControllerThread; run()V at line 157 at [0x71560ff8, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 7-daemon-2-RUNNABLE acquireCount for my monitor: 5261 yieldpoints taken: 1618 (0x0000000000000652) yieldpoints taken fully: 1618 (0x0000000000000652) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x715d6f50, 0x646ef017] Lorg/jikesrvm/scheduler/Latch; waitAndClose()V at line 90 at [0x715d6f6c, 0x646ef0d6] Lorg/jikesrvm/adaptive/measurements/organizers/Organizer; passivate()V at line 91 at [0x715d6fc0, 0x64ce8c58] Lorg/jikesrvm/adaptive/measurements/organizers/Organizer; run()V at line 52 at [0x715d6ff8, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 8-daemon-5-RUNNABLE acquireCount for my monitor: 381 yieldpoints taken: 15 (0x000000000000000f) yieldpoints taken fully: 15 (0x000000000000000f) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x7164cf50, 0x646ef017] Lorg/jikesrvm/scheduler/Latch; waitAndClose()V at line 90 at [0x7164cf6c, 0x646ef0d6] Lorg/jikesrvm/adaptive/measurements/organizers/Organizer; passivate()V at line 91 at [0x7164cfc0, 0x64ce8c58] Lorg/jikesrvm/adaptive/measurements/organizers/Organizer; run()V at line 52 at [0x7164cff8, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 9-daemon-5-RUNNABLE acquireCount for my monitor: 1458 yieldpoints taken: 360 (0x0000000000000168) yieldpoints taken fully: 360 (0x0000000000000168) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x716c2f50, 0x646ef017] Lorg/jikesrvm/scheduler/Latch; waitAndClose()V at line 90 at [0x716c2f6c, 0x646ef0d6] Lorg/jikesrvm/adaptive/measurements/organizers/Organizer; passivate()V at line 91 at [0x716c2fc0, 0x64ce8c58] Lorg/jikesrvm/adaptive/measurements/organizers/Organizer; run()V at line 52 at [0x716c2ff8, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 10-daemon-2-RUNNABLE acquireCount for my monitor: 674 yieldpoints taken: 76 (0x000000000000004c) yieldpoints taken fully: 76 (0x000000000000004c) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x71738fc0, 0x64ce8c58] Lorg/jikesrvm/adaptive/OSROrganizerThread; run()V at line 43 at [0x71738ff8, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 11-daemon-2-WAITING acquireCount for my monitor: 2406 yieldpoints taken: 703 (0x00000000000002bf) yieldpoints taken fully: 646 (0x0000000000000286) native entered blocked: 0 (0x0000000000000000) JNI entered blocked: 0 (0x0000000000000000) -- Stack -- at [0x717aef60, 0x64830173] Lorg/jikesrvm/scheduler/RVMThread; waitImpl(Ljava/lang/Object;ZJ)V at line 2139 at [0x717aefa4, 0x646eeee8] Lorg/jikesrvm/scheduler/RVMThread; wait(Ljava/lang/Object;)V at line 2188 at [0x717aefa4, 0x646eeee8] Ljava/lang/Object; wait()V at line 66 at [0x717aefa4, 0x646eeee8] Lorg/jikesrvm/adaptive/util/BlockingPriorityQueue; deleteMin()Ljava/lang/Object; at line 80 at [0x717aefc0, 0x64ce8c58] Lorg/jikesrvm/adaptive/recompilation/CompilationThread; run()V at line 50 at [0x717aeff8, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 Handling async stack trace request... 12-main-4-RUNNABLE Dumping stack for Thread #12 -- Stack -- at [0x71874b28, 0x646d847e] Lorg/jikesrvm/scheduler/RVMThread; handleHandshakeRequest()V at line 2530 at [0x71874b6c, 0x64a46daf] Lorg/jikesrvm/scheduler/RVMThread; checkBlockNoSaveContext()V at line 1199 at [0x71874b90, 0x70837f30] Lorg/jikesrvm/scheduler/RVMThread; leaveJNIBlockedFromCallIntoNative()V at line 1374 at [0x71874b90, 0x70837f30] Lorg/jikesrvm/jni/JNIEnvironment; exitFromJNI(I)Ljava/lang/Object; at line 372 at [0x71874bbc, 0x640b3b2c] Lgnu/java/nio/VMChannel; close(I)V at [0x71874be4, 0x640b9549] Lgnu/java/nio/VMChannel$State; close()V at line 698 at [0x71874c00, 0x641293b2] Lgnu/java/nio/VMChannel; close()V at line 625 at [0x71874c00, 0x641293b2] Lgnu/java/nio/FileChannelImpl; implCloseChannel()V at line 218 at [0x71874c00, 0x641293b2] Ljava/nio/channels/spi/AbstractInterruptibleChannel; close()V at line 80 at [0x71874c00, 0x641293b2] Ljava/io/FileOutputStream; close()V at line 295 at [0x71874c20, 0x708cd065] Ljava/io/OutputStreamWriter; close()V at line 271 at [0x71874c70, 0x64125f2f] Lantlr/PreservingFileWriter; close()V at line 59 at [0x71874c90, 0x70973140] Ljava/io/PrintWriter; close()V at line 259 at [0x71874cc4, 0x7095a31c] Lantlr/JavaCodeGenerator; genTokenTypes(Lantlr/TokenManager;)V at line 2970 at [0x71874cf0, 0x70848a24] Lantlr/JavaCodeGenerator; gen()V at line 124 at [0x71874d3c, 0x7083df4b] Lantlr/Tool; doEverything([Ljava/lang/String;)I at line 254 at [0x71874d60, 0x708634b5] Lantlr/Tool; main([Ljava/lang/String;)V at line 399 at [0x71874da0, 0x7086cc14] Ldacapo/antlr/AntlrHarness; iterate(Ljava/lang/String;)V at line 71 at [0x71874dd0, 0x70847f9f] Ldacapo/Benchmark; run(Ldacapo/Callback;Ljava/lang/String;Z)Z at line 126 at [0x71874e1c, 0x70851d43] Ldacapo/TestHarness; runBenchmark(Ljava/io/File;Ljava/lang/String;Ldacapo/TestHarness;)V at line 302 at [0x71874e58, 0x7083720a] Ldacapo/TestHarness; main([Ljava/lang/String;)V at line 242 at [0x71874e74, 0x64b8a93a] LHarness; main([Ljava/lang/String;)V at line 5 at [0x71874e8c, 0x6499db8b] <invisible method> at [0x71874f1c, 0x64833ada] Lorg/jikesrvm/runtime/Reflection; outOfLineInvoke(Lorg/jikesrvm/classloader/RVMMethod;Ljava/lang/Object;[Ljava/lang/Object;Z)Ljava/lang/Object; at line 186 at [0x71874f7c, 0x64a472d7] Lorg/jikesrvm/runtime/Reflection; invoke(Lorg/jikesrvm/classloader/RVMMethod;Lorg/jikesrvm/runtime/ReflectionBase;Ljava/lang/Object;[Ljava/lang/Object;Z)Ljava/lang/Object; at line 77 at [0x71874f7c, 0x64a472d7] Lorg/jikesrvm/scheduler/MainThread; run()V at line 202 at [0x71874fc0, 0x64ce8c58] Lorg/jikesrvm/scheduler/RVMThread; run()V at line 1720 at [0x71874ff8, 0x080500b5] Lorg/jikesrvm/scheduler/RVMThread; startoff()V at line 1758 Ian 2009/1/19 Filip Pizlo <pizlo@...> I second this. ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Steve Blackburn
::
Rate this Message:
On 20/01/2009, at 8:22 AM, Filip Pizlo wrote:
> I second this. What are you seconding, Fil? >> with the push to get the native thread work into the trunk, could we >> switch to more frequent regression tests of the branch? Currently we >> get a regression test result every other day, which slows progress. No, they are running every 24 hours. As it is you have a complete machine to this branch 24/7. The branch right now is unstable enough that we're getting cascading time overruns. If you want you can cut back on what is being tested. >> I'd be happy to see other regressions scaled back for there to be >> enough time on the regression test machines. Sure. You can do that Ian, if you and Fil would like. I have no objections. However, fixing the remaining bugs that surfaced after the latest merge may be a better route. >> If we could also start >> testing on PPC 32 and 64 it would be great (I expect them to break, >> but at least we can know and track this). It is a big waste of resources to run lots of tests we know are going to fail. Better to find a test set that fairly closely reflects the state of play (aim for, say, a 90% pass rate so it is telling you something about what is failing and what is passing, but is not dominated by failures). Then ramp that test set up. Do we know anything about the state on PPC? I thought this branch was not working at all on PPC. --Steve ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
2009/1/19 Steve Blackburn <Steve.Blackburn@...> On 20/01/2009, at 8:22 AM, Filip Pizlo wrote: I did a quick untested port of the JNI compiler changes to PPC. I haven't ported the changes to libvm.c. Anyone with knowledge of C should be able to do that port. Regards, Ian ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
Looking at the dump more deeply, VMChannel.close is trying to leave JNI but it has been asked to block (presumably for an in progress GC). It appears this is never getting cleared. So the tests are hanging when GC occurs whilst a JNI call is in progress. We now have 1 regression run that shows the native threads are nearly back at the point Fil had them during the SoC:
http://jikesrvm.anu.edu.au/cattrack/results/kumataka.anu.edu.au/sanity-nativethreads/7357/regression_report I believe there is an understanding that no significant changes are to be made to the trunk until we can get this work merged in. Regards, Ian 2009/1/19 Ian Rogers <ian.rogers@...> Some tests (e.g. DaCapo antlr with a development build) are freezing. Below is the thread dump: ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Steve Blackburn
::
Rate this Message:
On 20/01/2009, at 9:28 PM, Ian Rogers wrote: Looking at the dump more deeply, VMChannel.close is trying to leave JNI but it has been asked to block (presumably for an in progress GC). It appears this is never getting cleared. So the tests are hanging when GC occurs whilst a JNI call is in progress. We now have 1 regression run that shows the native threads are nearly back at the point Fil had them during the SoC: OK. This is great work. Thank you Ian. As of today, there appear to still be six builds that are no longer running (0% @ 19 Jan) that were at least "somewhat" working prior to the merge (> 0% at 9 Jan): BaseBaseConcMS FullAdaptiveConcMS BaseBaseMarkCompact FullAdaptiveMarkCompact FullAdaptiveStickyImmix FullAdaptiveStickyMS My reading of the info in cattrack is that FullAdaptiveStickyMS failed due to a random HotSpot failure during build. What is the story with the other five? Have they just been culled from the nightly test? It's not obvious to me why they are at 0%---except that they don't appear to have been built. If things are otherwise stable, we should probably bring those five builds back. --Steve
------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
2009/1/20 Steve Blackburn <Steve.Blackburn@...>
Hi Steve, with the exception of FullAdaptiveStickyMS these have gone to 0% because we're running sanity regressions on the native thread branch and these configs were moved into the sanity-tier2 test-run. We could add these tests back to sanity on the native thread branch or also run the sanity-tier2 test-run on the branch. Regards, Ian
------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Steve Blackburn
::
Rate this Message:
On 20/01/2009, at 9:55 PM, Ian Rogers wrote: > with the exception of FullAdaptiveStickyMS these have gone to 0% > because we're running sanity regressions on the native thread branch > and these configs were moved into the sanity-tier2 test-run. We > could add these tests back to sanity on the native thread branch or > also run the sanity-tier2 test-run on the branch. I wondered that. The change happened only recently (since Jan 9), a long time after the change of the composition of sanity, so I'm a bit confused. Perhaps the files on the nativethread test machine were for some reason not updated. Anyway, no big deal. --Steve ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
2009/1/20 Steve Blackburn <Steve.Blackburn@...>
It's no big mystery. Fil updated his branch to a release prior to my 64bit Intel work. A change to the sanity regression was made to trunk. On Jan 9th I merged changes from the trunk into Fil's native thread branch, which included the change to the sanity test-run. I was more focused on solving the merge conflicts in the JNI compiler (Fil changed how the IN_JAVA to IN_JNI transition occured, I made the JNI compiler 64bit and utilized a helper routine to enter and leave JNI code). Regards, Ian ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
So the fix is that when leaving JNI the exit code shouldn't loop if the transition of IN_JNI to IN_JAVA fails. I'm not sure this is sensible in particular with PowerPC ll/sc, but I'll put a fix in and add more assertions.
Regards, Ian 2009/1/20 Ian Rogers <ian.rogers@...>
------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
There's a fix in r15290 but it seems the machine is locked up as we've not had regression results in two days. Could someone kill any runs on pre r15290 and restart them (it'll probably be Daniel, so thanks to Daniel in advance if it is).
The native thread branch is not performing state transitions as we do in the green thread code. In the green thread code a helper method changeThreadState will transition from one state to another, the purpose of this is many: (1) it documents the transitions of state (2) it catches unexpected behaviour (3) consequently it helps find bugs (the code is less brittle). Adding basic assertions to some of the code showed that not doing things in this way is a serious regression, and the benchmarks that would fail the most obvious assertions were lusearch and xalan (that are failing in general on the native thread branch). There are clearly more jobs to do before this code is ready to merge with the trunk, but I understand this is everyone's priority. I've started adding sub-tasks to RVM-91 [1], if people could add to them and start knocking them off it'd be great. Thanks, Ian [1] http://jira.codehaus.org/browse/RVM-91 2009/1/20 Ian Rogers <ian.rogers@...> So the fix is that when leaving JNI the exit code shouldn't loop if the transition of IN_JNI to IN_JAVA fails. I'm not sure this is sensible in particular with PowerPC ll/sc, but I'll put a fix in and add more assertions. ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Daniel Frampton
::
Rate this Message:
The most recent run [1] is from r15287, and is from when I manually
kicked it off on Monday. I have manually kicked off another run, if it also reports back in under 24 hours we can start automating the daily runs again. Cheers, Daniel. [1] http://jikesrvm/cattrack/results/kumataka.anu.edu.au/sanity-nativethreads/7357 On Thu, Jan 22, 2009 at 10:30 AM, Ian Rogers <ian.rogers@...> wrote: > There's a fix in r15290 but it seems the machine is locked up as we've not > had regression results in two days. Could someone kill any runs on pre > r15290 and restart them (it'll probably be Daniel, so thanks to Daniel in > advance if it is). > > The native thread branch is not performing state transitions as we do in the > green thread code. In the green thread code a helper method > changeThreadState will transition from one state to another, the purpose of > this is many: (1) it documents the transitions of state (2) it catches > unexpected behaviour (3) consequently it helps find bugs (the code is less > brittle). Adding basic assertions to some of the code showed that not doing > things in this way is a serious regression, and the benchmarks that would > fail the most obvious assertions were lusearch and xalan (that are failing > in general on the native thread branch). > > There are clearly more jobs to do before this code is ready to merge with > the trunk, but I understand this is everyone's priority. I've started adding > sub-tasks to RVM-91 [1], if people could add to them and start knocking them > off it'd be great. > > Thanks, > Ian > > [1] http://jira.codehaus.org/browse/RVM-91 > > 2009/1/20 Ian Rogers <ian.rogers@...> >> >> So the fix is that when leaving JNI the exit code shouldn't loop if the >> transition of IN_JNI to IN_JAVA fails. I'm not sure this is sensible in >> particular with PowerPC ll/sc, but I'll put a fix in and add more >> assertions. >> >> Regards, >> Ian >> >> 2009/1/20 Ian Rogers <ian.rogers@...> >>> >>> 2009/1/20 Steve Blackburn <Steve.Blackburn@...> >>>> >>>> On 20/01/2009, at 9:55 PM, Ian Rogers wrote: >>>> >>>> > with the exception of FullAdaptiveStickyMS these have gone to 0% >>>> > because we're running sanity regressions on the native thread branch >>>> > and these configs were moved into the sanity-tier2 test-run. We >>>> > could add these tests back to sanity on the native thread branch or >>>> > also run the sanity-tier2 test-run on the branch. >>>> >>>> I wondered that. The change happened only recently (since Jan 9), a >>>> long time after the change of the composition of sanity, so I'm a bit >>>> confused. Perhaps the files on the nativethread test machine were >>>> for some reason not updated. Anyway, no big deal. >>>> >>>> --Steve >>> >>> It's no big mystery. Fil updated his branch to a release prior to my >>> 64bit Intel work. A change to the sanity regression was made to trunk. On >>> Jan 9th I merged changes from the trunk into Fil's native thread branch, >>> which included the change to the sanity test-run. I was more focused on >>> solving the merge conflicts in the JNI compiler (Fil changed how the IN_JAVA >>> to IN_JNI transition occured, I made the JNI compiler 64bit and utilized a >>> helper routine to enter and leave JNI code). >>> >>> Regards, >>> Ian >> > > > ------------------------------------------------------------------------------ > This SF.net email is sponsored by: > SourcForge Community > SourceForge wants to tell your story. > http://p.sf.net/sfu/sf-spreadtheword > _______________________________________________ > Jikesrvm-core mailing list > Jikesrvm-core@... > https://lists.sourceforge.net/lists/listinfo/jikesrvm-core > > ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Ian Rogers (nabble)
::
Rate this Message:
2009/1/21 Ian Rogers <ian.rogers@...> There's a fix in r15290 but it seems the machine is locked up as we've not had regression results in two days. Could someone kill any runs on pre r15290 and restart them (it'll probably be Daniel, so thanks to Daniel in advance if it is). Just to expand on my concerns, one thing that bothers me in the native branch is the IN_NATIVE state: 1) the javadoc implies it is a state transition that a call to a syscall will cause to occur, this isn't the case code voluntarily switches to this state, VMMath.sin won't transition the thread state therefore 2) the javadoc states that it is a GC safe point, this needs a huge health warning about the use of references that should only be to objects that aren't going to move 3) the name seems wrong as it appears to have more to do with the thread being at a safe point than in native code Code like enterNative [1] doesn't help (although this is slightly cleaned up), there's no javadoc or assertions to tell me what state the thread is going to be in following the call, could the state legitimately be blocked? Who will clean up the mess if it is? Ian [1] http://jikesrvm.svn.sourceforge.net/viewvc/jikesrvm/rvmroot/branches/RVM-PureNativeThread/workingMergeUp/rvm/src/org/jikesrvm/scheduler/RVMThread.java?revision=15290&view=markup#l_1709 2009/1/20 Ian Rogers <ian.rogers@...> So the fix is that when leaving JNI the exit code shouldn't loop if the transition of IN_JNI to IN_JAVA fails. I'm not sure this is sensible in particular with PowerPC ll/sc, but I'll put a fix in and add more assertions. ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by David P Grove
::
Rate this Message:
> Just to expand on my concerns, one thing that bothers me in the |
|
|
Re: [rvm-core] Nightly sanity tests for native thread branch
by Filip Pizlo-2
::
Rate this Message:
Hi Ian,
Thanks for taking a look at this. I think that some of your concerns are due to nomenclature, and you're probably right that the documentation is thin in this particular part of the system. To address your concerns: - You're right, the native thread branch performs state transitions in a very different way. That's because, unlike the green threads code, the states in the native threads code are *not* for scheduling, in the sense that the scheduling is left entirely up to the OS. They are almost exclusively for communicating with the GC. As such, there are fewer states, and the state transitions occur in far fewer places. In particular, the transition to IN_NATIVE only occurs deep within the code. Thus, while some additional assertions might be put in, the need for assertions is much smaller, as the transitions are only being done from highly privileged internal code. - You're absolutely right about IN_NATIVE being incorrectly documented. I'll try to fix that - see the patch below. In particular, it's exactly true that the transition to IN_NATIVE is supposed to *only* happen from deep in the native thread code, and most of the VM code should never do it directly. As well, the IN_NATIVE name may indeed be the wrong one, probably something like IN_PRIVILEGED, or IN_GC_SAFE, would be better. I like IN_PRIVILEGED the best, as it does the best job of conveying that this is a very special state, not to be used lightly. - I'm not sure about your concerns about enterNative(). To be clear, the state prior to the call can be IN_JAVA or IN_JAVA_TO_BLOCK; the latter occurs if the GC had already requested the thread to block. After the call, the state could be IN_NATIVE or BLOCKED_IN_NATIVE, depending on whether the previous state was IN_JAVA or IN_JAVA_TO_BLOCK, respectively. This is more complicated than the green threads code, and necessarily so - the green threads code was badly broken in the case of GC requests that occur after the last safepoint before entry into JNI code, and furthermore, the green threads code did not have a clean mechanism of idling a VP if no threads are runnable, which is exactly what the IN_NATIVE mechanism is for. I've attached a patch with further documentation of the situation. I'd like to also change the IN_NATIVE state to something like IN_PRIVILEGED, but I'm not sure that this is a change worth making, and if it is, I'm not sure that IN_PRIVILEGED does a better job of conveying the semantics. Let me know to what extent this addresses your concerns. I understand that this is hairy code - but it's the best I could come up with given the multitude of concerns it has to deal with (PPC isync, mutator flushes, suspend requests, park/wait/sleep, GC requests, etc.). -F Index: rvm/src/org/jikesrvm/scheduler/RVMThread.java =================================================================== --- rvm/src/org/jikesrvm/scheduler/RVMThread.java (revision 15291) +++ rvm/src/org/jikesrvm/scheduler/RVMThread.java (working copy) @@ -77,6 +77,70 @@ /** * A generic java thread's execution context. + * <p> + * Threads use a state machine to indicate to other threads, as well as VM + * services, how this thread should be treated in the case of an asynchronous + * request, for example in the case of GC. The state machine uses the + * following states: + * <ul> + * <li>NEW</li> + * <li>IN_JAVA</li> + * <li>IN_NATIVE</li> + * <li>IN_JNI</li> + * <li>IN_JAVA_TO_BLOCK</li> + * <li>BLOCKED_IN_NATIVE</li> + * <li>BLOCKED_IN_JNI</li> + * <li>TERMINATED</li> + * </ul> + * The following state transitions are legal: + * <ul> + * <li>NEW to IN_JAVA: occurs when the thread is actually started. At this + * point it is safe to expect that the thread will reach a safe point in + * some bounded amount of time, at which point it will have a complete + * execution context, and this will be able to have its stack traces by GC.</li> + * <li>IN_JAVA to IN_JAVA_TO_BLOCK: occurs when an asynchronous request is + * made, for example to stop for GC, do a mutator flush, or do an isync on PPC.</li> + * <li>IN_JAVA to IN_NATIVE: occurs when the code opts to run in privileged mode, + * without synchronizing with GC. This state transition is only performed by + * HeavyCondLock, in cases where the thread is about to go idle while waiting + * for notifications (such as in the case of park, wait, or sleep).</li> + * <li>IN_JAVA to IN_JNI: occurs in response to a JNI downcall, or return from a JNI + * upcall.</li> + * <li>IN_JAVA_TO_BLOCK to BLOCKED_IN_NATIVE: occurs when a thread that had been + * asked to perform an async activity decides to go idle instead. This state + * always corresponds to a notification being sent to other threads, letting + * them know that this thread is idle. When the thread is idle, any asynchronous + * requests (such as mutator flushes) can instead be performed on behalf of this + * thread by other threads, since this thread is guaranteed not to be running + * any user Java code, and will not be able to return to running Java code without + * first blocking, and waiting to be unblocked (see BLOCKED_IN_NATIVE to IN_JAVA + * transition.</li> + * <li>IN_JAVA_TO_BLOCK to BLOCKED_IN_JNI: occurs when a thread that had been + * asked to perform an async activity decides to make a JNI downcall, or return + * from a JNI upcall, instead. In all other regards, this is identical to the + * IN_JAVA_TO_BLOCK to BLOCKED_IN_NATIVE transition.</li> + * <li>IN_NATIVE to IN_JAVA: occurs when a thread returns from idling or running + * privileged code to running Java code.</li> + * <li>BLOCKED_IN_NATIVE to IN_JAVA: occurs when a thread that had been asked to + * perform an async activity while running privileged code or idling decides to + * go back to running Java code. The actual transition is preceded by the + * thread first performing any requested actions (such as mutator flushes) and + * waiting for a notification that it is safe to continue running (for example, + * the thread may wait until GC is finished).</li> + * <li>IN_JNI to IN_JAVA: occurs when a thread returns from a JNI downcall, or makes + * a JNI upcall.</li> + * <li>BLOCKED_IN_JNI to IN_JAVA: same as BLOCKED_IN_NATIVE to IN_JAVA, except that + * this occurs in response to a return from a JNI downcall, or as the thread + * makes a JNI upcall.</li> + * <li>IN_JAVA to TERMINATED: the thread has terminated, and will never reach any + * more safe points, and thus will not be able to respond to any more requests + * for async activities.</li> + * </ul> + * Observe that the transitions from BLOCKED_IN_NATIVE and BLOCKED_IN_JNI to IN_JAVA + * constitute a safe point. Code running in BLOCKED_IN_NATIVE or BLOCKED_IN_JNI is + * "GC safe" but is not quite at a safe point; safe points are special in that + * they allow the thread to perform async activities (such as mutator flushes or + * isyncs), while GC safe code will not necessarily perform either. * * @see org.jikesrvm.scheduler.greenthreads.GreenThread * @see org.jikesrvm.mm.mminterface.CollectorThread @@ -129,7 +193,8 @@ /* * definitions for thread status for interaction of Java-native transitions - * and requests for threads to stop. + * and requests for threads to stop. THESE ARE PRIVATE TO THE SCHEDULER, and + * are only used deep within the stack. */ /** * Thread has not yet started. This state holds right up until just before we @@ -143,34 +208,50 @@ public static final int IN_JAVA = 1; /** - * A state used by the scheduler to mark that a thread is in native code (a - * syscall). The point is that for now, the thread is not executing Java code - * and is effectively at a safe point, but it may transition back into Java - * code at any moment. + * A state used by the scheduler to mark that a thread is in privileged code + * that does not need to synchronize with the collector. This is a special + * state, similar to the IN_JNI state but requiring different interaction with + * the collector (as there is no JNI stack frame, the registers have to be + * saved in contextRegisters). As well, this state should only be entered + * from privileged code in the org.jikesrvm.scheduler package. Typically, + * this state is entered using a call to enterNative() just prior to idling + * the thread; though it would not be wrong to enter it prior to any other + * long-running activity that does not require interaction with the GC. */ public static final int IN_NATIVE = 2; /** - * A thread is executing native code and is effectively at a GC safe point. - * Care must be taken if GC occurs and the native code execution finishes. + * Same as IN_NATIVE, except that we're executing JNI code and thus have a + * JNI stack frame and JNI environment, and thus the GC can load registers + * from there rather than using contextRegisters. */ public static final int IN_JNI = 3; /** - * thread is in Java code but is expected to block. the point is that we're - * waiting for the thread to reach a safe point and expect this to happen in - * bounded time; but if the thread were to escape to native we want to know - * about it. thus, transitions into native code while in the IN_JAVA_TO_BLOCK - * state result in a notification (broadcast on the thread's monitor) and a - * state change to BLOCKED_IN_NATIVE. Observe that it is always safe to - * conservatively change IN_JAVA to IN_JAVA_TO_BLOCK. + * thread is in Java code but is expected to block. the transition from IN_JAVA + * to IN_jAVA_TO_BLOCK happens as a result of an asynchronous call by the GC + * or any other internal VM code that requires this thread to perform an + * asynchronous activity (another example is the request to do an isync on PPC). + * the point is that we're waiting for the thread to reach a safe point and + * expect this to happen in bounded time; but if the thread were to escape to + * native we want to know about it. thus, transitions into native code while + * in the IN_JAVA_TO_BLOCK state result in a notification (broadcast on the + * thread's monitor) and a state change to BLOCKED_IN_NATIVE. Observe that it + * is always safe to conservatively change IN_JAVA to IN_JAVA_TO_BLOCK. */ public static final int IN_JAVA_TO_BLOCK = 4; /** * thread is in native code, and is to block before returning to Java code. - * the point is that the thread is guaranteed not to execute any Java code - * until: + * the transition from IN_NATIVE to BLOCKED_IN_NATIVE happens as a result + * of an asynchronous call by the GC or any other internal VM code that + * requires this thread to perform an asynchronous activity (another example + * is the request to do an isync on PPC). as well, code entering privileged + * code that would otherwise go from IN_JAVA to IN_NATIVE will go to + * BLOCKED_IN_NATIVE instead, if the state was IN_JAVA_TO_BLOCK. + * <p> + * the point of this state is that the thread is guaranteed not to execute + * any Java code until: * <ol> * <li>The state changes to IN_NATIVE, and * <li>The thread gets a broadcast on its monitor. @@ -1393,9 +1474,8 @@ } if (traceBlock) - VM - .sysWriteln("Thread #", threadSlot, - " has acknowledged soft handshakes"); + VM.sysWriteln("Thread #", threadSlot, + " has acknowledged soft handshakes"); for (;;) { // deal with block requests @@ -1706,6 +1786,20 @@ return block(ba, false); } + /** + * Indicate that we'd like the current thread to be executing privileged code that + * does not require synchronization with the GC. This call may be made on a thread + * that is IN_JAVA or IN_JAVA_TO_BLOCK, and will result in the thread being either + * IN_NATIVE or BLOCKED_IN_NATIVE. In the case of an + * IN_JAVA_TO_BLOCK->BLOCKED_IN_NATIVE transition, this call will acquire the + * thread's lock and send out a notification to any threads waiting for this thread + * to reach a safepoint. This notification serves to notify them that the thread + * is in GC-safe code, but will not reach an actual safepoint for an indetermined + * amount of time. This is significant, because safepoints may perform additional + * actions (such as handling handshake requests, which may include things like + * mutator flushes and running isync) that IN_NATIVE code will not perform until + * returning to IN_JAVA by way of a leaveNative() call. + */ public static void enterNative() { RVMThread t = getCurrentThread(); if (ALWAYS_LOCK_ON_STATE_TRANSITION) { @@ -1721,6 +1815,9 @@ if (VM.VerifyAssertions) VM._assert(oldState == IN_JAVA_TO_BLOCK); t.enterNativeBlocked(); return; + } else { + VM.sysFail("entering native with wrong exec status"); + return; // make javac happy } } while (!(Synchronization.tryCompareAndSwap(t, offset, oldState, newState))); } @@ -1747,11 +1844,20 @@ if (VM.VerifyAssertions) VM._assert(oldState == BLOCKED_IN_NATIVE || oldState == BLOCKED_IN_JNI); return false; + } else { + VM.sysFail("leaving native with wrong exec status"); + return true; // make javac happy } } while (!(Synchronization.tryCompareAndSwap(t, offset, oldState, newState))); return true; } + /** + * Leave privileged code. This is valid for threads that are either IN_NATIVE, + * IN_JNI, BLOCKED_IN_NATIVE, or BLOCKED_IN_JNI, and always results in the thread + * being IN_JAVA. If the thread was previously BLOCKED_IN_NATIVE or BLOCKED_IN_JNI, + * the thread will block until notified that it can run again. + */ @Unpreemptible("May block if the thread was asked to do so; otherwise does no actions that would lead to blocking") public static void leaveNative() { if (!attemptLeaveNativeNoBlock()) { On Jan 21, 2009, at 7:18 PM, Ian Rogers wrote:
------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Jikesrvm-core mailing list Jikesrvm-core@... https://lists.sourceforge.net/lists/listinfo/jikesrvm-core |
| < Prev | 1 - 2 | Next > |
| Free embeddable forum powered by Nabble | Forum Help |
