[rvm-core] Jikes RVM 3.1 or 3.2? Syscalls and Java Native Access

[Ian Rogers]

Hi,

I wanted to mail the core team list to discuss what I think are the major works in progress and what should be the major milestones for a significant version number bump.

1) 64bit support for Intel

The baseline compiler has support for 64bit code generation and helloworld will run, GC maps and the opt compiler are broken.

2) Native threads

I believe we're agreed on moving ahead with Fil's SoC work to be the new underlying thread layer for Jikes RVM. I recently resolved merge conflicts with the work on the branch, but changes related to VMAccessController are causing major new regressions. PowerPC support also needs testing for this work.

3) Windows support

I have a patch to enable building Harmony from pre-built binary blobs, I've also hacked up the build.xml to work with MSVC. As this work conflicts with changes from the native threads, I haven't solved problems in the syscalls where the obvious solution is to move to using the apache portable runtime.


There are many other things that need to get sorted (not least BK's JDWP support work) but I think these 3 are the key for a revision bump, as the issues more underly the implementation. With both the native threading and windows support requiring work on the syscalls I wanted to question their implementation before more work was undertaken.

Currently we define syscalls to take an address for the C function that provides their implementation. The address is typically in the boot record and filled in by the boot image runner. Another approach to syscalls is to try to wrap existing libraries with an object that has methods that call the functions within the library. I believe this approach was used by Moxie and Java Native Access [1] (JNA) provides a library that also does this job. By using a JNA style approach most of the syscall implementation can be provided in Java code, and it would be nice to refactor our syscalls to use this approach. An obvious downside is how to handle the bootstrap (presumably some core set of syscalls would need to remain).

What I'm hoping from this discussion is that people can say what they think the syscalls should look like and be before rewriting sys.C to be written using APR. It seems better if we wrap APR with somethin like JNA and then provide the syscall implementation in Java. Experience, particularly from Moxie, would be useful for this.

Thanks,
Ian

[1] http://jna.dev.java.net/

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers]

2009/1/13 Ian Rogers <rogers.email@...>

Hi,

I wanted to mail the core team list to discuss what I think are the major works in progress and what should be the major milestones for a significant version number bump.

1) 64bit support for Intel

The baseline compiler has support for 64bit code generation and helloworld will run, GC maps and the opt compiler are broken.

2) Native threads

I believe we're agreed on moving ahead with Fil's SoC work to be the new underlying thread layer for Jikes RVM. I recently resolved merge conflicts with the work on the branch, but changes related to VMAccessController are causing major new regressions. PowerPC support also needs testing for this work.

3) Windows support

I have a patch to enable building Harmony from pre-built binary blobs, I've also hacked up the build.xml to work with MSVC. As this work conflicts with changes from the native threads, I haven't solved problems in the syscalls where the obvious solution is to move to using the apache portable runtime.


There are many other things that need to get sorted (not least BK's JDWP support work) but I think these 3 are the key for a revision bump, as the issues more underly the implementation. With both the native threading and windows support requiring work on the syscalls I wanted to question their implementation before more work was undertaken.

Currently we define syscalls to take an address for the C function that provides their implementation. The address is typically in the boot record and filled in by the boot image runner. Another approach to syscalls is to try to wrap existing libraries with an object that has methods that call the functions within the library. I believe this approach was used by Moxie and Java Native Access [1] (JNA) provides a library that also does this job. By using a JNA style approach most of the syscall implementation can be provided in Java code, and it would be nice to refactor our syscalls to use this approach. An obvious downside is how to handle the bootstrap (presumably some core set of syscalls would need to remain).

What I'm hoping from this discussion is that people can say what they think the syscalls should look like and be before rewriting sys.C to be written using APR. It seems better if we wrap APR with somethin like JNA and then provide the syscall implementation in Java. Experience, particularly from Moxie, would be useful for this.

Thanks,
Ian

[1] http://jna.dev.java.net/

PearColator has support for generating readers of structures using alignment information gleamed from annotations. This code is CPL. Possibly this could be used in a Jikes RVM JNA like implementation.

Ian

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[David P Grove]

"Ian Rogers" <rogers.email@...> wrote on 01/13/2009 10:46:12 AM:
>
> There are many other things that need to get sorted (not least BK's
> JDWP support work) but I think these 3 are the key for a revision
> bump, as the issues more underly the implementation. With both the
> native threading and windows support requiring work on the syscalls
> I wanted to question their implementation before more work was undertaken.
>
> Currently we define syscalls to take an address for the C function
> that provides their implementation. The address is typically in the
> boot record and filled in by the boot image runner. Another approach
> to syscalls is to try to wrap existing libraries with an object that
> has methods that call the functions within the library. I believe
> this approach was used by Moxie and Java Native Access [1] (JNA)
> provides a library that also does this job. By using a JNA style
> approach most of the syscall implementation can be provided in Java
> code, and it would be nice to refactor our syscalls to use this
> approach. An obvious downside is how to handle the bootstrap
> (presumably some core set of syscalls would need to remain).
>
> What I'm hoping from this discussion is that people can say what
> they think the syscalls should look like and be before rewriting
> sys.C to be written using APR. It seems better if we wrap APR with
> somethin like JNA and then provide the syscall implementation in
> Java. Experience, particularly from Moxie, would be useful for this.

For this particular point, my design suggestion is that at the Java source level, a sysCall is simply a native method with a @SysCall annotation.  The semantics are just "call the ip associated with this symbol, using the OS platform ABI."

To implement calls to a method annotated with a SysCall generated during bootimage writing time we must generate code that indirects to a slot in the bootrecord (or at a fixed slow in some new function table perhaps reachable from the RVMThread/Processor register) to get the function pointer to call.  Any such syscalls must be defined by a class in the bootimage. This table will be filled by generated C code during bootstrap of the runtime.  Any dynamically generated code that calls a sysCall will simply look up the address of the target symbol (like we do for native methods now) to avoid the indirect and to allow us to let users define new syscalls without having to hack them into the JVM (as long as they define the matching symbol in some native library that was properly loaded before the syscall was resolved).  

Half baked, so maybe there are problems I haven't worked out, but I think there could be real value in letting us have a more extensible non-JNI FFI.

--dave

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Steve Blackburn]

I'll add to the list:

4) Make GenImmix the production collector.   I've been meaning to do  
this for ages, but have not had sufficient time to feel brave enough  
to pull the switch.

5) (possibly not for 3.2...) We should try to get ourselves passing  
the dacapo benchmark svn head.  First job is for me to make the error  
messages visible on the dacapo web page.   We're failing quite a few  
of the benchmarks on the head.  It would be nice if our release could  
run all of the workloads once the new dacapo release emerges.

Also, what is the status of the (linux) Harmony libs?  Once they are  
robust enough I'd like to see a performance evaluation.  Anecdotal  
evidence suggests that they may give us a performance boost.

--Steve


------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers]

2009/1/13 Steve Blackburn <Steve.Blackburn@...>

I'll add to the list:

4) Make GenImmix the production collector.   I've been meaning to do
this for ages, but have not had sufficient time to feel brave enough
to pull the switch.

5) (possibly not for 3.2...) We should try to get ourselves passing
the dacapo benchmark svn head.  First job is for me to make the error
messages visible on the dacapo web page.   We're failing quite a few
of the benchmarks on the head.  It would be nice if our release could
run all of the workloads once the new dacapo release emerges.

We're also failing benchmarks on the dacapo web page because we need an X installation (such as chart on trunk).
 

Also, what is the status of the (linux) Harmony libs?  Once they are
robust enough I'd like to see a performance evaluation.  Anecdotal
evidence suggests that they may give us a performance boost.

So the Harmony libs and Jikes RVM will run SpecJVM 2008 but I think they fail bits of DaCapo. I've not tested the SpecJVM tests that need a working javac compiler in tools.jar. The Harmony libs don't support >1 thread, but there seems little point adding support when we're just about to alter the thread model and add support for threading via APR to get Windows running. There are lots of things I'd like to see happen with the Harmony libs, such as using the Harmony launcher, ... I think Alexey and I knocked a number of low hanging issues off, but I'd be more than happy if this was getting more attention. I agree there should be some performance improvements.

Probably the lowest hanging fruit at the moment is working out why the baseline GC map iterator is off and get Intel 64bit with the baseline compiler working :-)

Regards,
Ian
 

--Steve

------------------------------------------------------------------------------
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core