[rvm-core] The future of the thread model, interactions with class libraries

[Ian Rogers (nabble)]

Hi,

so the preliminaries:

1) native threads should be the Jikes RVM default!
2) great work Fil!

There is a desire to have green threading again (at least from me) and
to get Harmony running again, so let's play out some examples of how
to say support the sleep operation with different thread models
(ignoring cleaning up the arguments and throwing any exceptions that
may have been flagged while we were asleep):

greentheads - take the running thread, put some time information into
it (or a proxy - I forget) and then put it on the waiting queue,
schedule other work

pthreads - take the thread, get a pthread cond lock, do a syscall to
do a timed wait on the pthread cond lock

harmony threads - take the thread, call through to a harmony interface
that abstracts the sleep

so what abstractions do we need: things that are processor local (ie.
processors we used to have these, they should probably be an interface
to make native threads efficient), a singleton thing to ask global
questions to like how many locks are there (ie. the scheduler we used
to have), different thread model implementations. I see the packages
of:

org.jikesrvm.scheduler

this is the interface to the rest of the world, nothing externally
should be accessing packages beneath here

org.jikesrvm.scheduler.greenthreads - the implementation for
greenthreads (queues, etc.) that we had for GNU Classpath, to fully
support this with Harmony requires creating a DLL wrapper. Support for
Classpath requires the syswrap hack.

org.jikesrvm.scheduler.nativethreads.pthreads - Fil's work on
providing a full pthread model, cond locks won't be exposed above
here.

org.jikesrvm.scheduler.nativethreads.harmony - As with the original
native thread goal, just punt to a native thread library, ie Harmony
threads

There is likely to be a chance to share things, like queues, between
the pthread and green thread code, we can therefore have a utility
directory. We should probably not have a lot of different lock or
other implementations in the top-level scheduler package, they can be
pushed down into other packages.

I believe this proposal is controversial as it's bringing back the
interface as it was around 3.0.1. I don't propose we copy and paste
that code, but the structure will look like it from the outside. Feel
free to flame me about why the old interface was the stupidest thing
ever as I would like to shake out what are myths and realities (it's
not a secret that the old thread model wasn't 100% complete).

Can we implement the Harmony code a different way? Yes, I will try
this in the short-term. We can implement the Harmony threads using the
Apache Portable Runtime. This is diving under the feet of the harmony
thread implementation. I believe it is best in the long-term we use
the Harmony interface for Harmony, as it means we are abdicating
responsibility for threading to Harmony and Harmony can give us
compatibility on all the platforms Harmony supports (we are a Java VM,
we should run on all platforms as Java is portable). I could also try
emulating pthread cond locks with Harmony's Java like monitors, but
this would be vile.

What will break? AIX ;-) Nothing should break, as this change is just
*adding functionality*. The build can be configured to use the models
that are complete for the particular architecture. Long-term I'd
expect Harmony native threads to be the default. Green threads has a
user base. Native pthreads for Classpath will be there for the curious
and platforms like AIX where Harmony may not build straight away.

What about OpenJDK? Depending on the abstraction it will probably
closely resemble either Harmony or pthreads, my belief is probably
more like pthreads. Having the flexibility in our model should mean
little work in matching OpenJDKs thread model to ours.

What does this mean for 3.1? I think the thread model is in a state of
flux and it's not in the place it should be. I'd be tempted to make
the next release 3.0.2 and bring back the green thread model
temporarily (maybe keep native threads on the trunk and revert them on
the 3.0.2 branch). I think the current code exposes too many pthread
internals and this will be a portability issue for users interfacing
with the thread library. Ideally the changes above can be written
before the next release and then it could be 3.1, this may be pushing
things (there are still many outstanding pthread JIRAs without the
addition of native threads for Harmony and incorporating green threads
- it would be nice to see more progress on the pthread JIRAs for
low-hanging things like method names).

So I am off to get all my clothes asbestos lined. This is deliberate,
not just flippant, I want everyone who has a complaint about this idea
to complain. I feel with the current pthread code going into the trunk
I complained and wasn't listened to. I hope I will listen to other
people's pov and we can get the abstraction that fits everyone's
requirements (or learn why this is a mistake).

Regards,
Ian

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[David P Grove]

1) This really shouldn't involve a flame war or heated argument. In theory we are all adults and this is a purely technical discussion. Let's keep it that way. If on reading someone's email there appears to be a way to read it and get mad, try reading it again under the assumption that English is an ambiguous language, email a blunt instrument, and we're all professionals who don't want to indulge in flame wars because they reflect poorly on us as individuals and the project as a whole.

2) Before proposing solutions or writing any code, I'd like us to collectively take a few steps back and understand what the actual use cases for green threading are so we can figure out the best way to support them. In particular, I want to understand if it makes sense to support these "below" the "native thread" APIs instead of beside them. This is the approach we took in a project I was involved in [1] where we took J9 (which uses a port library abstraction that bears a very strong similarity to the one used by Harmony...) and ported it with minimal effort to a libraryOS/hypervisor without a pthread package by rolling our own threading library below J9's portlibrary abstraction. We felt that this resulted in a much better overall result since there was still only 1 scheduler, it was at the place where it could have all the information it needed to make the best possible decisions, and there were no changes required in the actual JVM code (including the JVM's locking code, implementation of sleep, etc).

--dave

[1] http://domino.research.ibm.com/comm/research_people.nsf/pages/dgrove.vee2007.html


------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

2009/2/19 David P Grove <groved@...>:
> 1) This really shouldn't involve a flame war or heated argument. In theory
> we are all adults and this is a purely technical discussion. Let's keep it
> that way. If on reading someone's email there appears to be a way to read it
> and get mad, try reading it again under the assumption that English is an
> ambiguous language, email a blunt instrument, and we're all professionals
> who don't want to indulge in flame wars because they reflect poorly on us as
> individuals and the project as a whole.

As a theoretical adult, I agree completely :-)

> 2) Before proposing solutions or writing any code, I'd like us to
> collectively take a few steps back and understand what the actual use cases
> for green threading are so we can figure out the best way to support them.
> In particular, I want to understand if it makes sense to support these
> "below" the "native thread" APIs instead of beside them. This is the
> approach we took in a project I was involved in [1] where we took J9 (which
> uses a port library abstraction that bears a very strong similarity to the
> one used by Harmony...) and ported it with minimal effort to a
> libraryOS/hypervisor without a pthread package by rolling our own threading
> library below J9's portlibrary abstraction. We felt that this resulted in a
> much better overall result since there was still only 1 scheduler, it was at
> the place where it could have all the information it needed to make the best
> possible decisions, and there were no changes required in the actual JVM
> code (including the JVM's locking code, implementation of sleep, etc).

I agree with taking steps back, and hope this e-mail exchange can be
used for this.

So, my understanding of your proposal would be that in
org.jikesrvm.scheduler we provide a thin veneer that punts everything
to syscalls. In the syscalls we implement greenthreads as C code,
pthreads with cond locks as C code or map to harmony style calls. Why
is writing everything as C code advantageous to Java code? It seems we
give up part of our metacircularity in this approach and also have to
reinvent all of the thread models.

Ian

> --dave
>
> [1]
> http://domino.research.ibm.com/comm/research_people.nsf/pages/dgrove.vee2007.html

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Filip Pizlo-2]

Here's my position:

- Let's not bring back the old green threads for a 3.0.2 release.

- I'm fine with any green threads implementation being in there, and  
would be happy to see the RVM support further configurability.  
However, this should be done in a way that: a) preserves the  
simplicity, stability, and performance of the system, and b) results  
in a system that we can all agree upon.

To that end, I tend to think that a green-threads-underneath approach  
is best.  The current code is thread-model-agnostic; it just happens  
to make pthreads calls because that 's what we happen to have.  I  
don't foresee any big issues in porting that code to use Harmony or  
even Win32 thread APIs.  It could call into any thread  
implementation's API, so long as it has some basic notions of locks,  
notification (either semaphores or condition variables), and of course  
a CreateThread call.  The interesting questions are:

1) Can the implementation of threading, which sits beneath the current  
native thread code, be written in Java?

2) Can such an implementation be done in an orthogonal fashion?  I.e.  
can it be done without disturbing native threads?

For (1) we may have to reconsider the notion that all running code  
always has an RVMThread as its context.  Introducing some other notion  
of context - why not just call it RVMContext, or just Context - might  
be appropriate.  That way, the Context could either be a thread or a  
processor.  You might even go further, and add a notion of an  
InterruptContext, which is a special kind of Context not associated  
with a RVMThread, used for interrupt handlers.  I've seen cases where  
such a thing would be useful (Ovm, some OS's written in managed  
languages, etc.).

For (2) my main concern is that native threads is what most of us  
use.  We shouldn't sacrifice the simplicity of the system to support  
exotic functionality.  I tend to think that the native threads code  
is, currently, more stable and robust than green threads ever was.  It  
also supports more functionality (granted, some of that functionality  
is "deprecated", like suspend(), but having it is really great for  
implementing, for example, biased locking ... more on that  
later :-)).  So, whatever we do to support multiple threading models,  
we should do it in such a way that native threads remains the primary  
production configuration, and its stability, performance, and  
robustness are not sacrificed.

I'm sure this is all eminently doable, but I would just like to argue  
strongly for keeping the default configuration clean and sane.

-F



On Feb 19, 2009, at 8:13 AM, Ian Rogers wrote:

> Hi,
>
> so the preliminaries:
>
> 1) native threads should be the Jikes RVM default!
> 2) great work Fil!
>
> There is a desire to have green threading again (at least from me) and
> to get Harmony running again, so let's play out some examples of how
> to say support the sleep operation with different thread models
> (ignoring cleaning up the arguments and throwing any exceptions that
> may have been flagged while we were asleep):
>
> greentheads - take the running thread, put some time information into
> it (or a proxy - I forget) and then put it on the waiting queue,
> schedule other work
>
> pthreads - take the thread, get a pthread cond lock, do a syscall to
> do a timed wait on the pthread cond lock
>
> harmony threads - take the thread, call through to a harmony interface
> that abstracts the sleep
>
> so what abstractions do we need: things that are processor local (ie.
> processors we used to have these, they should probably be an interface
> to make native threads efficient), a singleton thing to ask global
> questions to like how many locks are there (ie. the scheduler we used
> to have), different thread model implementations. I see the packages
> of:
>
> org.jikesrvm.scheduler
>
> this is the interface to the rest of the world, nothing externally
> should be accessing packages beneath here
>
> org.jikesrvm.scheduler.greenthreads - the implementation for
> greenthreads (queues, etc.) that we had for GNU Classpath, to fully
> support this with Harmony requires creating a DLL wrapper. Support for
> Classpath requires the syswrap hack.
>
> org.jikesrvm.scheduler.nativethreads.pthreads - Fil's work on
> providing a full pthread model, cond locks won't be exposed above
> here.
>
> org.jikesrvm.scheduler.nativethreads.harmony - As with the original
> native thread goal, just punt to a native thread library, ie Harmony
> threads
>
> There is likely to be a chance to share things, like queues, between
> the pthread and green thread code, we can therefore have a utility
> directory. We should probably not have a lot of different lock or
> other implementations in the top-level scheduler package, they can be
> pushed down into other packages.
>
> I believe this proposal is controversial as it's bringing back the
> interface as it was around 3.0.1. I don't propose we copy and paste
> that code, but the structure will look like it from the outside. Feel
> free to flame me about why the old interface was the stupidest thing
> ever as I would like to shake out what are myths and realities (it's
> not a secret that the old thread model wasn't 100% complete).
>
> Can we implement the Harmony code a different way? Yes, I will try
> this in the short-term. We can implement the Harmony threads using the
> Apache Portable Runtime. This is diving under the feet of the harmony
> thread implementation. I believe it is best in the long-term we use
> the Harmony interface for Harmony, as it means we are abdicating
> responsibility for threading to Harmony and Harmony can give us
> compatibility on all the platforms Harmony supports (we are a Java VM,
> we should run on all platforms as Java is portable). I could also try
> emulating pthread cond locks with Harmony's Java like monitors, but
> this would be vile.
>
> What will break? AIX ;-) Nothing should break, as this change is just
> *adding functionality*. The build can be configured to use the models
> that are complete for the particular architecture. Long-term I'd
> expect Harmony native threads to be the default. Green threads has a
> user base. Native pthreads for Classpath will be there for the curious
> and platforms like AIX where Harmony may not build straight away.
>
> What about OpenJDK? Depending on the abstraction it will probably
> closely resemble either Harmony or pthreads, my belief is probably
> more like pthreads. Having the flexibility in our model should mean
> little work in matching OpenJDKs thread model to ours.
>
> What does this mean for 3.1? I think the thread model is in a state of
> flux and it's not in the place it should be. I'd be tempted to make
> the next release 3.0.2 and bring back the green thread model
> temporarily (maybe keep native threads on the trunk and revert them on
> the 3.0.2 branch). I think the current code exposes too many pthread
> internals and this will be a portability issue for users interfacing
> with the thread library. Ideally the changes above can be written
> before the next release and then it could be 3.1, this may be pushing
> things (there are still many outstanding pthread JIRAs without the
> addition of native threads for Harmony and incorporating green threads
> - it would be nice to see more progress on the pthread JIRAs for
> low-hanging things like method names).
>
> So I am off to get all my clothes asbestos lined. This is deliberate,
> not just flippant, I want everyone who has a complaint about this idea
> to complain. I feel with the current pthread code going into the trunk
> I complained and wasn't listened to. I hope I will listen to other
> people's pov and we can get the abstraction that fits everyone's
> requirements (or learn why this is a mistake).
>
> Regards,
> Ian
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San  
> Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the  
> Enterprise
> -Strategies to boost innovation and cut costs with open source  
> participation
> -Receive a $600 discount off the registration fee with the source  
> code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

2009/2/19 Filip Pizlo <pizlo@...>:
> Here's my position:
>
> - Let's not bring back the old green threads for a 3.0.2 release.

I think not moving would make sense when it is not clear the current
API is where we should be. Even the JIRAs off of RVM-91 don't argue
the API is where we want it to be.

> - I'm fine with any green threads implementation being in there, and
> would be happy to see the RVM support further configurability.
> However, this should be done in a way that: a) preserves the
> simplicity, stability, and performance of the system, and b) results
> in a system that we can all agree upon.
>
> To that end, I tend to think that a green-threads-underneath approach
> is best.  The current code is thread-model-agnostic; it just happens
> to make pthreads calls because that 's what we happen to have.  I
> don't foresee any big issues in porting that code to use Harmony or
> even Win32 thread APIs.  It could call into any thread
> implementation's API, so long as it has some basic notions of locks,
> notification (either semaphores or condition variables), and of course
> a CreateThread call.

So the problem is that we end up replicating pthread locks using Java
like monitors to implement Java like monitors. The current pthread
model in trunk is agnostic in the sense it doesn't assume Linux or
Windows, however, for Harmony the best option is to simply pick up
their work on making an architecture agnostic thread library, not to
implement any low-level thread or locking code of our own. We're
guaranteed a good interaction with the Class library this way and
don't need to implement our own thread compatibility library (which we
could have to with the stacked model where we're building cond locks
out of monitors).

> The interesting questions are:
>
> 1) Can the implementation of threading, which sits beneath the current
> native thread code, be written in Java?
>
> 2) Can such an implementation be done in an orthogonal fashion?  I.e.
> can it be done without disturbing native threads?
>
> For (1) we may have to reconsider the notion that all running code
> always has an RVMThread as its context.  Introducing some other notion
> of context - why not just call it RVMContext, or just Context - might
> be appropriate.  That way, the Context could either be a thread or a
> processor.  You might even go further, and add a notion of an
> InterruptContext, which is a special kind of Context not associated
> with a RVMThread, used for interrupt handlers.  I've seen cases where
> such a thing would be useful (Ovm, some OS's written in managed
> languages, etc.).

Context sounds like another name for Processor, I think it is easy to
confuse with a thread so I'd stick with Processor. There may be
another use-case for a Context but we've not needed it yet.

Whilst there are lots of things that could be done the expedient way
to get green threads working is to use the green thread code that was
already working (and running the JSR166 TCK as part of the sanity test
runs). Using the old code also means that we are in some way
backwardly compatible, it's just the code has moved around.

> For (2) my main concern is that native threads is what most of us
> use.  We shouldn't sacrifice the simplicity of the system to support
> exotic functionality.  I tend to think that the native threads code
> is, currently, more stable and robust than green threads ever was.  It
> also supports more functionality (granted, some of that functionality
> is "deprecated", like suspend(), but having it is really great for
> implementing, for example, biased locking ... more on that
> later :-)).  So, whatever we do to support multiple threading models,
> we should do it in such a way that native threads remains the primary
> production configuration, and its stability, performance, and
> robustness are not sacrificed.

So whilst pthreads are good they don't:
1) run SPEC JVM 2008 - they are assume/rely on Classpath :-( losing
Harmony support is a massive regression from our 3.0.0 and 3.0.1
releases and is a release blocker imo
2) support a lot of existing research implemented on top of the green
thread model
For specific problems there are the open issues on RVM-91.

> I'm sure this is all eminently doable, but I would just like to argue
> strongly for keeping the default configuration clean and sane.
>
> -F

So I think this is the problem. The code as is argues that a sane
configuration consists of a bunch of primitives on top of a cond lock.
That's fine, but it is 1 of 3 ways of writing this code. We can do the
old green thread approach or punt in the Harmony approach. Most of the
current API is identical to the green thread API, however, some of
what were separate concerns previously all got thrown on top of
RVMThread and I think this should be moved back out again. I think
this is a minor issue. As the APIs are largely recycled in the new API
from the green thread API we had before, I'm not sure why such an
approach would be any less clean or sane. My concerns are to get back
green threads, for those of us working on top of it, and fix Harmony.

Ian

> On Feb 19, 2009, at 8:13 AM, Ian Rogers wrote:
>
>> Hi,
>>
>> so the preliminaries:
>>
>> 1) native threads should be the Jikes RVM default!
>> 2) great work Fil!
>>
>> There is a desire to have green threading again (at least from me) and
>> to get Harmony running again, so let's play out some examples of how
>> to say support the sleep operation with different thread models
>> (ignoring cleaning up the arguments and throwing any exceptions that
>> may have been flagged while we were asleep):
>>
>> greentheads - take the running thread, put some time information into
>> it (or a proxy - I forget) and then put it on the waiting queue,
>> schedule other work
>>
>> pthreads - take the thread, get a pthread cond lock, do a syscall to
>> do a timed wait on the pthread cond lock
>>
>> harmony threads - take the thread, call through to a harmony interface
>> that abstracts the sleep
>>
>> so what abstractions do we need: things that are processor local (ie.
>> processors we used to have these, they should probably be an interface
>> to make native threads efficient), a singleton thing to ask global
>> questions to like how many locks are there (ie. the scheduler we used
>> to have), different thread model implementations. I see the packages
>> of:
>>
>> org.jikesrvm.scheduler
>>
>> this is the interface to the rest of the world, nothing externally
>> should be accessing packages beneath here
>>
>> org.jikesrvm.scheduler.greenthreads - the implementation for
>> greenthreads (queues, etc.) that we had for GNU Classpath, to fully
>> support this with Harmony requires creating a DLL wrapper. Support for
>> Classpath requires the syswrap hack.
>>
>> org.jikesrvm.scheduler.nativethreads.pthreads - Fil's work on
>> providing a full pthread model, cond locks won't be exposed above
>> here.
>>
>> org.jikesrvm.scheduler.nativethreads.harmony - As with the original
>> native thread goal, just punt to a native thread library, ie Harmony
>> threads
>>
>> There is likely to be a chance to share things, like queues, between
>> the pthread and green thread code, we can therefore have a utility
>> directory. We should probably not have a lot of different lock or
>> other implementations in the top-level scheduler package, they can be
>> pushed down into other packages.
>>
>> I believe this proposal is controversial as it's bringing back the
>> interface as it was around 3.0.1. I don't propose we copy and paste
>> that code, but the structure will look like it from the outside. Feel
>> free to flame me about why the old interface was the stupidest thing
>> ever as I would like to shake out what are myths and realities (it's
>> not a secret that the old thread model wasn't 100% complete).
>>
>> Can we implement the Harmony code a different way? Yes, I will try
>> this in the short-term. We can implement the Harmony threads using the
>> Apache Portable Runtime. This is diving under the feet of the harmony
>> thread implementation. I believe it is best in the long-term we use
>> the Harmony interface for Harmony, as it means we are abdicating
>> responsibility for threading to Harmony and Harmony can give us
>> compatibility on all the platforms Harmony supports (we are a Java VM,
>> we should run on all platforms as Java is portable). I could also try
>> emulating pthread cond locks with Harmony's Java like monitors, but
>> this would be vile.
>>
>> What will break? AIX ;-) Nothing should break, as this change is just
>> *adding functionality*. The build can be configured to use the models
>> that are complete for the particular architecture. Long-term I'd
>> expect Harmony native threads to be the default. Green threads has a
>> user base. Native pthreads for Classpath will be there for the curious
>> and platforms like AIX where Harmony may not build straight away.
>>
>> What about OpenJDK? Depending on the abstraction it will probably
>> closely resemble either Harmony or pthreads, my belief is probably
>> more like pthreads. Having the flexibility in our model should mean
>> little work in matching OpenJDKs thread model to ours.
>>
>> What does this mean for 3.1? I think the thread model is in a state of
>> flux and it's not in the place it should be. I'd be tempted to make
>> the next release 3.0.2 and bring back the green thread model
>> temporarily (maybe keep native threads on the trunk and revert them on
>> the 3.0.2 branch). I think the current code exposes too many pthread
>> internals and this will be a portability issue for users interfacing
>> with the thread library. Ideally the changes above can be written
>> before the next release and then it could be 3.1, this may be pushing
>> things (there are still many outstanding pthread JIRAs without the
>> addition of native threads for Harmony and incorporating green threads
>> - it would be nice to see more progress on the pthread JIRAs for
>> low-hanging things like method names).
>>
>> So I am off to get all my clothes asbestos lined. This is deliberate,
>> not just flippant, I want everyone who has a complaint about this idea
>> to complain. I feel with the current pthread code going into the trunk
>> I complained and wasn't listened to. I hope I will listen to other
>> people's pov and we can get the abstraction that fits everyone's
>> requirements (or learn why this is a mistake).
>>
>> Regards,
>> Ian
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source
>> participation
>> -Receive a $600 discount off the registration fee with the source
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[David P Grove]

What I was proposing is (I think) more or less what Fil described as part of his longer note.

Suppose we take the sysCalls made at the "bottom" of the native threading layer as the specification of what the rest of Jikes RVM expects from its underlying thread abstraction. Pull those into a class/interface so that we can have multiple implementations, selecting one at build time just like we do for GCs in MMtk. Perhaps make minor cleanups, renamings, and generalizations.
(1) There's a default trivial implementation where those methods are implemented as sysCalls to C code that wraps pthread calls. That's native threading against GNU Classpath/pthreads.
(2) There's a default trivial implementation where those methods are implemented as sysCalls to C code that wraps Harmony threading APIs. That's native threading against Harmony/pthreads. Perhaps this is very close to (1) and is just some ifdefs in the C code. I haven't looked at the Harmony library enough to know.
(3) Same as (1) and (2), except for Open JDK.
(4) There's an implementation, most likely written in Java, that is not at all trivial where something functionally like the m-n threading we had before is provided. Perhaps some old green threading code can be re-used. A key question here is do you want n-1 or n-m. n-1 is significantly simpler to implement. This is really a question about use cases and target platforms. In theory we might want to have both n-1 and n-m.
(5) (In theory...the Libra project is over so I'm not motivated to go and write it) there's an implementation, written in C against the Libra library OS functions, which lets Jikes RVM run on a hypervisor partition without an OS underneath it at all.
(6) ...more options here...JNode?

The main point is that we avoid re-introducing scheduler APIs in the default case where we are running with native threading. Instead, green threading sits below the rest of the JVM and Jikes RVM as a whole can be oblivious to how those APIs are implemented. I believe this should yield a simpler and easier to maintain green threading layer than what we had before (which had 10+ years of evolution and multiple attempts to accommodate hostile native code embedded in it). I'm sure it will yield a simpler and easier to maintain native threading layer.

--dave

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Filip Pizlo-2]

> 2009/2/19 Filip Pizlo <pizlo@...>:
>> Here's my position:
>>
>> - Let's not bring back the old green threads for a 3.0.2 release.
>
> I think not moving would make sense when it is not clear the current
> API is where we should be. Even the JIRAs off of RVM-91 don't argue
> the API is where we want it to be.

APIs are trivial to refactor later.  None of us have argued that the  
APIs are "wrong" other than alpha renaming.  That's what most of the  
RVM-91 sub-tasks are about.

On the other hand, dropping and then reintegrating native threads is a  
lot of work.  A lot of us are already relying on native threads; I've  
got a research branch that relies on them and so does Daniel for some  
MMTk work.

>
>
>> - I'm fine with any green threads implementation being in there, and
>> would be happy to see the RVM support further configurability.
>> However, this should be done in a way that: a) preserves the
>> simplicity, stability, and performance of the system, and b) results
>> in a system that we can all agree upon.
>>
>> To that end, I tend to think that a green-threads-underneath approach
>> is best.  The current code is thread-model-agnostic; it just happens
>> to make pthreads calls because that 's what we happen to have.  I
>> don't foresee any big issues in porting that code to use Harmony or
>> even Win32 thread APIs.  It could call into any thread
>> implementation's API, so long as it has some basic notions of locks,
>> notification (either semaphores or condition variables), and of  
>> course
>> a CreateThread call.
>
> So the problem is that we end up replicating pthread locks using Java
> like monitors to implement Java like monitors. The current pthread
> model in trunk is agnostic in the sense it doesn't assume Linux or
> Windows, however, for Harmony the best option is to simply pick up
> their work on making an architecture agnostic thread library, not to
> implement any low-level thread or locking code of our own. We're
> guaranteed a good interaction with the Class library this way and
> don't need to implement our own thread compatibility library (which we
> could have to with the stacked model where we're building cond locks
> out of monitors).

I don't understand.  A lock is a lock.  The current thread code just  
needs a non-recursive lock and a condition variable.  Condition  
variables are found in almost every threading API, and if they're  
absent, they're trivial to implement yourself.

And what are you meaning by low-level thread or locking code?  We rely  
on the following functions and expect only very loose semantics of them:

thread_create
mutex_lock (unrecursive)
mutex_unlock (also unrecursive)
cond_wait
cond_wait_timeout
cond_broadcast
key_create
getspecific
setspecific

That's it.  The thread_create, key_create, getspecific, and  
setspecific calls are used in more-or-less the same way they were used  
by green threads.  So the native thread code cannot possibly be  
considered a regression in this department.  As for mutex_ and cond_  
operations, these are standard, and we don't require pthread semantics  
(like, we don't need a recursive lock option, we don't need single-
thread-wakeup; these choices were deliberate to ensure maximum  
portability).

So I don't understand the problem.  Why is it so hard to just wrap  
these calls and have them call the Harmony equivalents?

>
>
>> The interesting questions are:
>>
>> 1) Can the implementation of threading, which sits beneath the  
>> current
>> native thread code, be written in Java?
>>
>> 2) Can such an implementation be done in an orthogonal fashion?  I.e.
>> can it be done without disturbing native threads?
>>
>> For (1) we may have to reconsider the notion that all running code
>> always has an RVMThread as its context.  Introducing some other  
>> notion
>> of context - why not just call it RVMContext, or just Context - might
>> be appropriate.  That way, the Context could either be a thread or a
>> processor.  You might even go further, and add a notion of an
>> InterruptContext, which is a special kind of Context not associated
>> with a RVMThread, used for interrupt handlers.  I've seen cases where
>> such a thing would be useful (Ovm, some OS's written in managed
>> languages, etc.).
>
> Context sounds like another name for Processor, I think it is easy to
> confuse with a thread so I'd stick with Processor. There may be
> another use-case for a Context but we've not needed it yet.

It's not a processor.  It could be a thread or a processor.

>
>
> Whilst there are lots of things that could be done the expedient way
> to get green threads working is to use the green thread code that was
> already working (and running the JSR166 TCK as part of the sanity test
> runs). Using the old code also means that we are in some way
> backwardly compatible, it's just the code has moved around.

I disagree.  Some of the biggest changes have been in MMTk, which now  
expects the semantics of native threads.  We could implement a new,  
different green thread library that obeys those semantics, but moving  
to the old code would only introduce unnecessary complexity and bugs.

>
>
>> For (2) my main concern is that native threads is what most of us
>> use.  We shouldn't sacrifice the simplicity of the system to support
>> exotic functionality.  I tend to think that the native threads code
>> is, currently, more stable and robust than green threads ever was.  
>> It
>> also supports more functionality (granted, some of that functionality
>> is "deprecated", like suspend(), but having it is really great for
>> implementing, for example, biased locking ... more on that
>> later :-)).  So, whatever we do to support multiple threading models,
>> we should do it in such a way that native threads remains the primary
>> production configuration, and its stability, performance, and
>> robustness are not sacrificed.
>
> So whilst pthreads are good they don't:
> 1) run SPEC JVM 2008 - they are assume/rely on Classpath :-( losing
> Harmony support is a massive regression from our 3.0.0 and 3.0.1
> releases and is a release blocker imo

Unrelated issues.  We don't rely on "pthreads".  We rely on "native  
threads".  Tweaking the code to use Harmony threads, or any other  
threads, via either #ifdefs or some other mechanism, isn't going to be  
difficult.  We already talked about this, and you agreed.

>
> 2) support a lot of existing research implemented on top of the green
> thread model
> For specific problems there are the open issues on RVM-91.

The only relevant one I see is Harmony and setPriority().  As for the  
latter, a lot of non-RT VMs do not implement setPriority().  I think  
that's the case with HotSpot on Linux from all that I've heard.

>
>
>> I'm sure this is all eminently doable, but I would just like to argue
>> strongly for keeping the default configuration clean and sane.
>>
>> -F
>
> So I think this is the problem. The code as is argues that a sane
> configuration consists of a bunch of primitives on top of a cond lock.
> That's fine, but it is 1 of 3 ways of writing this code. We can do the
> old green thread approach or punt in the Harmony approach. Most of the
> current API is identical to the green thread API, however, some of
> what were separate concerns previously all got thrown on top of
> RVMThread and I think this should be moved back out again. I think
> this is a minor issue. As the APIs are largely recycled in the new API
> from the green thread API we had before, I'm not sure why such an
> approach would be any less clean or sane. My concerns are to get back
> green threads, for those of us working on top of it, and fix Harmony.

The API is different from the green threads API.  Just look at the  
MMTk glue.  The native thread version is cleaner and less buggy.

I don't understand why you keep bringing up Harmony.  Does it not give  
you some manner of condition-variable-like notification mechanism?  
What's the problem?

Condition variables can be implemented on top of any other  
notification mechanism.  This is a known fact.  Just give each thread  
a semaphore and you're good to go.  Or on Windows, you can pretty much  
just use AutoResetEvents from what I remember.

-F

>
>
> Ian
>
>> On Feb 19, 2009, at 8:13 AM, Ian Rogers wrote:
>>
>>> Hi,
>>>
>>> so the preliminaries:
>>>
>>> 1) native threads should be the Jikes RVM default!
>>> 2) great work Fil!
>>>
>>> There is a desire to have green threading again (at least from me)  
>>> and
>>> to get Harmony running again, so let's play out some examples of how
>>> to say support the sleep operation with different thread models
>>> (ignoring cleaning up the arguments and throwing any exceptions that
>>> may have been flagged while we were asleep):
>>>
>>> greentheads - take the running thread, put some time information  
>>> into
>>> it (or a proxy - I forget) and then put it on the waiting queue,
>>> schedule other work
>>>
>>> pthreads - take the thread, get a pthread cond lock, do a syscall to
>>> do a timed wait on the pthread cond lock
>>>
>>> harmony threads - take the thread, call through to a harmony  
>>> interface
>>> that abstracts the sleep
>>>
>>> so what abstractions do we need: things that are processor local  
>>> (ie.
>>> processors we used to have these, they should probably be an  
>>> interface
>>> to make native threads efficient), a singleton thing to ask global
>>> questions to like how many locks are there (ie. the scheduler we  
>>> used
>>> to have), different thread model implementations. I see the packages
>>> of:
>>>
>>> org.jikesrvm.scheduler
>>>
>>> this is the interface to the rest of the world, nothing externally
>>> should be accessing packages beneath here
>>>
>>> org.jikesrvm.scheduler.greenthreads - the implementation for
>>> greenthreads (queues, etc.) that we had for GNU Classpath, to fully
>>> support this with Harmony requires creating a DLL wrapper. Support  
>>> for
>>> Classpath requires the syswrap hack.
>>>
>>> org.jikesrvm.scheduler.nativethreads.pthreads - Fil's work on
>>> providing a full pthread model, cond locks won't be exposed above
>>> here.
>>>
>>> org.jikesrvm.scheduler.nativethreads.harmony - As with the original
>>> native thread goal, just punt to a native thread library, ie Harmony
>>> threads
>>>
>>> There is likely to be a chance to share things, like queues, between
>>> the pthread and green thread code, we can therefore have a utility
>>> directory. We should probably not have a lot of different lock or
>>> other implementations in the top-level scheduler package, they can  
>>> be
>>> pushed down into other packages.
>>>
>>> I believe this proposal is controversial as it's bringing back the
>>> interface as it was around 3.0.1. I don't propose we copy and paste
>>> that code, but the structure will look like it from the outside.  
>>> Feel
>>> free to flame me about why the old interface was the stupidest thing
>>> ever as I would like to shake out what are myths and realities (it's
>>> not a secret that the old thread model wasn't 100% complete).
>>>
>>> Can we implement the Harmony code a different way? Yes, I will try
>>> this in the short-term. We can implement the Harmony threads using  
>>> the
>>> Apache Portable Runtime. This is diving under the feet of the  
>>> harmony
>>> thread implementation. I believe it is best in the long-term we use
>>> the Harmony interface for Harmony, as it means we are abdicating
>>> responsibility for threading to Harmony and Harmony can give us
>>> compatibility on all the platforms Harmony supports (we are a Java  
>>> VM,
>>> we should run on all platforms as Java is portable). I could also  
>>> try
>>> emulating pthread cond locks with Harmony's Java like monitors, but
>>> this would be vile.
>>>
>>> What will break? AIX ;-) Nothing should break, as this change is  
>>> just
>>> *adding functionality*. The build can be configured to use the  
>>> models
>>> that are complete for the particular architecture. Long-term I'd
>>> expect Harmony native threads to be the default. Green threads has a
>>> user base. Native pthreads for Classpath will be there for the  
>>> curious
>>> and platforms like AIX where Harmony may not build straight away.
>>>
>>> What about OpenJDK? Depending on the abstraction it will probably
>>> closely resemble either Harmony or pthreads, my belief is probably
>>> more like pthreads. Having the flexibility in our model should mean
>>> little work in matching OpenJDKs thread model to ours.
>>>
>>> What does this mean for 3.1? I think the thread model is in a  
>>> state of
>>> flux and it's not in the place it should be. I'd be tempted to make
>>> the next release 3.0.2 and bring back the green thread model
>>> temporarily (maybe keep native threads on the trunk and revert  
>>> them on
>>> the 3.0.2 branch). I think the current code exposes too many pthread
>>> internals and this will be a portability issue for users interfacing
>>> with the thread library. Ideally the changes above can be written
>>> before the next release and then it could be 3.1, this may be  
>>> pushing
>>> things (there are still many outstanding pthread JIRAs without the
>>> addition of native threads for Harmony and incorporating green  
>>> threads
>>> - it would be nice to see more progress on the pthread JIRAs for
>>> low-hanging things like method names).
>>>
>>> So I am off to get all my clothes asbestos lined. This is  
>>> deliberate,
>>> not just flippant, I want everyone who has a complaint about this  
>>> idea
>>> to complain. I feel with the current pthread code going into the  
>>> trunk
>>> I complained and wasn't listened to. I hope I will listen to other
>>> people's pov and we can get the abstraction that fits everyone's
>>> requirements (or learn why this is a mistake).
>>>
>>> Regards,
>>> Ian
>>>
>>> ------------------------------------------------------------------------------
>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>> Francisco, CA
>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>> Enterprise
>>> -Strategies to boost innovation and cut costs with open source
>>> participation
>>> -Receive a $600 discount off the registration fee with the source
>>> code: SFAD
>>> http://p.sf.net/sfu/XcvMzF8H
>>> _______________________________________________
>>> Jikesrvm-core mailing list
>>> Jikesrvm-core@...
>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San  
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the  
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source  
>> participation
>> -Receive a $600 discount off the registration fee with the source  
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San  
> Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the  
> Enterprise
> -Strategies to boost innovation and cut costs with open source  
> participation
> -Receive a $600 discount off the registration fee with the source  
> code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Daniel Frampton]

I agree that getting Harmony working with native threads is an
important goal, but I can not see how we can list it as a blocker.

The previous implementation of Harmony was not ready for production
use. From the tests we were running it never supported multiple
processors, and when running with a single processor had a significant
number of failures. Obviously moving to a system without green threads
makes running with a single processor no longer possible. This was a
well understood problem, and work on Harmony to fix this issue was
deliberately delayed until after native threads was implemented.

I think that the decision to make a release should be based on
stability and improvements in our supported configurations---those
that are running (and passing) sanity runs. Obviously when we have a
stable implementation using the Harmony libraries that is a
significant release milestone. When this happens, it may even be
sensible for Harmony to become the default class library
implementation. If we can have Harmony working for our next release
then all the better, but it does not make sense to significantly delay
releases solely for this.

This relates to a significant issue that I don't think was adequately
addressed at the time. The announcement of our initial
compatibility/support for the Harmony libraries was premature, and I
know made a lot of the core team uncomfortable. Such announcements
have the potential to significantly affect the credibility of Jikes
RVM, and should not be made without some consultation within the core
team. What we claim to support officially in releases of JikesRVM has
a direct impact on everyone working on the project.

I think it is clear that we all want to work toward compatibility with
Harmony (and possibly OpenJDK) but we need to do so carefully.

I have left aside my thoughts on the low level technical detail here;
it is important to take a step back from that and solve the strategic
issues first.

Cheers,
Daniel.

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

So firstly here is the Harmony thread API:

http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html

I have problems with pretty much everything that's been said here:

Dave> There's a default trivial implementation where those methods are
implemented as sysCalls to C code that wraps pthread calls. That's
native threading against GNU Classpath/pthreads."

in the planning for 3.1 I stated that I don't see Classpath as our
long term class library, assuming it in the thread model, to me,
appears wrong.

Dave> There's a default trivial implementation where those methods are
implemented as sysCalls to C code that wraps Harmony threading APIs.
That's native threading against Harmony/pthreads. Perhaps this is very
close to (1) and is just some ifdefs in the C code. I haven't looked
at the Harmony library enough to know.

this doesn't fit Harmony. In the current threading code we assume cond locks.

Dave> There's an implementation, most likely written in Java, that is
not at all trivial where something functionally like the m-n threading
we had before is provided. Perhaps some old green threading code can
be re-used. A key question here is do you want n-1 or n-m. n-1 is
significantly simpler to implement. This is really a question about
use cases and target platforms. In theory we might want to have both
n-1 and n-m.

So this appears to be creating a task for people working off green
threads to create a new green thread implementation. The expedient
thing is to work off where we were (sufficient to run all
applications, X10 and the majority of the JSR166 TCK).

Dave> The main point is that we avoid re-introducing scheduler APIs in
the default case where we are running with native threading. Instead,
green threading sits below the rest of the JVM and Jikes RVM as a
whole can be oblivious to how those APIs are implemented. I believe
this should yield a simpler and easier to maintain green threading
layer than what we had before (which had 10+ years of evolution and
multiple attempts to accommodate hostile native code embedded in it).
I'm sure it will yield a simpler and easier to maintain native
threading layer.

But we inherently get a non-metacircular implementation and have to
reinvent the green thread world.

Fil, look at the Harmony thread API and you'll see what my point is. I
agree that there are changes elsewhere in the scheduler API and with
reintroducing green threads and a harmony model they should be adhered
to. What I disagree with is exposing the cond locks in the scheduler.

Daniel> The previous implementation of Harmony was not ready for
production use. From the tests we were running it never supported
multiple processors, and when running with a single processor had a
significant number of failures. Obviously moving to a system without
green threads makes running with a single processor no longer
possible. This was a well understood problem, and work on Harmony to
fix this issue was deliberately delayed until after native threads was
implemented.

I agree, we have been running against and leading Classpath
development for a while. Harmony support was at a good level. We ran
SPEC JVM 2008, which I see as essential for a research virtual
machine. There is much research that can be done on a single core.

Daniel> This relates to a significant issue that I don't think was
adequately addressed at the time. The announcement of our initial
compatibility/support for the Harmony libraries was premature, and I
know made a lot of the core team uncomfortable. Such announcements
have the potential to significantly affect the credibility of Jikes
RVM, and should not be made without some consultation within the core
team. What we claim to support officially in releases of JikesRVM has
a direct impact on everyone working on the project.

So this is highly irritating. If you have concerns:
1) e-mail me, and/or
2) open JIRAs, there's a whole place for these in the Harmony support, and/or
3) mail the core mailing list
Saying that the Harmony support was announced prematurely should
really have been stated at the time, when it could have been
rectified. Saying it now is just ridiculous.

In terms of credibility, it is far far more important for our
credibility that we run on a class library that will give enough
support to at a least run major benchmark suites. In this case SPEC
JVM 2008.

I would dearly love help with the Harmony implementation, so far
Alexey has been the only contributor.

Daniel> I think it is clear that we all want to work toward
compatibility with Harmony (and possibly OpenJDK) but we need to do so
carefully.

As we were already compatible with Harmony, I don't know why we're
taking a view that we're not.

Daniel> I have left aside my thoughts on the low level technical
detail here; it is important to take a step back from that and solve
the strategic issues first.

I agree. We either have a minimalist interface or something where we
have multiple configurations. I need multiple configurations and if I
can't do that against the trunk I will have to do my own thing.

In terms of premature decisions, I think merging Fil's code with the
trunk was premature. We should have worked these issues out first. We
should have closed the JIRAs on RVM-91 first. We should have asked the
core list before the merge, first.

Ian

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Filip Pizlo-2]

Ian,

I've looked at the Harmony threading APIs.  The hythread API has  
"monitors".  We can just use these as cond locks.

Or we could just use pthread mutexes and cond's just as we're doing  
now.  This doesn't really hurt portability.  Even Win32 has cond locks  
now...  I don't see anything wrong with having some #ifdefs for code  
like this.  It would be ~40 lines of isolated nastiness in sys.C (or  
its equivalent).

Or are you suggesting that we use hythread's monitors as our Java  
monitors?  And have pointers from the Java head into the C heap?  And  
require fat lock acquisition to do a C downcall?  That could get quite  
messy, given how our thin locks rely on very specific invariants from  
our fat locks, which essentially forces us to implement fat locks  
ourselves.  I guess my whole problem here is that I don't see what  
design you're arguing for.

-F




On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:

> So firstly here is the Harmony thread API:
>
> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>
> I have problems with pretty much everything that's been said here:
>
> Dave> There's a default trivial implementation where those methods are
> implemented as sysCalls to C code that wraps pthread calls. That's
> native threading against GNU Classpath/pthreads."
>
> in the planning for 3.1 I stated that I don't see Classpath as our
> long term class library, assuming it in the thread model, to me,
> appears wrong.
>
> Dave> There's a default trivial implementation where those methods are
> implemented as sysCalls to C code that wraps Harmony threading APIs.
> That's native threading against Harmony/pthreads. Perhaps this is very
> close to (1) and is just some ifdefs in the C code. I haven't looked
> at the Harmony library enough to know.
>
> this doesn't fit Harmony. In the current threading code we assume  
> cond locks.
>
> Dave> There's an implementation, most likely written in Java, that is
> not at all trivial where something functionally like the m-n threading
> we had before is provided. Perhaps some old green threading code can
> be re-used. A key question here is do you want n-1 or n-m. n-1 is
> significantly simpler to implement. This is really a question about
> use cases and target platforms. In theory we might want to have both
> n-1 and n-m.
>
> So this appears to be creating a task for people working off green
> threads to create a new green thread implementation. The expedient
> thing is to work off where we were (sufficient to run all
> applications, X10 and the majority of the JSR166 TCK).
>
> Dave> The main point is that we avoid re-introducing scheduler APIs in
> the default case where we are running with native threading. Instead,
> green threading sits below the rest of the JVM and Jikes RVM as a
> whole can be oblivious to how those APIs are implemented. I believe
> this should yield a simpler and easier to maintain green threading
> layer than what we had before (which had 10+ years of evolution and
> multiple attempts to accommodate hostile native code embedded in it).
> I'm sure it will yield a simpler and easier to maintain native
> threading layer.
>
> But we inherently get a non-metacircular implementation and have to
> reinvent the green thread world.
>
> Fil, look at the Harmony thread API and you'll see what my point is. I
> agree that there are changes elsewhere in the scheduler API and with
> reintroducing green threads and a harmony model they should be adhered
> to. What I disagree with is exposing the cond locks in the scheduler.
>
> Daniel> The previous implementation of Harmony was not ready for
> production use. From the tests we were running it never supported
> multiple processors, and when running with a single processor had a
> significant number of failures. Obviously moving to a system without
> green threads makes running with a single processor no longer
> possible. This was a well understood problem, and work on Harmony to
> fix this issue was deliberately delayed until after native threads was
> implemented.
>
> I agree, we have been running against and leading Classpath
> development for a while. Harmony support was at a good level. We ran
> SPEC JVM 2008, which I see as essential for a research virtual
> machine. There is much research that can be done on a single core.
>
> Daniel> This relates to a significant issue that I don't think was
> adequately addressed at the time. The announcement of our initial
> compatibility/support for the Harmony libraries was premature, and I
> know made a lot of the core team uncomfortable. Such announcements
> have the potential to significantly affect the credibility of Jikes
> RVM, and should not be made without some consultation within the core
> team. What we claim to support officially in releases of JikesRVM has
> a direct impact on everyone working on the project.
>
> So this is highly irritating. If you have concerns:
> 1) e-mail me, and/or
> 2) open JIRAs, there's a whole place for these in the Harmony  
> support, and/or
> 3) mail the core mailing list
> Saying that the Harmony support was announced prematurely should
> really have been stated at the time, when it could have been
> rectified. Saying it now is just ridiculous.
>
> In terms of credibility, it is far far more important for our
> credibility that we run on a class library that will give enough
> support to at a least run major benchmark suites. In this case SPEC
> JVM 2008.
>
> I would dearly love help with the Harmony implementation, so far
> Alexey has been the only contributor.
>
> Daniel> I think it is clear that we all want to work toward
> compatibility with Harmony (and possibly OpenJDK) but we need to do so
> carefully.
>
> As we were already compatible with Harmony, I don't know why we're
> taking a view that we're not.
>
> Daniel> I have left aside my thoughts on the low level technical
> detail here; it is important to take a step back from that and solve
> the strategic issues first.
>
> I agree. We either have a minimalist interface or something where we
> have multiple configurations. I need multiple configurations and if I
> can't do that against the trunk I will have to do my own thing.
>
> In terms of premature decisions, I think merging Fil's code with the
> trunk was premature. We should have worked these issues out first. We
> should have closed the JIRAs on RVM-91 first. We should have asked the
> core list before the merge, first.
>
> Ian
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San  
> Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the  
> Enterprise
> -Strategies to boost innovation and cut costs with open source  
> participation
> -Receive a $600 discount off the registration fee with the source  
> code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

2009/2/20 Filip Pizlo <pizlo@...>:
> Ian,
>
> I've looked at the Harmony threading APIs.  The hythread API has
> "monitors".  We can just use these as cond locks.

Yep, I've been saying this but it is inherently ugly and would require
changing the interface to fit (why change the interface when it's not
necessary for Classpath/OpenJDK..).

> Or we could just use pthread mutexes and cond's just as we're doing
> now.  This doesn't really hurt portability.  Even Win32 has cond locks
> now...  I don't see anything wrong with having some #ifdefs for code
> like this.  It would be ~40 lines of isolated nastiness in sys.C (or
> its equivalent).

Except you need to change all our cond lock code to also acquire and
have behind it a monitor. This ends up being both ugly and
inefficient.

> Or are you suggesting that we use hythread's monitors as our Java
> monitors?  And have pointers from the Java head into the C heap?  And
> require fat lock acquisition to do a C downcall?  That could get quite
> messy, given how our thin locks rely on very specific invariants from
> our fat locks, which essentially forces us to implement fat locks
> ourselves.  I guess my whole problem here is that I don't see what
> design you're arguing for.
>
> -F

I am proposing this as the interface between cond lock and the monitor
doesn't fit. If we use cond locks we end up with a stack of:

Java Monitors (on contention, sleeping, etc. require)
Jikes Cond Locks (are implemented by)
Harmony Monitors (are implemented by)
APR Condition Variables, etc. (are implemented by)
Pthread Condition Variables, etc.

I'm proposing:

Java Monitors (on contention, sleeping, etc. require)
Harmony Monitors (are implemented by)
APR Condition Variables, etc. (are implemented by)
Pthread Condition Variables, etc.

The reason is to avoid requiring Jikes Cond Locks to know anything
about Harmony Monitors. In the Classpath case we have a stack of:

Java Monitors (on contention, sleeping, etc. require)
Jikes Cond Locks (are implemented by)
Pthread Condition Variables, etc.

And I agree that's entirely sensible and shouldn't know anything about
Harmony's monitors.

What I propose is that we start from where we are, implement Harmony
support by refactoring RVMThread into an abstraction again. In the
Harmony implementation we go through to Harmony syscalls, in the
native thread implementation we go through to pthread syscalls. We use
an abstraction rather than if-then-else as we're an OO metacircular VM
and that's just how you do things.

I'd go further and say that once we're in this position we're not far
from having an abstraction where green threads would once again fit,
but having processor local is a useful abstraction rather than thread
in that circumstance. So I think the way forward is:

1) start with Classpath and native threads

2) prioritize getting Harmony working by abstracting the current
native threads to support 2 sys call implementations. As this would
mean Jikes Cond Locks were a feature of Classpath/pthread/OpenJDK
native threading it would make sense to move them a package below
scheduler. I think this task should block a major release as it will
get Harmony support back again (an expedient to get a release out in
the mean-time would be to work from before native threading and add in
the half-dozen or so bug fixes, I'd propose this be 3.0.2 - this work
should be off green threads due to Harmony support)

3) as a low-priority for people like me, get green thread support back
by fitting it into this abstraction and extending where appropriate

I think we're all nearly on the same page, so possibly +1s for this
proposal would be useful.

Ian

> On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:
>
>> So firstly here is the Harmony thread API:
>>
>> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>>
>> I have problems with pretty much everything that's been said here:
>>
>> Dave> There's a default trivial implementation where those methods are
>> implemented as sysCalls to C code that wraps pthread calls. That's
>> native threading against GNU Classpath/pthreads."
>>
>> in the planning for 3.1 I stated that I don't see Classpath as our
>> long term class library, assuming it in the thread model, to me,
>> appears wrong.
>>
>> Dave> There's a default trivial implementation where those methods are
>> implemented as sysCalls to C code that wraps Harmony threading APIs.
>> That's native threading against Harmony/pthreads. Perhaps this is very
>> close to (1) and is just some ifdefs in the C code. I haven't looked
>> at the Harmony library enough to know.
>>
>> this doesn't fit Harmony. In the current threading code we assume
>> cond locks.
>>
>> Dave> There's an implementation, most likely written in Java, that is
>> not at all trivial where something functionally like the m-n threading
>> we had before is provided. Perhaps some old green threading code can
>> be re-used. A key question here is do you want n-1 or n-m. n-1 is
>> significantly simpler to implement. This is really a question about
>> use cases and target platforms. In theory we might want to have both
>> n-1 and n-m.
>>
>> So this appears to be creating a task for people working off green
>> threads to create a new green thread implementation. The expedient
>> thing is to work off where we were (sufficient to run all
>> applications, X10 and the majority of the JSR166 TCK).
>>
>> Dave> The main point is that we avoid re-introducing scheduler APIs in
>> the default case where we are running with native threading. Instead,
>> green threading sits below the rest of the JVM and Jikes RVM as a
>> whole can be oblivious to how those APIs are implemented. I believe
>> this should yield a simpler and easier to maintain green threading
>> layer than what we had before (which had 10+ years of evolution and
>> multiple attempts to accommodate hostile native code embedded in it).
>> I'm sure it will yield a simpler and easier to maintain native
>> threading layer.
>>
>> But we inherently get a non-metacircular implementation and have to
>> reinvent the green thread world.
>>
>> Fil, look at the Harmony thread API and you'll see what my point is. I
>> agree that there are changes elsewhere in the scheduler API and with
>> reintroducing green threads and a harmony model they should be adhered
>> to. What I disagree with is exposing the cond locks in the scheduler.
>>
>> Daniel> The previous implementation of Harmony was not ready for
>> production use. From the tests we were running it never supported
>> multiple processors, and when running with a single processor had a
>> significant number of failures. Obviously moving to a system without
>> green threads makes running with a single processor no longer
>> possible. This was a well understood problem, and work on Harmony to
>> fix this issue was deliberately delayed until after native threads was
>> implemented.
>>
>> I agree, we have been running against and leading Classpath
>> development for a while. Harmony support was at a good level. We ran
>> SPEC JVM 2008, which I see as essential for a research virtual
>> machine. There is much research that can be done on a single core.
>>
>> Daniel> This relates to a significant issue that I don't think was
>> adequately addressed at the time. The announcement of our initial
>> compatibility/support for the Harmony libraries was premature, and I
>> know made a lot of the core team uncomfortable. Such announcements
>> have the potential to significantly affect the credibility of Jikes
>> RVM, and should not be made without some consultation within the core
>> team. What we claim to support officially in releases of JikesRVM has
>> a direct impact on everyone working on the project.
>>
>> So this is highly irritating. If you have concerns:
>> 1) e-mail me, and/or
>> 2) open JIRAs, there's a whole place for these in the Harmony
>> support, and/or
>> 3) mail the core mailing list
>> Saying that the Harmony support was announced prematurely should
>> really have been stated at the time, when it could have been
>> rectified. Saying it now is just ridiculous.
>>
>> In terms of credibility, it is far far more important for our
>> credibility that we run on a class library that will give enough
>> support to at a least run major benchmark suites. In this case SPEC
>> JVM 2008.
>>
>> I would dearly love help with the Harmony implementation, so far
>> Alexey has been the only contributor.
>>
>> Daniel> I think it is clear that we all want to work toward
>> compatibility with Harmony (and possibly OpenJDK) but we need to do so
>> carefully.
>>
>> As we were already compatible with Harmony, I don't know why we're
>> taking a view that we're not.
>>
>> Daniel> I have left aside my thoughts on the low level technical
>> detail here; it is important to take a step back from that and solve
>> the strategic issues first.
>>
>> I agree. We either have a minimalist interface or something where we
>> have multiple configurations. I need multiple configurations and if I
>> can't do that against the trunk I will have to do my own thing.
>>
>> In terms of premature decisions, I think merging Fil's code with the
>> trunk was premature. We should have worked these issues out first. We
>> should have closed the JIRAs on RVM-91 first. We should have asked the
>> core list before the merge, first.
>>
>> Ian
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source
>> participation
>> -Receive a $600 discount off the registration fee with the source
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Daniel Frampton]

I am happy to have something like:

abstract class OSThreadInterface
 static final osti;
 abstract create_mutex()
...
etc.

With a concrete singleton HyThreadInterface and PThreadInterface that
calls the appropriate syscalls. This just pushes the ifdefs into Java,
which might have an advantage. We have more flexibility to pull other
things into here also.

At one level this is not much different from the syscall solution, but
it might make life easier. If others feel strongly I will defer.

It is possible to conflate this with RVMThread, but I am less
convinced this is the right thing to do. Having it separate should
make it remain a crisp interface.

On the question of condition variables vs. monitors, we pragmatically
pick what is best for what we care about most now. Because you can
implement them using each other, this isnt really a big deal now. We
can make the system work first and change this without a lot of effort
later, so lets not get hung up on it like it is a critical issue.
Perhaps we change this when Harmony is our default implementation.

It is really essential here that we try and cut to the core of what is
important, it is too easy to get sucked into detail that really
confuses the issue. The details fall into place once it is clear where
we want to head.

Cheers,
Daniel.

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

2009/2/20 Daniel Frampton <zyridium@...>:

> I am happy to have something like:
>
> abstract class OSThreadInterface
>  static final osti;
>  abstract create_mutex()
> ...
> etc.
>
> With a concrete singleton HyThreadInterface and PThreadInterface that
> calls the appropriate syscalls. This just pushes the ifdefs into Java,
> which might have an advantage. We have more flexibility to pull other
> things into here also.

So the problem here is that we're exposing mutexes and other pthread
data structures via the API. Harmony provides what is essentially the
meat behind the RVMThread API which the current code in trunk builds
out of pthreads. Building pthread data structures out of Harmony's is
adding complexity.

> At one level this is not much different from the syscall solution, but
> it might make life easier. If others feel strongly I will defer.
>
> It is possible to conflate this with RVMThread, but I am less
> convinced this is the right thing to do. Having it separate should
> make it remain a crisp interface.

I agree with the intention, the fact that things sub-class RVMThread
means having a single RVMThread is nice and it'd be good to push
things into helper abstractions. Not having the helper abstractions
expose thread model implementation details would be good.

> On the question of condition variables vs. monitors, we pragmatically
> pick what is best for what we care about most now. Because you can
> implement them using each other, this isnt really a big deal now. We
> can make the system work first and change this without a lot of effort
> later, so lets not get hung up on it like it is a critical issue.
> Perhaps we change this when Harmony is our default implementation.
>
> It is really essential here that we try and cut to the core of what is
> important, it is too easy to get sucked into detail that really
> confuses the issue. The details fall into place once it is clear where
> we want to head.
>
> Cheers,
> Daniel.

If I can expand on my motivation more, if we build Harmony's threading
out of the existing threading code we must implement thread and lock
support. This can be a simple matter of attaching Harmony thread's to
our threads and making a CondLock out of monitors. So where is the
problem? Well making CondLocks out of monitors isn't very nice, we're
essentially building pthread building blocks out of Java building
blocks to implement Java building blocks. Ignoring this for now, the
major problem is that we are building a thread interface that won't
port to Windows or other Harmony platforms. To port this solution to
Windows all of our thread code would need rewriting to be Windows
threads, and this would also require rewriting the signal handling
code.

Ideally the thread API will be as simple and robust as possible. For
Harmony my firm belief is this means a call to sleep should bottom out
in hythread_sleep, not as a call to wait on a monitor representing a
CondLock.

If we abstract at the level of calls into RVMThread, using helpers if
necessary to avoid the inheritance problem, then we leave the door
open to bring back the green thread code. This solves my 2 problems
with the current scheduler design, lack of green threading, simple and
robust port of Harmony to all architectures Jikes RVM supports (and
operating systems Harmony supports). So getting the abstraction back
into the RVM is really worth it!

Ian

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[David P Grove]

Also stepping back, I'd like to get a better understanding for the scenarios where you think it is desirable to apply green threading. This is going to impact what functions we need to support. Here are some possibilities I can think of, are one or more of these what is driving you, or have I missed a key scenario?

(a) Running Jikes RVM on a platform that doesn't have a native threading layer. Some of the questions here would be:
(1) Is the platform multi-core, so we need m-n, or is it single core, in which case n-1 is most likely what is actually needed.
(2) It would seem possible at least that you don't need syswrap and all of the other attempts to make blocking native code work with greenthreading in this scenario. If the platform doesn't have native threading, I'd speculate that it might not have native APIs like select, poll, etc. that we hijack.

(b) Running Jikes RVM on a platform that has a native threading layer, but you want to run more Java threads than the native threading layer can efficiently support. This was the main original motivation for m-n threading in Jikes RVM. I think improvements in native threading layers since 1997 have made this a significantly less compelling scenario. There are still cases where it might make sense, but they are fairly specialized.
For what it's worth, I'm working on X10 now, and with the X10 1.7 runtime design, green threading is not an option we are pursuing for efficient implementation of asyncs.

(c) Running Jikes RVM on a platform that has a native threading layer and running "standard" Java applications with thread counts that the native threading layer can efficiently support. (ie, every single one of the standard benchmarks used in typical research papers). I'm unconvinced that this scenario is worth supporting, but after spending so much time over the last 10 years trying to get it to work well, I admit I'm perhaps irrationally prejudiced against it.

--dave

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

So firstly the need for abstraction has been highlighted by getting
Harmony threads right, green threads is something else I want but I'm
not sure what percentage of the code in this direction will be made
public in the short term. Ideally we'd have a green thread
implementation on non-bare metal so that it can be kept in good shape.

2009/2/20 David P Grove <groved@...>:
> Also stepping back, I'd like to get a better understanding for the scenarios
> where you think it is desirable to apply green threading. This is going to
> impact what functions we need to support. Here are some possibilities I can
> think of, are one or more of these what is driving you, or have I missed a
> key scenario?
>
> (a) Running Jikes RVM on a platform that doesn't have a native threading
> layer. Some of the questions here would be:

This is our scenario.

> (1) Is the platform multi-core, so we need m-n, or is it single core, in
> which case n-1 is most likely what is actually needed.

Multi-core.

> (2) It would seem possible at least that you don't need syswrap and all of
> the other attempts to make blocking native code work with greenthreading in
> this scenario. If the platform doesn't have native threading, I'd speculate
> that it might not have native APIs like select, poll, etc. that we hijack.

We don't need syswrap, but to test green threads away from our effort
then syswrap may be necessary. However, syswrap shouldn't be brought
in for any of the native thread implementations.

> (b) Running Jikes RVM on a platform that has a native threading layer, but
> you want to run more Java threads than the native threading layer can
> efficiently support. This was the main original motivation for m-n threading
> in Jikes RVM. I think improvements in native threading layers since 1997
> have made this a significantly less compelling scenario. There are still
> cases where it might make sense, but they are fairly specialized.

This scenario doesn't apply to us.

> For what it's worth, I'm working on X10 now, and with the X10 1.7 runtime
> design, green threading is not an option we are pursuing for efficient
> implementation of asyncs.

Right, with the java.util.concurrent Executors framework the
efficiency point is pretty much gone.

> (c) Running Jikes RVM on a platform that has a native threading layer and
> running "standard" Java applications with thread counts that the native
> threading layer can efficiently support. (ie, every single one of the
> standard benchmarks used in typical research papers). I'm unconvinced that
> this scenario is worth supporting, but after spending so much time over the
> last 10 years trying to get it to work well, I admit I'm perhaps
> irrationally prejudiced against it.
>
> --dave

I'm not sure what you mean by this one, but as we don't have a native
thread layer this scenario doesn't apply.

Regards,
Ian

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Filip Pizlo-2]

Ian,

We could leave HeavyCondLock unchanged while supporting Harmony.  It  
can continue to use pthread_cond and pthread_mutex, and everything  
would just work.

What we seem to need, from a threading standpoint, for Harmony is:

1) That its threading API be initialized so that the classlib can use  
it without dying, and

2) That each thread is at the end of the day a Harmony thread, so that  
Harmony TLS and such is guaranteed to work.

This is not difficult to do.  The rest of this discussion strikes me  
as somewhat superfluous.  Why do you want to provide these  
abstractions?  What's wrong with having the same threading  
infrastructure with all class libraries?

>> I've looked at the Harmony threading APIs.  The hythread API has
>> "monitors".  We can just use these as cond locks.
>
> Yep, I've been saying this but it is inherently ugly and would require
> changing the interface to fit (why change the interface when it's not
> necessary for Classpath/OpenJDK..).

It's not ugly.

>
>
>> Or we could just use pthread mutexes and cond's just as we're doing
>> now.  This doesn't really hurt portability.  Even Win32 has cond  
>> locks
>> now...  I don't see anything wrong with having some #ifdefs for code
>> like this.  It would be ~40 lines of isolated nastiness in sys.C (or
>> its equivalent).
>
> Except you need to change all our cond lock code to also acquire and
> have behind it a monitor. This ends up being both ugly and
> inefficient.

I don't get this.  How is it ugly?  And who cares if it's  
inefficient?  You realize that this code is only called on slow paths,  
right?  And that most of it is already baseline compiled, not inlined,  
and makes in some cases multiple sysCalls?  None of this is fast,  
because we don't need it to be!

>
>
>> Or are you suggesting that we use hythread's monitors as our Java
>> monitors?  And have pointers from the Java head into the C heap?  And
>> require fat lock acquisition to do a C downcall?  That could get  
>> quite
>> messy, given how our thin locks rely on very specific invariants from
>> our fat locks, which essentially forces us to implement fat locks
>> ourselves.  I guess my whole problem here is that I don't see what
>> design you're arguing for.
>>
>> -F
>
> I am proposing this as the interface between cond lock and the monitor
> doesn't fit. If we use cond locks we end up with a stack of:
>
> Java Monitors (on contention, sleeping, etc. require)
> Jikes Cond Locks (are implemented by)
> Harmony Monitors (are implemented by)
> APR Condition Variables, etc. (are implemented by)
> Pthread Condition Variables, etc.
>
> I'm proposing:
>
> Java Monitors (on contention, sleeping, etc. require)
> Harmony Monitors (are implemented by)
> APR Condition Variables, etc. (are implemented by)
> Pthread Condition Variables, etc.

How do you envision handling safepoints?  The whole point of cond  
locks is to gracefully handle asynchronous block and soft handshake  
requests that occur while holding, waiting on, or attempting to  
acquire a cond lock.

I don't see similar functionality in Harmony Monitors.  However, we  
can keep HeavyCondLock's API and implement it on top of Harmony  
Monitors, and everything would be fine.  But this would be a waste of  
time in my opinion, because there is no good reason to use Harmony  
monitors when we could just as easily continue to use pthread_cond and  
pthread_mutex.

>
>
> The reason is to avoid requiring Jikes Cond Locks to know anything
> about Harmony Monitors. In the Classpath case we have a stack of:
>
> Java Monitors (on contention, sleeping, etc. require)
> Jikes Cond Locks (are implemented by)
> Pthread Condition Variables, etc.
>
> And I agree that's entirely sensible and shouldn't know anything about
> Harmony's monitors.

All of this seems to me to be a very dangerous idea.

>
>
> What I propose is that we start from where we are, implement Harmony
> support by refactoring RVMThread into an abstraction again. In the
> Harmony implementation we go through to Harmony syscalls, in the
> native thread implementation we go through to pthread syscalls. We use
> an abstraction rather than if-then-else as we're an OO metacircular VM
> and that's just how you do things.
>

So you're suggesting that when running on Harmony we let Harmony  
implement RVMThread?  In C?  And that's OO metacircularity?

> I'd go further and say that once we're in this position we're not far
> from having an abstraction where green threads would once again fit,
> but having processor local is a useful abstraction rather than thread
> in that circumstance. So I think the way forward is:
>
> 1) start with Classpath and native threads
>
> 2) prioritize getting Harmony working by abstracting the current
> native threads to support 2 sys call implementations. As this would
> mean Jikes Cond Locks were a feature of Classpath/pthread/OpenJDK
> native threading it would make sense to move them a package below
> scheduler. I think this task should block a major release as it will
> get Harmony support back again (an expedient to get a release out in
> the mean-time would be to work from before native threading and add in
> the half-dozen or so bug fixes, I'd propose this be 3.0.2 - this work
> should be off green threads due to Harmony support)

I'm confused.  You are making reference to having 2 sys call  
implementations.  Are you suggesting two implementations of  
HeavyCondLock or two implementations of RVMThread?

HeavyCondLock is not just an API wrapper around pthread_cond_t and  
pthread_mutex_t - it actually has added functionality.  So the stacks  
you'd want, if you really wanted to use Harmony's monitors (which I  
think is pointless), would be:

Classpath:   HeavyCondLock -> abstraction for creating mutex/cond pair  
-> pthreads

Harmony:   HeavyCondLock -> abstraction for creating mutex/cond pair -
 > Harmony

And so what you'd be changing is that middle abstraction, rather than  
the HeavyCondLock itself, which actually contains useful  
functionality.  In particular, the HeavyCondLock, and all of its  
Nicely and non-Nicely methods that you like so much, play a crucial  
role in ensuring that we:

1) Support proper thread accounting while avoiding deadlock and  
spinning.

2) Support proper thread-to-thread notification while avoiding  
deadlock and spinning.

3) Support proper soft handshaking (a feature we care about deeply!)  
while avoiding deadlock  and spinning.

4) Support proper hard handshaking (i.e. GC stop-the-world) while  
avoiding deadlock and spinning.

5) Support proper thread hijacking (so - a hard handshake between just  
two threads) while avoiding deadlock and spinning.  This is kind of  
what suspend() does.  But we also need it to support biased locks.

If we used Harmony monitors directly, this would all become harder,  
because we would not be able to do things like reset the recursion  
count of the lock, which would then make it harder to do things like  
waitNicely(), which is really a central method in how native threads  
works.

>
>
> 3) as a low-priority for people like me, get green thread support back
> by fitting it into this abstraction and extending where appropriate
>
> I think we're all nearly on the same page, so possibly +1s for this
> proposal would be useful.

I don't know what your proposal is yet.  Sometimes you are talking  
about replacing RVMThread, other times you are talking about  
abstracting system calls.  Furthermore, I don't see the point of it.  
The threading code could be largely kept unchanged while supporting  
Harmony.  You have not convinced me otherwise.

-F

>
>
> Ian
>
>> On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:
>>
>>> So firstly here is the Harmony thread API:
>>>
>>> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>>>
>>> I have problems with pretty much everything that's been said here:
>>>
>>> Dave> There's a default trivial implementation where those methods  
>>> are
>>> implemented as sysCalls to C code that wraps pthread calls. That's
>>> native threading against GNU Classpath/pthreads."
>>>
>>> in the planning for 3.1 I stated that I don't see Classpath as our
>>> long term class library, assuming it in the thread model, to me,
>>> appears wrong.
>>>
>>> Dave> There's a default trivial implementation where those methods  
>>> are
>>> implemented as sysCalls to C code that wraps Harmony threading APIs.
>>> That's native threading against Harmony/pthreads. Perhaps this is  
>>> very
>>> close to (1) and is just some ifdefs in the C code. I haven't looked
>>> at the Harmony library enough to know.
>>>
>>> this doesn't fit Harmony. In the current threading code we assume
>>> cond locks.
>>>
>>> Dave> There's an implementation, most likely written in Java, that  
>>> is
>>> not at all trivial where something functionally like the m-n  
>>> threading
>>> we had before is provided. Perhaps some old green threading code can
>>> be re-used. A key question here is do you want n-1 or n-m. n-1 is
>>> significantly simpler to implement. This is really a question about
>>> use cases and target platforms. In theory we might want to have both
>>> n-1 and n-m.
>>>
>>> So this appears to be creating a task for people working off green
>>> threads to create a new green thread implementation. The expedient
>>> thing is to work off where we were (sufficient to run all
>>> applications, X10 and the majority of the JSR166 TCK).
>>>
>>> Dave> The main point is that we avoid re-introducing scheduler  
>>> APIs in
>>> the default case where we are running with native threading.  
>>> Instead,
>>> green threading sits below the rest of the JVM and Jikes RVM as a
>>> whole can be oblivious to how those APIs are implemented. I believe
>>> this should yield a simpler and easier to maintain green threading
>>> layer than what we had before (which had 10+ years of evolution and
>>> multiple attempts to accommodate hostile native code embedded in  
>>> it).
>>> I'm sure it will yield a simpler and easier to maintain native
>>> threading layer.
>>>
>>> But we inherently get a non-metacircular implementation and have to
>>> reinvent the green thread world.
>>>
>>> Fil, look at the Harmony thread API and you'll see what my point  
>>> is. I
>>> agree that there are changes elsewhere in the scheduler API and with
>>> reintroducing green threads and a harmony model they should be  
>>> adhered
>>> to. What I disagree with is exposing the cond locks in the  
>>> scheduler.
>>>
>>> Daniel> The previous implementation of Harmony was not ready for
>>> production use. From the tests we were running it never supported
>>> multiple processors, and when running with a single processor had a
>>> significant number of failures. Obviously moving to a system without
>>> green threads makes running with a single processor no longer
>>> possible. This was a well understood problem, and work on Harmony to
>>> fix this issue was deliberately delayed until after native threads  
>>> was
>>> implemented.
>>>
>>> I agree, we have been running against and leading Classpath
>>> development for a while. Harmony support was at a good level. We ran
>>> SPEC JVM 2008, which I see as essential for a research virtual
>>> machine. There is much research that can be done on a single core.
>>>
>>> Daniel> This relates to a significant issue that I don't think was
>>> adequately addressed at the time. The announcement of our initial
>>> compatibility/support for the Harmony libraries was premature, and I
>>> know made a lot of the core team uncomfortable. Such announcements
>>> have the potential to significantly affect the credibility of Jikes
>>> RVM, and should not be made without some consultation within the  
>>> core
>>> team. What we claim to support officially in releases of JikesRVM  
>>> has
>>> a direct impact on everyone working on the project.
>>>
>>> So this is highly irritating. If you have concerns:
>>> 1) e-mail me, and/or
>>> 2) open JIRAs, there's a whole place for these in the Harmony
>>> support, and/or
>>> 3) mail the core mailing list
>>> Saying that the Harmony support was announced prematurely should
>>> really have been stated at the time, when it could have been
>>> rectified. Saying it now is just ridiculous.
>>>
>>> In terms of credibility, it is far far more important for our
>>> credibility that we run on a class library that will give enough
>>> support to at a least run major benchmark suites. In this case SPEC
>>> JVM 2008.
>>>
>>> I would dearly love help with the Harmony implementation, so far
>>> Alexey has been the only contributor.
>>>
>>> Daniel> I think it is clear that we all want to work toward
>>> compatibility with Harmony (and possibly OpenJDK) but we need to  
>>> do so
>>> carefully.
>>>
>>> As we were already compatible with Harmony, I don't know why we're
>>> taking a view that we're not.
>>>
>>> Daniel> I have left aside my thoughts on the low level technical
>>> detail here; it is important to take a step back from that and solve
>>> the strategic issues first.
>>>
>>> I agree. We either have a minimalist interface or something where we
>>> have multiple configurations. I need multiple configurations and  
>>> if I
>>> can't do that against the trunk I will have to do my own thing.
>>>
>>> In terms of premature decisions, I think merging Fil's code with the
>>> trunk was premature. We should have worked these issues out first.  
>>> We
>>> should have closed the JIRAs on RVM-91 first. We should have asked  
>>> the
>>> core list before the merge, first.
>>>
>>> Ian
>>>
>>> ------------------------------------------------------------------------------
>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>> Francisco, CA
>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>> Enterprise
>>> -Strategies to boost innovation and cut costs with open source
>>> participation
>>> -Receive a $600 discount off the registration fee with the source
>>> code: SFAD
>>> http://p.sf.net/sfu/XcvMzF8H
>>> _______________________________________________
>>> Jikesrvm-core mailing list
>>> Jikesrvm-core@...
>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San  
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the  
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source  
>> participation
>> -Receive a $600 discount off the registration fee with the source  
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San  
> Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the  
> Enterprise
> -Strategies to boost innovation and cut costs with open source  
> participation
> -Receive a $600 discount off the registration fee with the source  
> code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

2009/2/20 Filip Pizlo <pizlo@...>:

> Ian,
>
> We could leave HeavyCondLock unchanged while supporting Harmony.  It
> can continue to use pthread_cond and pthread_mutex, and everything
> would just work.
>
> What we seem to need, from a threading standpoint, for Harmony is:
>
> 1) That its threading API be initialized so that the classlib can use
> it without dying, and
>
> 2) That each thread is at the end of the day a Harmony thread, so that
> Harmony TLS and such is guaranteed to work.
>
> This is not difficult to do.  The rest of this discussion strikes me
> as somewhat superfluous.  Why do you want to provide these
> abstractions?  What's wrong with having the same threading
> infrastructure with all class libraries?

We don't have pthreads on Windows. I want the Harmony code to be
portable, simple and robust.

>>> I've looked at the Harmony threading APIs.  The hythread API has
>>> "monitors".  We can just use these as cond locks.
>>
>> Yep, I've been saying this but it is inherently ugly and would require
>> changing the interface to fit (why change the interface when it's not
>> necessary for Classpath/OpenJDK..).
>
> It's not ugly.
>
>>
>>
>>> Or we could just use pthread mutexes and cond's just as we're doing
>>> now.  This doesn't really hurt portability.  Even Win32 has cond
>>> locks
>>> now...  I don't see anything wrong with having some #ifdefs for code
>>> like this.  It would be ~40 lines of isolated nastiness in sys.C (or
>>> its equivalent).
>>
>> Except you need to change all our cond lock code to also acquire and
>> have behind it a monitor. This ends up being both ugly and
>> inefficient.
>
> I don't get this.  How is it ugly?  And who cares if it's
> inefficient?  You realize that this code is only called on slow paths,
> right?  And that most of it is already baseline compiled, not inlined,
> and makes in some cases multiple sysCalls?  None of this is fast,
> because we don't need it to be!
>
>>
>>
>>> Or are you suggesting that we use hythread's monitors as our Java
>>> monitors?  And have pointers from the Java head into the C heap?  And
>>> require fat lock acquisition to do a C downcall?  That could get
>>> quite
>>> messy, given how our thin locks rely on very specific invariants from
>>> our fat locks, which essentially forces us to implement fat locks
>>> ourselves.  I guess my whole problem here is that I don't see what
>>> design you're arguing for.
>>>
>>> -F
>>
>> I am proposing this as the interface between cond lock and the monitor
>> doesn't fit. If we use cond locks we end up with a stack of:
>>
>> Java Monitors (on contention, sleeping, etc. require)
>> Jikes Cond Locks (are implemented by)
>> Harmony Monitors (are implemented by)
>> APR Condition Variables, etc. (are implemented by)
>> Pthread Condition Variables, etc.
>>
>> I'm proposing:
>>
>> Java Monitors (on contention, sleeping, etc. require)
>> Harmony Monitors (are implemented by)
>> APR Condition Variables, etc. (are implemented by)
>> Pthread Condition Variables, etc.
>
> How do you envision handling safepoints?  The whole point of cond
> locks is to gracefully handle asynchronous block and soft handshake
> requests that occur while holding, waiting on, or attempting to
> acquire a cond lock.

Copy DRLVM as the reference Harmony VM implementation (that doesn't
require cond locks). The point is not to reinvent a wheel. In the case
of DRLVM for sleep, it changes the thread status and calls
hythread_sleep.

> I don't see similar functionality in Harmony Monitors.  However, we
> can keep HeavyCondLock's API and implement it on top of Harmony
> Monitors, and everything would be fine.  But this would be a waste of
> time in my opinion, because there is no good reason to use Harmony
> monitors when we could just as easily continue to use pthread_cond and
> pthread_mutex.

pthread != Windows != any other Harmony platform where the job of
porting hythread has been tackled

>> The reason is to avoid requiring Jikes Cond Locks to know anything
>> about Harmony Monitors. In the Classpath case we have a stack of:
>>
>> Java Monitors (on contention, sleeping, etc. require)
>> Jikes Cond Locks (are implemented by)
>> Pthread Condition Variables, etc.
>>
>> And I agree that's entirely sensible and shouldn't know anything about
>> Harmony's monitors.
>
> All of this seems to me to be a very dangerous idea.
>
>>
>>
>> What I propose is that we start from where we are, implement Harmony
>> support by refactoring RVMThread into an abstraction again. In the
>> Harmony implementation we go through to Harmony syscalls, in the
>> native thread implementation we go through to pthread syscalls. We use
>> an abstraction rather than if-then-else as we're an OO metacircular VM
>> and that's just how you do things.
>>
>
> So you're suggesting that when running on Harmony we let Harmony
> implement RVMThread?  In C?  And that's OO metacircularity?

No, in green threads we are metacircular. For Harmony we do the simple
& robust thing.

>> I'd go further and say that once we're in this position we're not far
>> from having an abstraction where green threads would once again fit,
>> but having processor local is a useful abstraction rather than thread
>> in that circumstance. So I think the way forward is:
>>
>> 1) start with Classpath and native threads
>>
>> 2) prioritize getting Harmony working by abstracting the current
>> native threads to support 2 sys call implementations. As this would
>> mean Jikes Cond Locks were a feature of Classpath/pthread/OpenJDK
>> native threading it would make sense to move them a package below
>> scheduler. I think this task should block a major release as it will
>> get Harmony support back again (an expedient to get a release out in
>> the mean-time would be to work from before native threading and add in
>> the half-dozen or so bug fixes, I'd propose this be 3.0.2 - this work
>> should be off green threads due to Harmony support)
>
> I'm confused.  You are making reference to having 2 sys call
> implementations.  Are you suggesting two implementations of
> HeavyCondLock or two implementations of RVMThread?
>
> HeavyCondLock is not just an API wrapper around pthread_cond_t and
> pthread_mutex_t - it actually has added functionality.  So the stacks
> you'd want, if you really wanted to use Harmony's monitors (which I
> think is pointless), would be:
>
> Classpath:   HeavyCondLock -> abstraction for creating mutex/cond pair
> -> pthreads
>
> Harmony:   HeavyCondLock -> abstraction for creating mutex/cond pair -
>  > Harmony
>
> And so what you'd be changing is that middle abstraction, rather than
> the HeavyCondLock itself, which actually contains useful
> functionality.  In particular, the HeavyCondLock, and all of its
> Nicely and non-Nicely methods that you like so much, play a crucial
> role in ensuring that we:
>
> 1) Support proper thread accounting while avoiding deadlock and
> spinning.
>
> 2) Support proper thread-to-thread notification while avoiding
> deadlock and spinning.
>
> 3) Support proper soft handshaking (a feature we care about deeply!)
> while avoiding deadlock  and spinning.
>
> 4) Support proper hard handshaking (i.e. GC stop-the-world) while
> avoiding deadlock and spinning.
>
> 5) Support proper thread hijacking (so - a hard handshake between just
> two threads) while avoiding deadlock and spinning.  This is kind of
> what suspend() does.  But we also need it to support biased locks.
>
> If we used Harmony monitors directly, this would all become harder,
> because we would not be able to do things like reset the recursion
> count of the lock, which would then make it harder to do things like
> waitNicely(), which is really a central method in how native threads
> works.

Cond locks are an artifact of pthreads and unnecessary for Harmony threads.

>>
>>
>> 3) as a low-priority for people like me, get green thread support back
>> by fitting it into this abstraction and extending where appropriate
>>
>> I think we're all nearly on the same page, so possibly +1s for this
>> proposal would be useful.
>
> I don't know what your proposal is yet.  Sometimes you are talking
> about replacing RVMThread, other times you are talking about
> abstracting system calls.  Furthermore, I don't see the point of it.
> The threading code could be largely kept unchanged while supporting
> Harmony.  You have not convinced me otherwise.
>
> -F

The proposal is simple, anything that is a pthread artifact and not
simply and robustly portable to Harmony is pushed into a package in
sheduler. We abstract the implementation so that pthreads/Classpath
work using the sub package. Harmony gets the standard Harmony native
thread implementation.

This abstraction should mean we can also get the previous green thread
model back.

Ian

>>
>>
>> Ian
>>
>>> On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:
>>>
>>>> So firstly here is the Harmony thread API:
>>>>
>>>> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>>>>
>>>> I have problems with pretty much everything that's been said here:
>>>>
>>>> Dave> There's a default trivial implementation where those methods
>>>> are
>>>> implemented as sysCalls to C code that wraps pthread calls. That's
>>>> native threading against GNU Classpath/pthreads."
>>>>
>>>> in the planning for 3.1 I stated that I don't see Classpath as our
>>>> long term class library, assuming it in the thread model, to me,
>>>> appears wrong.
>>>>
>>>> Dave> There's a default trivial implementation where those methods
>>>> are
>>>> implemented as sysCalls to C code that wraps Harmony threading APIs.
>>>> That's native threading against Harmony/pthreads. Perhaps this is
>>>> very
>>>> close to (1) and is just some ifdefs in the C code. I haven't looked
>>>> at the Harmony library enough to know.
>>>>
>>>> this doesn't fit Harmony. In the current threading code we assume
>>>> cond locks.
>>>>
>>>> Dave> There's an implementation, most likely written in Java, that
>>>> is
>>>> not at all trivial where something functionally like the m-n
>>>> threading
>>>> we had before is provided. Perhaps some old green threading code can
>>>> be re-used. A key question here is do you want n-1 or n-m. n-1 is
>>>> significantly simpler to implement. This is really a question about
>>>> use cases and target platforms. In theory we might want to have both
>>>> n-1 and n-m.
>>>>
>>>> So this appears to be creating a task for people working off green
>>>> threads to create a new green thread implementation. The expedient
>>>> thing is to work off where we were (sufficient to run all
>>>> applications, X10 and the majority of the JSR166 TCK).
>>>>
>>>> Dave> The main point is that we avoid re-introducing scheduler
>>>> APIs in
>>>> the default case where we are running with native threading.
>>>> Instead,
>>>> green threading sits below the rest of the JVM and Jikes RVM as a
>>>> whole can be oblivious to how those APIs are implemented. I believe
>>>> this should yield a simpler and easier to maintain green threading
>>>> layer than what we had before (which had 10+ years of evolution and
>>>> multiple attempts to accommodate hostile native code embedded in
>>>> it).
>>>> I'm sure it will yield a simpler and easier to maintain native
>>>> threading layer.
>>>>
>>>> But we inherently get a non-metacircular implementation and have to
>>>> reinvent the green thread world.
>>>>
>>>> Fil, look at the Harmony thread API and you'll see what my point
>>>> is. I
>>>> agree that there are changes elsewhere in the scheduler API and with
>>>> reintroducing green threads and a harmony model they should be
>>>> adhered
>>>> to. What I disagree with is exposing the cond locks in the
>>>> scheduler.
>>>>
>>>> Daniel> The previous implementation of Harmony was not ready for
>>>> production use. From the tests we were running it never supported
>>>> multiple processors, and when running with a single processor had a
>>>> significant number of failures. Obviously moving to a system without
>>>> green threads makes running with a single processor no longer
>>>> possible. This was a well understood problem, and work on Harmony to
>>>> fix this issue was deliberately delayed until after native threads
>>>> was
>>>> implemented.
>>>>
>>>> I agree, we have been running against and leading Classpath
>>>> development for a while. Harmony support was at a good level. We ran
>>>> SPEC JVM 2008, which I see as essential for a research virtual
>>>> machine. There is much research that can be done on a single core.
>>>>
>>>> Daniel> This relates to a significant issue that I don't think was
>>>> adequately addressed at the time. The announcement of our initial
>>>> compatibility/support for the Harmony libraries was premature, and I
>>>> know made a lot of the core team uncomfortable. Such announcements
>>>> have the potential to significantly affect the credibility of Jikes
>>>> RVM, and should not be made without some consultation within the
>>>> core
>>>> team. What we claim to support officially in releases of JikesRVM
>>>> has
>>>> a direct impact on everyone working on the project.
>>>>
>>>> So this is highly irritating. If you have concerns:
>>>> 1) e-mail me, and/or
>>>> 2) open JIRAs, there's a whole place for these in the Harmony
>>>> support, and/or
>>>> 3) mail the core mailing list
>>>> Saying that the Harmony support was announced prematurely should
>>>> really have been stated at the time, when it could have been
>>>> rectified. Saying it now is just ridiculous.
>>>>
>>>> In terms of credibility, it is far far more important for our
>>>> credibility that we run on a class library that will give enough
>>>> support to at a least run major benchmark suites. In this case SPEC
>>>> JVM 2008.
>>>>
>>>> I would dearly love help with the Harmony implementation, so far
>>>> Alexey has been the only contributor.
>>>>
>>>> Daniel> I think it is clear that we all want to work toward
>>>> compatibility with Harmony (and possibly OpenJDK) but we need to
>>>> do so
>>>> carefully.
>>>>
>>>> As we were already compatible with Harmony, I don't know why we're
>>>> taking a view that we're not.
>>>>
>>>> Daniel> I have left aside my thoughts on the low level technical
>>>> detail here; it is important to take a step back from that and solve
>>>> the strategic issues first.
>>>>
>>>> I agree. We either have a minimalist interface or something where we
>>>> have multiple configurations. I need multiple configurations and
>>>> if I
>>>> can't do that against the trunk I will have to do my own thing.
>>>>
>>>> In terms of premature decisions, I think merging Fil's code with the
>>>> trunk was premature. We should have worked these issues out first.
>>>> We
>>>> should have closed the JIRAs on RVM-91 first. We should have asked
>>>> the
>>>> core list before the merge, first.
>>>>
>>>> Ian
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>> Francisco, CA
>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>> Enterprise
>>>> -Strategies to boost innovation and cut costs with open source
>>>> participation
>>>> -Receive a $600 discount off the registration fee with the source
>>>> code: SFAD
>>>> http://p.sf.net/sfu/XcvMzF8H
>>>> _______________________________________________
>>>> Jikesrvm-core mailing list
>>>> Jikesrvm-core@...
>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>> Francisco, CA
>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>> Enterprise
>>> -Strategies to boost innovation and cut costs with open source
>>> participation
>>> -Receive a $600 discount off the registration fee with the source
>>> code: SFAD
>>> http://p.sf.net/sfu/XcvMzF8H
>>> _______________________________________________
>>> Jikesrvm-core mailing list
>>> Jikesrvm-core@...
>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source
>> participation
>> -Receive a $600 discount off the registration fee with the source
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Filip Pizlo-2]

Four things:

1) We can get Windows support with a much simpler abstraction layer  
than what you propose.  Daniel, Dave, and I have already pointed out  
various ways of doing this.  I'm not sure why you're ignoring these  
proposals.

2) The HeavyCondLock/RVMThread code is not an artifact of pthreads,  
it's an artifact of the fact that we have a high-performance VM.  And  
besides, the Harmony VM code that I've seen actually overrides quite a  
bit of the Harmony classlib hythread code with its own stuff (monitors  
in particular).  So I don't see the problem with us doing the same (in  
particular, by just keeping the code we already have and ignoring the  
code that Harmony uses).

3) Your description of how DRLVM does sleep indicates that it's either  
wrong, requires STW GC to do unnecessary spinning, requires the use of  
signaling to implement STW GC, and/or completely eliminates the  
possibility of implementing on-the-fly concurrent GC.  Any one of  
those would be unacceptable from a performance and functionality  
standpoint.

4) Even with an abstraction on RVMThread you won't get green threads  
back, since green threads cross cut other parts of the system.  This  
was the bulk of the work in implementing native threading.

Your proposal requires doing more work than we need to do in order to  
support Harmony.  You're suggesting we refactor the threading model,  
again, only this time, you want to patch it into a foreign threading  
model.  You've given no argument for why this would be necessary,  
useful, or even correct.  You think matches the semantics we want, but  
you haven't given me any argument that it does.  On the contrary,  
you've shown examples that it doesn't, and have made claims that, if  
true, would imply that running with Harmony threads as per your  
proposal would cause livelocks and deadlocks in GC unless we made a  
lot of other big changes.

The proposal that me, Dave, and Daniel have all made - abstracting  
thread creation and HeavyCondLock instead of RVMThread - allows us to  
get Harmony in there quicker with fewer code changes.

I don't think that it makes sense to argue for rewriting more code if  
the same effect can be achieved by rewriting less code.

-F



On Feb 20, 2009, at 12:04 PM, Ian Rogers wrote:

> 2009/2/20 Filip Pizlo <pizlo@...>:
>> Ian,
>>
>> We could leave HeavyCondLock unchanged while supporting Harmony.  It
>> can continue to use pthread_cond and pthread_mutex, and everything
>> would just work.
>>
>> What we seem to need, from a threading standpoint, for Harmony is:
>>
>> 1) That its threading API be initialized so that the classlib can use
>> it without dying, and
>>
>> 2) That each thread is at the end of the day a Harmony thread, so  
>> that
>> Harmony TLS and such is guaranteed to work.
>>
>> This is not difficult to do.  The rest of this discussion strikes me
>> as somewhat superfluous.  Why do you want to provide these
>> abstractions?  What's wrong with having the same threading
>> infrastructure with all class libraries?
>
> We don't have pthreads on Windows. I want the Harmony code to be
> portable, simple and robust.
>
>>>> I've looked at the Harmony threading APIs.  The hythread API has
>>>> "monitors".  We can just use these as cond locks.
>>>
>>> Yep, I've been saying this but it is inherently ugly and would  
>>> require
>>> changing the interface to fit (why change the interface when it's  
>>> not
>>> necessary for Classpath/OpenJDK..).
>>
>> It's not ugly.
>>
>>>
>>>
>>>> Or we could just use pthread mutexes and cond's just as we're doing
>>>> now.  This doesn't really hurt portability.  Even Win32 has cond
>>>> locks
>>>> now...  I don't see anything wrong with having some #ifdefs for  
>>>> code
>>>> like this.  It would be ~40 lines of isolated nastiness in sys.C  
>>>> (or
>>>> its equivalent).
>>>
>>> Except you need to change all our cond lock code to also acquire and
>>> have behind it a monitor. This ends up being both ugly and
>>> inefficient.
>>
>> I don't get this.  How is it ugly?  And who cares if it's
>> inefficient?  You realize that this code is only called on slow  
>> paths,
>> right?  And that most of it is already baseline compiled, not  
>> inlined,
>> and makes in some cases multiple sysCalls?  None of this is fast,
>> because we don't need it to be!
>>
>>>
>>>
>>>> Or are you suggesting that we use hythread's monitors as our Java
>>>> monitors?  And have pointers from the Java head into the C heap?  
>>>> And
>>>> require fat lock acquisition to do a C downcall?  That could get
>>>> quite
>>>> messy, given how our thin locks rely on very specific invariants  
>>>> from
>>>> our fat locks, which essentially forces us to implement fat locks
>>>> ourselves.  I guess my whole problem here is that I don't see what
>>>> design you're arguing for.
>>>>
>>>> -F
>>>
>>> I am proposing this as the interface between cond lock and the  
>>> monitor
>>> doesn't fit. If we use cond locks we end up with a stack of:
>>>
>>> Java Monitors (on contention, sleeping, etc. require)
>>> Jikes Cond Locks (are implemented by)
>>> Harmony Monitors (are implemented by)
>>> APR Condition Variables, etc. (are implemented by)
>>> Pthread Condition Variables, etc.
>>>
>>> I'm proposing:
>>>
>>> Java Monitors (on contention, sleeping, etc. require)
>>> Harmony Monitors (are implemented by)
>>> APR Condition Variables, etc. (are implemented by)
>>> Pthread Condition Variables, etc.
>>
>> How do you envision handling safepoints?  The whole point of cond
>> locks is to gracefully handle asynchronous block and soft handshake
>> requests that occur while holding, waiting on, or attempting to
>> acquire a cond lock.
>
> Copy DRLVM as the reference Harmony VM implementation (that doesn't
> require cond locks). The point is not to reinvent a wheel. In the case
> of DRLVM for sleep, it changes the thread status and calls
> hythread_sleep.
>
>> I don't see similar functionality in Harmony Monitors.  However, we
>> can keep HeavyCondLock's API and implement it on top of Harmony
>> Monitors, and everything would be fine.  But this would be a waste of
>> time in my opinion, because there is no good reason to use Harmony
>> monitors when we could just as easily continue to use pthread_cond  
>> and
>> pthread_mutex.
>
> pthread != Windows != any other Harmony platform where the job of
> porting hythread has been tackled
>
>>> The reason is to avoid requiring Jikes Cond Locks to know anything
>>> about Harmony Monitors. In the Classpath case we have a stack of:
>>>
>>> Java Monitors (on contention, sleeping, etc. require)
>>> Jikes Cond Locks (are implemented by)
>>> Pthread Condition Variables, etc.
>>>
>>> And I agree that's entirely sensible and shouldn't know anything  
>>> about
>>> Harmony's monitors.
>>
>> All of this seems to me to be a very dangerous idea.
>>
>>>
>>>
>>> What I propose is that we start from where we are, implement Harmony
>>> support by refactoring RVMThread into an abstraction again. In the
>>> Harmony implementation we go through to Harmony syscalls, in the
>>> native thread implementation we go through to pthread syscalls. We  
>>> use
>>> an abstraction rather than if-then-else as we're an OO  
>>> metacircular VM
>>> and that's just how you do things.
>>>
>>
>> So you're suggesting that when running on Harmony we let Harmony
>> implement RVMThread?  In C?  And that's OO metacircularity?
>
> No, in green threads we are metacircular. For Harmony we do the simple
> & robust thing.
>
>>> I'd go further and say that once we're in this position we're not  
>>> far
>>> from having an abstraction where green threads would once again fit,
>>> but having processor local is a useful abstraction rather than  
>>> thread
>>> in that circumstance. So I think the way forward is:
>>>
>>> 1) start with Classpath and native threads
>>>
>>> 2) prioritize getting Harmony working by abstracting the current
>>> native threads to support 2 sys call implementations. As this would
>>> mean Jikes Cond Locks were a feature of Classpath/pthread/OpenJDK
>>> native threading it would make sense to move them a package below
>>> scheduler. I think this task should block a major release as it will
>>> get Harmony support back again (an expedient to get a release out in
>>> the mean-time would be to work from before native threading and  
>>> add in
>>> the half-dozen or so bug fixes, I'd propose this be 3.0.2 - this  
>>> work
>>> should be off green threads due to Harmony support)
>>
>> I'm confused.  You are making reference to having 2 sys call
>> implementations.  Are you suggesting two implementations of
>> HeavyCondLock or two implementations of RVMThread?
>>
>> HeavyCondLock is not just an API wrapper around pthread_cond_t and
>> pthread_mutex_t - it actually has added functionality.  So the stacks
>> you'd want, if you really wanted to use Harmony's monitors (which I
>> think is pointless), would be:
>>
>> Classpath:   HeavyCondLock -> abstraction for creating mutex/cond  
>> pair
>> -> pthreads
>>
>> Harmony:   HeavyCondLock -> abstraction for creating mutex/cond  
>> pair -
>>> Harmony
>>
>> And so what you'd be changing is that middle abstraction, rather than
>> the HeavyCondLock itself, which actually contains useful
>> functionality.  In particular, the HeavyCondLock, and all of its
>> Nicely and non-Nicely methods that you like so much, play a crucial
>> role in ensuring that we:
>>
>> 1) Support proper thread accounting while avoiding deadlock and
>> spinning.
>>
>> 2) Support proper thread-to-thread notification while avoiding
>> deadlock and spinning.
>>
>> 3) Support proper soft handshaking (a feature we care about deeply!)
>> while avoiding deadlock  and spinning.
>>
>> 4) Support proper hard handshaking (i.e. GC stop-the-world) while
>> avoiding deadlock and spinning.
>>
>> 5) Support proper thread hijacking (so - a hard handshake between  
>> just
>> two threads) while avoiding deadlock and spinning.  This is kind of
>> what suspend() does.  But we also need it to support biased locks.
>>
>> If we used Harmony monitors directly, this would all become harder,
>> because we would not be able to do things like reset the recursion
>> count of the lock, which would then make it harder to do things like
>> waitNicely(), which is really a central method in how native threads
>> works.
>
> Cond locks are an artifact of pthreads and unnecessary for Harmony  
> threads.
>
>>>
>>>
>>> 3) as a low-priority for people like me, get green thread support  
>>> back
>>> by fitting it into this abstraction and extending where appropriate
>>>
>>> I think we're all nearly on the same page, so possibly +1s for this
>>> proposal would be useful.
>>
>> I don't know what your proposal is yet.  Sometimes you are talking
>> about replacing RVMThread, other times you are talking about
>> abstracting system calls.  Furthermore, I don't see the point of it.
>> The threading code could be largely kept unchanged while supporting
>> Harmony.  You have not convinced me otherwise.
>>
>> -F
>
> The proposal is simple, anything that is a pthread artifact and not
> simply and robustly portable to Harmony is pushed into a package in
> sheduler. We abstract the implementation so that pthreads/Classpath
> work using the sub package. Harmony gets the standard Harmony native
> thread implementation.
>
> This abstraction should mean we can also get the previous green thread
> model back.
>
> Ian
>
>>>
>>>
>>> Ian
>>>
>>>> On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:
>>>>
>>>>> So firstly here is the Harmony thread API:
>>>>>
>>>>> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>>>>>
>>>>> I have problems with pretty much everything that's been said here:
>>>>>
>>>>> Dave> There's a default trivial implementation where those methods
>>>>> are
>>>>> implemented as sysCalls to C code that wraps pthread calls. That's
>>>>> native threading against GNU Classpath/pthreads."
>>>>>
>>>>> in the planning for 3.1 I stated that I don't see Classpath as our
>>>>> long term class library, assuming it in the thread model, to me,
>>>>> appears wrong.
>>>>>
>>>>> Dave> There's a default trivial implementation where those methods
>>>>> are
>>>>> implemented as sysCalls to C code that wraps Harmony threading  
>>>>> APIs.
>>>>> That's native threading against Harmony/pthreads. Perhaps this is
>>>>> very
>>>>> close to (1) and is just some ifdefs in the C code. I haven't  
>>>>> looked
>>>>> at the Harmony library enough to know.
>>>>>
>>>>> this doesn't fit Harmony. In the current threading code we assume
>>>>> cond locks.
>>>>>
>>>>> Dave> There's an implementation, most likely written in Java, that
>>>>> is
>>>>> not at all trivial where something functionally like the m-n
>>>>> threading
>>>>> we had before is provided. Perhaps some old green threading code  
>>>>> can
>>>>> be re-used. A key question here is do you want n-1 or n-m. n-1 is
>>>>> significantly simpler to implement. This is really a question  
>>>>> about
>>>>> use cases and target platforms. In theory we might want to have  
>>>>> both
>>>>> n-1 and n-m.
>>>>>
>>>>> So this appears to be creating a task for people working off green
>>>>> threads to create a new green thread implementation. The expedient
>>>>> thing is to work off where we were (sufficient to run all
>>>>> applications, X10 and the majority of the JSR166 TCK).
>>>>>
>>>>> Dave> The main point is that we avoid re-introducing scheduler
>>>>> APIs in
>>>>> the default case where we are running with native threading.
>>>>> Instead,
>>>>> green threading sits below the rest of the JVM and Jikes RVM as a
>>>>> whole can be oblivious to how those APIs are implemented. I  
>>>>> believe
>>>>> this should yield a simpler and easier to maintain green threading
>>>>> layer than what we had before (which had 10+ years of evolution  
>>>>> and
>>>>> multiple attempts to accommodate hostile native code embedded in
>>>>> it).
>>>>> I'm sure it will yield a simpler and easier to maintain native
>>>>> threading layer.
>>>>>
>>>>> But we inherently get a non-metacircular implementation and have  
>>>>> to
>>>>> reinvent the green thread world.
>>>>>
>>>>> Fil, look at the Harmony thread API and you'll see what my point
>>>>> is. I
>>>>> agree that there are changes elsewhere in the scheduler API and  
>>>>> with
>>>>> reintroducing green threads and a harmony model they should be
>>>>> adhered
>>>>> to. What I disagree with is exposing the cond locks in the
>>>>> scheduler.
>>>>>
>>>>> Daniel> The previous implementation of Harmony was not ready for
>>>>> production use. From the tests we were running it never supported
>>>>> multiple processors, and when running with a single processor  
>>>>> had a
>>>>> significant number of failures. Obviously moving to a system  
>>>>> without
>>>>> green threads makes running with a single processor no longer
>>>>> possible. This was a well understood problem, and work on  
>>>>> Harmony to
>>>>> fix this issue was deliberately delayed until after native threads
>>>>> was
>>>>> implemented.
>>>>>
>>>>> I agree, we have been running against and leading Classpath
>>>>> development for a while. Harmony support was at a good level. We  
>>>>> ran
>>>>> SPEC JVM 2008, which I see as essential for a research virtual
>>>>> machine. There is much research that can be done on a single core.
>>>>>
>>>>> Daniel> This relates to a significant issue that I don't think was
>>>>> adequately addressed at the time. The announcement of our initial
>>>>> compatibility/support for the Harmony libraries was premature,  
>>>>> and I
>>>>> know made a lot of the core team uncomfortable. Such announcements
>>>>> have the potential to significantly affect the credibility of  
>>>>> Jikes
>>>>> RVM, and should not be made without some consultation within the
>>>>> core
>>>>> team. What we claim to support officially in releases of JikesRVM
>>>>> has
>>>>> a direct impact on everyone working on the project.
>>>>>
>>>>> So this is highly irritating. If you have concerns:
>>>>> 1) e-mail me, and/or
>>>>> 2) open JIRAs, there's a whole place for these in the Harmony
>>>>> support, and/or
>>>>> 3) mail the core mailing list
>>>>> Saying that the Harmony support was announced prematurely should
>>>>> really have been stated at the time, when it could have been
>>>>> rectified. Saying it now is just ridiculous.
>>>>>
>>>>> In terms of credibility, it is far far more important for our
>>>>> credibility that we run on a class library that will give enough
>>>>> support to at a least run major benchmark suites. In this case  
>>>>> SPEC
>>>>> JVM 2008.
>>>>>
>>>>> I would dearly love help with the Harmony implementation, so far
>>>>> Alexey has been the only contributor.
>>>>>
>>>>> Daniel> I think it is clear that we all want to work toward
>>>>> compatibility with Harmony (and possibly OpenJDK) but we need to
>>>>> do so
>>>>> carefully.
>>>>>
>>>>> As we were already compatible with Harmony, I don't know why we're
>>>>> taking a view that we're not.
>>>>>
>>>>> Daniel> I have left aside my thoughts on the low level technical
>>>>> detail here; it is important to take a step back from that and  
>>>>> solve
>>>>> the strategic issues first.
>>>>>
>>>>> I agree. We either have a minimalist interface or something  
>>>>> where we
>>>>> have multiple configurations. I need multiple configurations and
>>>>> if I
>>>>> can't do that against the trunk I will have to do my own thing.
>>>>>
>>>>> In terms of premature decisions, I think merging Fil's code with  
>>>>> the
>>>>> trunk was premature. We should have worked these issues out first.
>>>>> We
>>>>> should have closed the JIRAs on RVM-91 first. We should have asked
>>>>> the
>>>>> core list before the merge, first.
>>>>>
>>>>> Ian
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>>> Francisco, CA
>>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>>> Enterprise
>>>>> -Strategies to boost innovation and cut costs with open source
>>>>> participation
>>>>> -Receive a $600 discount off the registration fee with the source
>>>>> code: SFAD
>>>>> http://p.sf.net/sfu/XcvMzF8H
>>>>> _______________________________________________
>>>>> Jikesrvm-core mailing list
>>>>> Jikesrvm-core@...
>>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>> Francisco, CA
>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>> Enterprise
>>>> -Strategies to boost innovation and cut costs with open source
>>>> participation
>>>> -Receive a $600 discount off the registration fee with the source
>>>> code: SFAD
>>>> http://p.sf.net/sfu/XcvMzF8H
>>>> _______________________________________________
>>>> Jikesrvm-core mailing list
>>>> Jikesrvm-core@...
>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>> Francisco, CA
>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>> Enterprise
>>> -Strategies to boost innovation and cut costs with open source
>>> participation
>>> -Receive a $600 discount off the registration fee with the source
>>> code: SFAD
>>> http://p.sf.net/sfu/XcvMzF8H
>>> _______________________________________________
>>> Jikesrvm-core mailing list
>>> Jikesrvm-core@...
>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San  
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the  
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source  
>> participation
>> -Receive a $600 discount off the registration fee with the source  
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San  
> Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the  
> Enterprise
> -Strategies to boost innovation and cut costs with open source  
> participation
> -Receive a $600 discount off the registration fee with the source  
> code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Ian Rogers (nabble)]

2009/2/20 Filip Pizlo <pizlo@...>:
> Four things:
>
> 1) We can get Windows support with a much simpler abstraction layer
> than what you propose.  Daniel, Dave, and I have already pointed out
> various ways of doing this.  I'm not sure why you're ignoring these
> proposals.

When we work on a non-standard abstraction layer we create a
maintenance problem. Having fixed most of the issues with Classpath
myself, I'm not keen to repeat this.

> 2) The HeavyCondLock/RVMThread code is not an artifact of pthreads,
> it's an artifact of the fact that we have a high-performance VM.  And
> besides, the Harmony VM code that I've seen actually overrides quite a
> bit of the Harmony classlib hythread code with its own stuff (monitors
> in particular).  So I don't see the problem with us doing the same (in
> particular, by just keeping the code we already have and ignoring the
> code that Harmony uses).

There is an issue for Harmony in keeping VM code in sync with the main
Harmony code. Needless to say the fast code will go into Harmony when
there are the cycles.

> 3) Your description of how DRLVM does sleep indicates that it's either
> wrong, requires STW GC to do unnecessary spinning, requires the use of
> signaling to implement STW GC, and/or completely eliminates the
> possibility of implementing on-the-fly concurrent GC.  Any one of
> those would be unacceptable from a performance and functionality
> standpoint.

If you are in native code you have called out and are at a GC safe
point. Why does this prohibit concurrent GC? Any native code has the
prospect of not going back into Java for a long time.

> 4) Even with an abstraction on RVMThread you won't get green threads
> back, since green threads cross cut other parts of the system.  This
> was the bulk of the work in implementing native threading.

Green threads are a low priority. If I don't get them back I'm forced
to work on a fork, that's not good for the RVM.

> Your proposal requires doing more work than we need to do in order to
> support Harmony.  You're suggesting we refactor the threading model,
> again, only this time, you want to patch it into a foreign threading
> model.  You've given no argument for why this would be necessary,
> useful, or even correct.  You think matches the semantics we want, but
> you haven't given me any argument that it does.  On the contrary,
> you've shown examples that it doesn't, and have made claims that, if
> true, would imply that running with Harmony threads as per your
> proposal would cause livelocks and deadlocks in GC unless we made a
> lot of other big changes.

This is incorrect. I pointed out the thread model I wanted many times.
I completed Harmony support ahead of the work on your SoC project, you
chose to ignore it in fact you and Tony took the impossible situation
of completely ignoring me.

> The proposal that me, Dave, and Daniel have all made - abstracting
> thread creation and HeavyCondLock instead of RVMThread - allows us to
> get Harmony in there quicker with fewer code changes.

It won't work on Windows. We have to port the RVMThread and
HeavyCondLock to Windows. We have to also port all the rest of the
RVM's thread and signal handling code, we create a maintenance
problem.

> I don't think that it makes sense to argue for rewriting more code if
> the same effect can be achieved by rewriting less code.
>
> -F

I'm arguing for a flexible thread model, have done since the
beginning, I implemented this in the RVM code base, had a project
student write a dissertation about it.... You have chosen to ignore
all of this and arrogantly stated that your one model fits everyone
best. I don't believe it does and claiming that Harmony VMs are
fundamentally broken I think just goes further to show the weakness of
your argument.

Ian

> On Feb 20, 2009, at 12:04 PM, Ian Rogers wrote:
>
>> 2009/2/20 Filip Pizlo <pizlo@...>:
>>> Ian,
>>>
>>> We could leave HeavyCondLock unchanged while supporting Harmony.  It
>>> can continue to use pthread_cond and pthread_mutex, and everything
>>> would just work.
>>>
>>> What we seem to need, from a threading standpoint, for Harmony is:
>>>
>>> 1) That its threading API be initialized so that the classlib can use
>>> it without dying, and
>>>
>>> 2) That each thread is at the end of the day a Harmony thread, so
>>> that
>>> Harmony TLS and such is guaranteed to work.
>>>
>>> This is not difficult to do.  The rest of this discussion strikes me
>>> as somewhat superfluous.  Why do you want to provide these
>>> abstractions?  What's wrong with having the same threading
>>> infrastructure with all class libraries?
>>
>> We don't have pthreads on Windows. I want the Harmony code to be
>> portable, simple and robust.
>>
>>>>> I've looked at the Harmony threading APIs.  The hythread API has
>>>>> "monitors".  We can just use these as cond locks.
>>>>
>>>> Yep, I've been saying this but it is inherently ugly and would
>>>> require
>>>> changing the interface to fit (why change the interface when it's
>>>> not
>>>> necessary for Classpath/OpenJDK..).
>>>
>>> It's not ugly.
>>>
>>>>
>>>>
>>>>> Or we could just use pthread mutexes and cond's just as we're doing
>>>>> now.  This doesn't really hurt portability.  Even Win32 has cond
>>>>> locks
>>>>> now...  I don't see anything wrong with having some #ifdefs for
>>>>> code
>>>>> like this.  It would be ~40 lines of isolated nastiness in sys.C
>>>>> (or
>>>>> its equivalent).
>>>>
>>>> Except you need to change all our cond lock code to also acquire and
>>>> have behind it a monitor. This ends up being both ugly and
>>>> inefficient.
>>>
>>> I don't get this.  How is it ugly?  And who cares if it's
>>> inefficient?  You realize that this code is only called on slow
>>> paths,
>>> right?  And that most of it is already baseline compiled, not
>>> inlined,
>>> and makes in some cases multiple sysCalls?  None of this is fast,
>>> because we don't need it to be!
>>>
>>>>
>>>>
>>>>> Or are you suggesting that we use hythread's monitors as our Java
>>>>> monitors?  And have pointers from the Java head into the C heap?
>>>>> And
>>>>> require fat lock acquisition to do a C downcall?  That could get
>>>>> quite
>>>>> messy, given how our thin locks rely on very specific invariants
>>>>> from
>>>>> our fat locks, which essentially forces us to implement fat locks
>>>>> ourselves.  I guess my whole problem here is that I don't see what
>>>>> design you're arguing for.
>>>>>
>>>>> -F
>>>>
>>>> I am proposing this as the interface between cond lock and the
>>>> monitor
>>>> doesn't fit. If we use cond locks we end up with a stack of:
>>>>
>>>> Java Monitors (on contention, sleeping, etc. require)
>>>> Jikes Cond Locks (are implemented by)
>>>> Harmony Monitors (are implemented by)
>>>> APR Condition Variables, etc. (are implemented by)
>>>> Pthread Condition Variables, etc.
>>>>
>>>> I'm proposing:
>>>>
>>>> Java Monitors (on contention, sleeping, etc. require)
>>>> Harmony Monitors (are implemented by)
>>>> APR Condition Variables, etc. (are implemented by)
>>>> Pthread Condition Variables, etc.
>>>
>>> How do you envision handling safepoints?  The whole point of cond
>>> locks is to gracefully handle asynchronous block and soft handshake
>>> requests that occur while holding, waiting on, or attempting to
>>> acquire a cond lock.
>>
>> Copy DRLVM as the reference Harmony VM implementation (that doesn't
>> require cond locks). The point is not to reinvent a wheel. In the case
>> of DRLVM for sleep, it changes the thread status and calls
>> hythread_sleep.
>>
>>> I don't see similar functionality in Harmony Monitors.  However, we
>>> can keep HeavyCondLock's API and implement it on top of Harmony
>>> Monitors, and everything would be fine.  But this would be a waste of
>>> time in my opinion, because there is no good reason to use Harmony
>>> monitors when we could just as easily continue to use pthread_cond
>>> and
>>> pthread_mutex.
>>
>> pthread != Windows != any other Harmony platform where the job of
>> porting hythread has been tackled
>>
>>>> The reason is to avoid requiring Jikes Cond Locks to know anything
>>>> about Harmony Monitors. In the Classpath case we have a stack of:
>>>>
>>>> Java Monitors (on contention, sleeping, etc. require)
>>>> Jikes Cond Locks (are implemented by)
>>>> Pthread Condition Variables, etc.
>>>>
>>>> And I agree that's entirely sensible and shouldn't know anything
>>>> about
>>>> Harmony's monitors.
>>>
>>> All of this seems to me to be a very dangerous idea.
>>>
>>>>
>>>>
>>>> What I propose is that we start from where we are, implement Harmony
>>>> support by refactoring RVMThread into an abstraction again. In the
>>>> Harmony implementation we go through to Harmony syscalls, in the
>>>> native thread implementation we go through to pthread syscalls. We
>>>> use
>>>> an abstraction rather than if-then-else as we're an OO
>>>> metacircular VM
>>>> and that's just how you do things.
>>>>
>>>
>>> So you're suggesting that when running on Harmony we let Harmony
>>> implement RVMThread?  In C?  And that's OO metacircularity?
>>
>> No, in green threads we are metacircular. For Harmony we do the simple
>> & robust thing.
>>
>>>> I'd go further and say that once we're in this position we're not
>>>> far
>>>> from having an abstraction where green threads would once again fit,
>>>> but having processor local is a useful abstraction rather than
>>>> thread
>>>> in that circumstance. So I think the way forward is:
>>>>
>>>> 1) start with Classpath and native threads
>>>>
>>>> 2) prioritize getting Harmony working by abstracting the current
>>>> native threads to support 2 sys call implementations. As this would
>>>> mean Jikes Cond Locks were a feature of Classpath/pthread/OpenJDK
>>>> native threading it would make sense to move them a package below
>>>> scheduler. I think this task should block a major release as it will
>>>> get Harmony support back again (an expedient to get a release out in
>>>> the mean-time would be to work from before native threading and
>>>> add in
>>>> the half-dozen or so bug fixes, I'd propose this be 3.0.2 - this
>>>> work
>>>> should be off green threads due to Harmony support)
>>>
>>> I'm confused.  You are making reference to having 2 sys call
>>> implementations.  Are you suggesting two implementations of
>>> HeavyCondLock or two implementations of RVMThread?
>>>
>>> HeavyCondLock is not just an API wrapper around pthread_cond_t and
>>> pthread_mutex_t - it actually has added functionality.  So the stacks
>>> you'd want, if you really wanted to use Harmony's monitors (which I
>>> think is pointless), would be:
>>>
>>> Classpath:   HeavyCondLock -> abstraction for creating mutex/cond
>>> pair
>>> -> pthreads
>>>
>>> Harmony:   HeavyCondLock -> abstraction for creating mutex/cond
>>> pair -
>>>> Harmony
>>>
>>> And so what you'd be changing is that middle abstraction, rather than
>>> the HeavyCondLock itself, which actually contains useful
>>> functionality.  In particular, the HeavyCondLock, and all of its
>>> Nicely and non-Nicely methods that you like so much, play a crucial
>>> role in ensuring that we:
>>>
>>> 1) Support proper thread accounting while avoiding deadlock and
>>> spinning.
>>>
>>> 2) Support proper thread-to-thread notification while avoiding
>>> deadlock and spinning.
>>>
>>> 3) Support proper soft handshaking (a feature we care about deeply!)
>>> while avoiding deadlock  and spinning.
>>>
>>> 4) Support proper hard handshaking (i.e. GC stop-the-world) while
>>> avoiding deadlock and spinning.
>>>
>>> 5) Support proper thread hijacking (so - a hard handshake between
>>> just
>>> two threads) while avoiding deadlock and spinning.  This is kind of
>>> what suspend() does.  But we also need it to support biased locks.
>>>
>>> If we used Harmony monitors directly, this would all become harder,
>>> because we would not be able to do things like reset the recursion
>>> count of the lock, which would then make it harder to do things like
>>> waitNicely(), which is really a central method in how native threads
>>> works.
>>
>> Cond locks are an artifact of pthreads and unnecessary for Harmony
>> threads.
>>
>>>>
>>>>
>>>> 3) as a low-priority for people like me, get green thread support
>>>> back
>>>> by fitting it into this abstraction and extending where appropriate
>>>>
>>>> I think we're all nearly on the same page, so possibly +1s for this
>>>> proposal would be useful.
>>>
>>> I don't know what your proposal is yet.  Sometimes you are talking
>>> about replacing RVMThread, other times you are talking about
>>> abstracting system calls.  Furthermore, I don't see the point of it.
>>> The threading code could be largely kept unchanged while supporting
>>> Harmony.  You have not convinced me otherwise.
>>>
>>> -F
>>
>> The proposal is simple, anything that is a pthread artifact and not
>> simply and robustly portable to Harmony is pushed into a package in
>> sheduler. We abstract the implementation so that pthreads/Classpath
>> work using the sub package. Harmony gets the standard Harmony native
>> thread implementation.
>>
>> This abstraction should mean we can also get the previous green thread
>> model back.
>>
>> Ian
>>
>>>>
>>>>
>>>> Ian
>>>>
>>>>> On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:
>>>>>
>>>>>> So firstly here is the Harmony thread API:
>>>>>>
>>>>>> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>>>>>>
>>>>>> I have problems with pretty much everything that's been said here:
>>>>>>
>>>>>> Dave> There's a default trivial implementation where those methods
>>>>>> are
>>>>>> implemented as sysCalls to C code that wraps pthread calls. That's
>>>>>> native threading against GNU Classpath/pthreads."
>>>>>>
>>>>>> in the planning for 3.1 I stated that I don't see Classpath as our
>>>>>> long term class library, assuming it in the thread model, to me,
>>>>>> appears wrong.
>>>>>>
>>>>>> Dave> There's a default trivial implementation where those methods
>>>>>> are
>>>>>> implemented as sysCalls to C code that wraps Harmony threading
>>>>>> APIs.
>>>>>> That's native threading against Harmony/pthreads. Perhaps this is
>>>>>> very
>>>>>> close to (1) and is just some ifdefs in the C code. I haven't
>>>>>> looked
>>>>>> at the Harmony library enough to know.
>>>>>>
>>>>>> this doesn't fit Harmony. In the current threading code we assume
>>>>>> cond locks.
>>>>>>
>>>>>> Dave> There's an implementation, most likely written in Java, that
>>>>>> is
>>>>>> not at all trivial where something functionally like the m-n
>>>>>> threading
>>>>>> we had before is provided. Perhaps some old green threading code
>>>>>> can
>>>>>> be re-used. A key question here is do you want n-1 or n-m. n-1 is
>>>>>> significantly simpler to implement. This is really a question
>>>>>> about
>>>>>> use cases and target platforms. In theory we might want to have
>>>>>> both
>>>>>> n-1 and n-m.
>>>>>>
>>>>>> So this appears to be creating a task for people working off green
>>>>>> threads to create a new green thread implementation. The expedient
>>>>>> thing is to work off where we were (sufficient to run all
>>>>>> applications, X10 and the majority of the JSR166 TCK).
>>>>>>
>>>>>> Dave> The main point is that we avoid re-introducing scheduler
>>>>>> APIs in
>>>>>> the default case where we are running with native threading.
>>>>>> Instead,
>>>>>> green threading sits below the rest of the JVM and Jikes RVM as a
>>>>>> whole can be oblivious to how those APIs are implemented. I
>>>>>> believe
>>>>>> this should yield a simpler and easier to maintain green threading
>>>>>> layer than what we had before (which had 10+ years of evolution
>>>>>> and
>>>>>> multiple attempts to accommodate hostile native code embedded in
>>>>>> it).
>>>>>> I'm sure it will yield a simpler and easier to maintain native
>>>>>> threading layer.
>>>>>>
>>>>>> But we inherently get a non-metacircular implementation and have
>>>>>> to
>>>>>> reinvent the green thread world.
>>>>>>
>>>>>> Fil, look at the Harmony thread API and you'll see what my point
>>>>>> is. I
>>>>>> agree that there are changes elsewhere in the scheduler API and
>>>>>> with
>>>>>> reintroducing green threads and a harmony model they should be
>>>>>> adhered
>>>>>> to. What I disagree with is exposing the cond locks in the
>>>>>> scheduler.
>>>>>>
>>>>>> Daniel> The previous implementation of Harmony was not ready for
>>>>>> production use. From the tests we were running it never supported
>>>>>> multiple processors, and when running with a single processor
>>>>>> had a
>>>>>> significant number of failures. Obviously moving to a system
>>>>>> without
>>>>>> green threads makes running with a single processor no longer
>>>>>> possible. This was a well understood problem, and work on
>>>>>> Harmony to
>>>>>> fix this issue was deliberately delayed until after native threads
>>>>>> was
>>>>>> implemented.
>>>>>>
>>>>>> I agree, we have been running against and leading Classpath
>>>>>> development for a while. Harmony support was at a good level. We
>>>>>> ran
>>>>>> SPEC JVM 2008, which I see as essential for a research virtual
>>>>>> machine. There is much research that can be done on a single core.
>>>>>>
>>>>>> Daniel> This relates to a significant issue that I don't think was
>>>>>> adequately addressed at the time. The announcement of our initial
>>>>>> compatibility/support for the Harmony libraries was premature,
>>>>>> and I
>>>>>> know made a lot of the core team uncomfortable. Such announcements
>>>>>> have the potential to significantly affect the credibility of
>>>>>> Jikes
>>>>>> RVM, and should not be made without some consultation within the
>>>>>> core
>>>>>> team. What we claim to support officially in releases of JikesRVM
>>>>>> has
>>>>>> a direct impact on everyone working on the project.
>>>>>>
>>>>>> So this is highly irritating. If you have concerns:
>>>>>> 1) e-mail me, and/or
>>>>>> 2) open JIRAs, there's a whole place for these in the Harmony
>>>>>> support, and/or
>>>>>> 3) mail the core mailing list
>>>>>> Saying that the Harmony support was announced prematurely should
>>>>>> really have been stated at the time, when it could have been
>>>>>> rectified. Saying it now is just ridiculous.
>>>>>>
>>>>>> In terms of credibility, it is far far more important for our
>>>>>> credibility that we run on a class library that will give enough
>>>>>> support to at a least run major benchmark suites. In this case
>>>>>> SPEC
>>>>>> JVM 2008.
>>>>>>
>>>>>> I would dearly love help with the Harmony implementation, so far
>>>>>> Alexey has been the only contributor.
>>>>>>
>>>>>> Daniel> I think it is clear that we all want to work toward
>>>>>> compatibility with Harmony (and possibly OpenJDK) but we need to
>>>>>> do so
>>>>>> carefully.
>>>>>>
>>>>>> As we were already compatible with Harmony, I don't know why we're
>>>>>> taking a view that we're not.
>>>>>>
>>>>>> Daniel> I have left aside my thoughts on the low level technical
>>>>>> detail here; it is important to take a step back from that and
>>>>>> solve
>>>>>> the strategic issues first.
>>>>>>
>>>>>> I agree. We either have a minimalist interface or something
>>>>>> where we
>>>>>> have multiple configurations. I need multiple configurations and
>>>>>> if I
>>>>>> can't do that against the trunk I will have to do my own thing.
>>>>>>
>>>>>> In terms of premature decisions, I think merging Fil's code with
>>>>>> the
>>>>>> trunk was premature. We should have worked these issues out first.
>>>>>> We
>>>>>> should have closed the JIRAs on RVM-91 first. We should have asked
>>>>>> the
>>>>>> core list before the merge, first.
>>>>>>
>>>>>> Ian
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>>>> Francisco, CA
>>>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>>>> Enterprise
>>>>>> -Strategies to boost innovation and cut costs with open source
>>>>>> participation
>>>>>> -Receive a $600 discount off the registration fee with the source
>>>>>> code: SFAD
>>>>>> http://p.sf.net/sfu/XcvMzF8H
>>>>>> _______________________________________________
>>>>>> Jikesrvm-core mailing list
>>>>>> Jikesrvm-core@...
>>>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>>> Francisco, CA
>>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>>> Enterprise
>>>>> -Strategies to boost innovation and cut costs with open source
>>>>> participation
>>>>> -Receive a $600 discount off the registration fee with the source
>>>>> code: SFAD
>>>>> http://p.sf.net/sfu/XcvMzF8H
>>>>> _______________________________________________
>>>>> Jikesrvm-core mailing list
>>>>> Jikesrvm-core@...
>>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>> Francisco, CA
>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>> Enterprise
>>>> -Strategies to boost innovation and cut costs with open source
>>>> participation
>>>> -Receive a $600 discount off the registration fee with the source
>>>> code: SFAD
>>>> http://p.sf.net/sfu/XcvMzF8H
>>>> _______________________________________________
>>>> Jikesrvm-core mailing list
>>>> Jikesrvm-core@...
>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>> Francisco, CA
>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>> Enterprise
>>> -Strategies to boost innovation and cut costs with open source
>>> participation
>>> -Receive a $600 discount off the registration fee with the source
>>> code: SFAD
>>> http://p.sf.net/sfu/XcvMzF8H
>>> _______________________________________________
>>> Jikesrvm-core mailing list
>>> Jikesrvm-core@...
>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source
>> participation
>> -Receive a $600 discount off the registration fee with the source
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

[Filip Pizlo-2]

>> Four things:
>>
>> 1) We can get Windows support with a much simpler abstraction layer
>> than what you propose.  Daniel, Dave, and I have already pointed out
>> various ways of doing this.  I'm not sure why you're ignoring these
>> proposals.
>
> When we work on a non-standard abstraction layer we create a
> maintenance problem. Having fixed most of the issues with Classpath
> myself, I'm not keen to repeat this.

And you are also proposing to create a non-standard abstraction  
layer.  Only mine is smaller.

>
>
>> 2) The HeavyCondLock/RVMThread code is not an artifact of pthreads,
>> it's an artifact of the fact that we have a high-performance VM.  And
>> besides, the Harmony VM code that I've seen actually overrides  
>> quite a
>> bit of the Harmony classlib hythread code with its own stuff  
>> (monitors
>> in particular).  So I don't see the problem with us doing the same  
>> (in
>> particular, by just keeping the code we already have and ignoring the
>> code that Harmony uses).
>
> There is an issue for Harmony in keeping VM code in sync with the main
> Harmony code. Needless to say the fast code will go into Harmony when
> there are the cycles.

No, the code is different because the VM code understands thinlocks,  
while the classlib code doesn't.  I'd be really worried about trying  
to tie our thin locking strategy to the monitor strategy used in  
Harmony.  The Harmony VM doesn't even support eager lock deflation  
last time I checked!

>
>
>> 3) Your description of how DRLVM does sleep indicates that it's  
>> either
>> wrong, requires STW GC to do unnecessary spinning, requires the use  
>> of
>> signaling to implement STW GC, and/or completely eliminates the
>> possibility of implementing on-the-fly concurrent GC.  Any one of
>> those would be unacceptable from a performance and functionality
>> standpoint.
>
> If you are in native code you have called out and are at a GC safe
> point. Why does this prohibit concurrent GC? Any native code has the
> prospect of not going back into Java for a long time.

And how do we implement downcalls to native code?  That's what  
HeavyCondLock is for.

HeavyCondLock is not just a replacement for Java monitors.  It's a low-
level synchronization mechanism used for the implementation of safe  
downcalls to native code.  That's why we can't just take any old lock  
implementation and stick it in there - you need one that gives you the  
low-level hooks needed for implementing native transitions.

The problem with on-the-fly GC is that you need to be able to  
synchronously observe a thread's state, and in the fast path, do so  
without having to block the thread.  This requires that the transition  
from a "running in Java" state to a "running in native" state happens  
with a CAS that redirects to a slow path that grabs a lock.  That's  
how we do it, and that's how most other VMs do it.  It doesn't just  
help concurrent GC, but it helps with:

- Biased locking or any kind of lock reservation

- Stop the world GC becomes much simpler

- Issuing isyncs upon classloading (which we need to be able to do)

- Other stuff, too.  I'm currently taking advantage of this to  
streamline our Java lock allocation code.  Daniel's using it to  
greatly simplify MMTk.

>
>
>> 4) Even with an abstraction on RVMThread you won't get green threads
>> back, since green threads cross cut other parts of the system.  This
>> was the bulk of the work in implementing native threading.
>
> Green threads are a low priority. If I don't get them back I'm forced
> to work on a fork, that's not good for the RVM.

It's not good for the RVM if native threads are broken just after  
being fixed!

>
>
>> Your proposal requires doing more work than we need to do in order to
>> support Harmony.  You're suggesting we refactor the threading model,
>> again, only this time, you want to patch it into a foreign threading
>> model.  You've given no argument for why this would be necessary,
>> useful, or even correct.  You think matches the semantics we want,  
>> but
>> you haven't given me any argument that it does.  On the contrary,
>> you've shown examples that it doesn't, and have made claims that, if
>> true, would imply that running with Harmony threads as per your
>> proposal would cause livelocks and deadlocks in GC unless we made a
>> lot of other big changes.
>
> This is incorrect. I pointed out the thread model I wanted many times.
> I completed Harmony support ahead of the work on your SoC project, you
> chose to ignore it in fact you and Tony took the impossible situation
> of completely ignoring me.

I didn't ignore anything.  Harmony wasn't ready for production use.  
And none of what I've done has prevented Harmony from working,  
provided it's integrated in a sensible fashion that doesn't completely  
break the RVM's ability to introspect about the state of its own  
threads.

>
>
>> The proposal that me, Dave, and Daniel have all made - abstracting
>> thread creation and HeavyCondLock instead of RVMThread - allows us to
>> get Harmony in there quicker with fewer code changes.
>
> It won't work on Windows. We have to port the RVMThread and
> HeavyCondLock to Windows. We have to also port all the rest of the
> RVM's thread and signal handling code, we create a maintenance
> problem.

It will work on Windows.

Windows supports the same threading abstractions that pthreads do.  
They even have condition variables these days.  And in the absence of  
condition variables, you can still use AutoResetEvents and/or  
Semaphores to get the same functionality.

As for signals, we only currently use them in a very restricted  
manner.  All we need is a way for a thread to raise a trap, and handle  
it.  The way Windows supports this is not so indifferent from the way  
Linux does it that we should be worried.  I don't remember the exact  
calls - it's been a while since I've done this - but I've hacked such  
code on VMs on Win32 and it's really not that hard.

>
>
>> I don't think that it makes sense to argue for rewriting more code if
>> the same effect can be achieved by rewriting less code.
>>
>> -F
>
> I'm arguing for a flexible thread model, have done since the
> beginning, I implemented this in the RVM code base, had a project
> student write a dissertation about it.... You have chosen to ignore
> all of this and arrogantly stated that your one model fits everyone
> best. I don't believe it does and claiming that Harmony VMs are
> fundamentally broken I think just goes further to show the weakness of
> your argument.

Calm down.

-F

>
>
> Ian
>
>> On Feb 20, 2009, at 12:04 PM, Ian Rogers wrote:
>>
>>> 2009/2/20 Filip Pizlo <pizlo@...>:
>>>> Ian,
>>>>
>>>> We could leave HeavyCondLock unchanged while supporting Harmony.  
>>>> It
>>>> can continue to use pthread_cond and pthread_mutex, and everything
>>>> would just work.
>>>>
>>>> What we seem to need, from a threading standpoint, for Harmony is:
>>>>
>>>> 1) That its threading API be initialized so that the classlib can  
>>>> use
>>>> it without dying, and
>>>>
>>>> 2) That each thread is at the end of the day a Harmony thread, so
>>>> that
>>>> Harmony TLS and such is guaranteed to work.
>>>>
>>>> This is not difficult to do.  The rest of this discussion strikes  
>>>> me
>>>> as somewhat superfluous.  Why do you want to provide these
>>>> abstractions?  What's wrong with having the same threading
>>>> infrastructure with all class libraries?
>>>
>>> We don't have pthreads on Windows. I want the Harmony code to be
>>> portable, simple and robust.
>>>
>>>>>> I've looked at the Harmony threading APIs.  The hythread API has
>>>>>> "monitors".  We can just use these as cond locks.
>>>>>
>>>>> Yep, I've been saying this but it is inherently ugly and would
>>>>> require
>>>>> changing the interface to fit (why change the interface when it's
>>>>> not
>>>>> necessary for Classpath/OpenJDK..).
>>>>
>>>> It's not ugly.
>>>>
>>>>>
>>>>>
>>>>>> Or we could just use pthread mutexes and cond's just as we're  
>>>>>> doing
>>>>>> now.  This doesn't really hurt portability.  Even Win32 has cond
>>>>>> locks
>>>>>> now...  I don't see anything wrong with having some #ifdefs for
>>>>>> code
>>>>>> like this.  It would be ~40 lines of isolated nastiness in sys.C
>>>>>> (or
>>>>>> its equivalent).
>>>>>
>>>>> Except you need to change all our cond lock code to also acquire  
>>>>> and
>>>>> have behind it a monitor. This ends up being both ugly and
>>>>> inefficient.
>>>>
>>>> I don't get this.  How is it ugly?  And who cares if it's
>>>> inefficient?  You realize that this code is only called on slow
>>>> paths,
>>>> right?  And that most of it is already baseline compiled, not
>>>> inlined,
>>>> and makes in some cases multiple sysCalls?  None of this is fast,
>>>> because we don't need it to be!
>>>>
>>>>>
>>>>>
>>>>>> Or are you suggesting that we use hythread's monitors as our Java
>>>>>> monitors?  And have pointers from the Java head into the C heap?
>>>>>> And
>>>>>> require fat lock acquisition to do a C downcall?  That could get
>>>>>> quite
>>>>>> messy, given how our thin locks rely on very specific invariants
>>>>>> from
>>>>>> our fat locks, which essentially forces us to implement fat locks
>>>>>> ourselves.  I guess my whole problem here is that I don't see  
>>>>>> what
>>>>>> design you're arguing for.
>>>>>>
>>>>>> -F
>>>>>
>>>>> I am proposing this as the interface between cond lock and the
>>>>> monitor
>>>>> doesn't fit. If we use cond locks we end up with a stack of:
>>>>>
>>>>> Java Monitors (on contention, sleeping, etc. require)
>>>>> Jikes Cond Locks (are implemented by)
>>>>> Harmony Monitors (are implemented by)
>>>>> APR Condition Variables, etc. (are implemented by)
>>>>> Pthread Condition Variables, etc.
>>>>>
>>>>> I'm proposing:
>>>>>
>>>>> Java Monitors (on contention, sleeping, etc. require)
>>>>> Harmony Monitors (are implemented by)
>>>>> APR Condition Variables, etc. (are implemented by)
>>>>> Pthread Condition Variables, etc.
>>>>
>>>> How do you envision handling safepoints?  The whole point of cond
>>>> locks is to gracefully handle asynchronous block and soft handshake
>>>> requests that occur while holding, waiting on, or attempting to
>>>> acquire a cond lock.
>>>
>>> Copy DRLVM as the reference Harmony VM implementation (that doesn't
>>> require cond locks). The point is not to reinvent a wheel. In the  
>>> case
>>> of DRLVM for sleep, it changes the thread status and calls
>>> hythread_sleep.
>>>
>>>> I don't see similar functionality in Harmony Monitors.  However, we
>>>> can keep HeavyCondLock's API and implement it on top of Harmony
>>>> Monitors, and everything would be fine.  But this would be a  
>>>> waste of
>>>> time in my opinion, because there is no good reason to use Harmony
>>>> monitors when we could just as easily continue to use pthread_cond
>>>> and
>>>> pthread_mutex.
>>>
>>> pthread != Windows != any other Harmony platform where the job of
>>> porting hythread has been tackled
>>>
>>>>> The reason is to avoid requiring Jikes Cond Locks to know anything
>>>>> about Harmony Monitors. In the Classpath case we have a stack of:
>>>>>
>>>>> Java Monitors (on contention, sleeping, etc. require)
>>>>> Jikes Cond Locks (are implemented by)
>>>>> Pthread Condition Variables, etc.
>>>>>
>>>>> And I agree that's entirely sensible and shouldn't know anything
>>>>> about
>>>>> Harmony's monitors.
>>>>
>>>> All of this seems to me to be a very dangerous idea.
>>>>
>>>>>
>>>>>
>>>>> What I propose is that we start from where we are, implement  
>>>>> Harmony
>>>>> support by refactoring RVMThread into an abstraction again. In the
>>>>> Harmony implementation we go through to Harmony syscalls, in the
>>>>> native thread implementation we go through to pthread syscalls. We
>>>>> use
>>>>> an abstraction rather than if-then-else as we're an OO
>>>>> metacircular VM
>>>>> and that's just how you do things.
>>>>>
>>>>
>>>> So you're suggesting that when running on Harmony we let Harmony
>>>> implement RVMThread?  In C?  And that's OO metacircularity?
>>>
>>> No, in green threads we are metacircular. For Harmony we do the  
>>> simple
>>> & robust thing.
>>>
>>>>> I'd go further and say that once we're in this position we're not
>>>>> far
>>>>> from having an abstraction where green threads would once again  
>>>>> fit,
>>>>> but having processor local is a useful abstraction rather than
>>>>> thread
>>>>> in that circumstance. So I think the way forward is:
>>>>>
>>>>> 1) start with Classpath and native threads
>>>>>
>>>>> 2) prioritize getting Harmony working by abstracting the current
>>>>> native threads to support 2 sys call implementations. As this  
>>>>> would
>>>>> mean Jikes Cond Locks were a feature of Classpath/pthread/OpenJDK
>>>>> native threading it would make sense to move them a package below
>>>>> scheduler. I think this task should block a major release as it  
>>>>> will
>>>>> get Harmony support back again (an expedient to get a release  
>>>>> out in
>>>>> the mean-time would be to work from before native threading and
>>>>> add in
>>>>> the half-dozen or so bug fixes, I'd propose this be 3.0.2 - this
>>>>> work
>>>>> should be off green threads due to Harmony support)
>>>>
>>>> I'm confused.  You are making reference to having 2 sys call
>>>> implementations.  Are you suggesting two implementations of
>>>> HeavyCondLock or two implementations of RVMThread?
>>>>
>>>> HeavyCondLock is not just an API wrapper around pthread_cond_t and
>>>> pthread_mutex_t - it actually has added functionality.  So the  
>>>> stacks
>>>> you'd want, if you really wanted to use Harmony's monitors (which I
>>>> think is pointless), would be:
>>>>
>>>> Classpath:   HeavyCondLock -> abstraction for creating mutex/cond
>>>> pair
>>>> -> pthreads
>>>>
>>>> Harmony:   HeavyCondLock -> abstraction for creating mutex/cond
>>>> pair -
>>>>> Harmony
>>>>
>>>> And so what you'd be changing is that middle abstraction, rather  
>>>> than
>>>> the HeavyCondLock itself, which actually contains useful
>>>> functionality.  In particular, the HeavyCondLock, and all of its
>>>> Nicely and non-Nicely methods that you like so much, play a crucial
>>>> role in ensuring that we:
>>>>
>>>> 1) Support proper thread accounting while avoiding deadlock and
>>>> spinning.
>>>>
>>>> 2) Support proper thread-to-thread notification while avoiding
>>>> deadlock and spinning.
>>>>
>>>> 3) Support proper soft handshaking (a feature we care about  
>>>> deeply!)
>>>> while avoiding deadlock  and spinning.
>>>>
>>>> 4) Support proper hard handshaking (i.e. GC stop-the-world) while
>>>> avoiding deadlock and spinning.
>>>>
>>>> 5) Support proper thread hijacking (so - a hard handshake between
>>>> just
>>>> two threads) while avoiding deadlock and spinning.  This is kind of
>>>> what suspend() does.  But we also need it to support biased locks.
>>>>
>>>> If we used Harmony monitors directly, this would all become harder,
>>>> because we would not be able to do things like reset the recursion
>>>> count of the lock, which would then make it harder to do things  
>>>> like
>>>> waitNicely(), which is really a central method in how native  
>>>> threads
>>>> works.
>>>
>>> Cond locks are an artifact of pthreads and unnecessary for Harmony
>>> threads.
>>>
>>>>>
>>>>>
>>>>> 3) as a low-priority for people like me, get green thread support
>>>>> back
>>>>> by fitting it into this abstraction and extending where  
>>>>> appropriate
>>>>>
>>>>> I think we're all nearly on the same page, so possibly +1s for  
>>>>> this
>>>>> proposal would be useful.
>>>>
>>>> I don't know what your proposal is yet.  Sometimes you are talking
>>>> about replacing RVMThread, other times you are talking about
>>>> abstracting system calls.  Furthermore, I don't see the point of  
>>>> it.
>>>> The threading code could be largely kept unchanged while supporting
>>>> Harmony.  You have not convinced me otherwise.
>>>>
>>>> -F
>>>
>>> The proposal is simple, anything that is a pthread artifact and not
>>> simply and robustly portable to Harmony is pushed into a package in
>>> sheduler. We abstract the implementation so that pthreads/Classpath
>>> work using the sub package. Harmony gets the standard Harmony native
>>> thread implementation.
>>>
>>> This abstraction should mean we can also get the previous green  
>>> thread
>>> model back.
>>>
>>> Ian
>>>
>>>>>
>>>>>
>>>>> Ian
>>>>>
>>>>>> On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:
>>>>>>
>>>>>>> So firstly here is the Harmony thread API:
>>>>>>>
>>>>>>> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>>>>>>>
>>>>>>> I have problems with pretty much everything that's been said  
>>>>>>> here:
>>>>>>>
>>>>>>> Dave> There's a default trivial implementation where those  
>>>>>>> methods
>>>>>>> are
>>>>>>> implemented as sysCalls to C code that wraps pthread calls.  
>>>>>>> That's
>>>>>>> native threading against GNU Classpath/pthreads."
>>>>>>>
>>>>>>> in the planning for 3.1 I stated that I don't see Classpath as  
>>>>>>> our
>>>>>>> long term class library, assuming it in the thread model, to me,
>>>>>>> appears wrong.
>>>>>>>
>>>>>>> Dave> There's a default trivial implementation where those  
>>>>>>> methods
>>>>>>> are
>>>>>>> implemented as sysCalls to C code that wraps Harmony threading
>>>>>>> APIs.
>>>>>>> That's native threading against Harmony/pthreads. Perhaps this  
>>>>>>> is
>>>>>>> very
>>>>>>> close to (1) and is just some ifdefs in the C code. I haven't
>>>>>>> looked
>>>>>>> at the Harmony library enough to know.
>>>>>>>
>>>>>>> this doesn't fit Harmony. In the current threading code we  
>>>>>>> assume
>>>>>>> cond locks.
>>>>>>>
>>>>>>> Dave> There's an implementation, most likely written in Java,  
>>>>>>> that
>>>>>>> is
>>>>>>> not at all trivial where something functionally like the m-n
>>>>>>> threading
>>>>>>> we had before is provided. Perhaps some old green threading code
>>>>>>> can
>>>>>>> be re-used. A key question here is do you want n-1 or n-m. n-1  
>>>>>>> is
>>>>>>> significantly simpler to implement. This is really a question
>>>>>>> about
>>>>>>> use cases and target platforms. In theory we might want to have
>>>>>>> both
>>>>>>> n-1 and n-m.
>>>>>>>
>>>>>>> So this appears to be creating a task for people working off  
>>>>>>> green
>>>>>>> threads to create a new green thread implementation. The  
>>>>>>> expedient
>>>>>>> thing is to work off where we were (sufficient to run all
>>>>>>> applications, X10 and the majority of the JSR166 TCK).
>>>>>>>
>>>>>>> Dave> The main point is that we avoid re-introducing scheduler
>>>>>>> APIs in
>>>>>>> the default case where we are running with native threading.
>>>>>>> Instead,
>>>>>>> green threading sits below the rest of the JVM and Jikes RVM  
>>>>>>> as a
>>>>>>> whole can be oblivious to how those APIs are implemented. I
>>>>>>> believe
>>>>>>> this should yield a simpler and easier to maintain green  
>>>>>>> threading
>>>>>>> layer than what we had before (which had 10+ years of evolution
>>>>>>> and
>>>>>>> multiple attempts to accommodate hostile native code embedded in
>>>>>>> it).
>>>>>>> I'm sure it will yield a simpler and easier to maintain native
>>>>>>> threading layer.
>>>>>>>
>>>>>>> But we inherently get a non-metacircular implementation and have
>>>>>>> to
>>>>>>> reinvent the green thread world.
>>>>>>>
>>>>>>> Fil, look at the Harmony thread API and you'll see what my point
>>>>>>> is. I
>>>>>>> agree that there are changes elsewhere in the scheduler API and
>>>>>>> with
>>>>>>> reintroducing green threads and a harmony model they should be
>>>>>>> adhered
>>>>>>> to. What I disagree with is exposing the cond locks in the
>>>>>>> scheduler.
>>>>>>>
>>>>>>> Daniel> The previous implementation of Harmony was not ready for
>>>>>>> production use. From the tests we were running it never  
>>>>>>> supported
>>>>>>> multiple processors, and when running with a single processor
>>>>>>> had a
>>>>>>> significant number of failures. Obviously moving to a system
>>>>>>> without
>>>>>>> green threads makes running with a single processor no longer
>>>>>>> possible. This was a well understood problem, and work on
>>>>>>> Harmony to
>>>>>>> fix this issue was deliberately delayed until after native  
>>>>>>> threads
>>>>>>> was
>>>>>>> implemented.
>>>>>>>
>>>>>>> I agree, we have been running against and leading Classpath
>>>>>>> development for a while. Harmony support was at a good level. We
>>>>>>> ran
>>>>>>> SPEC JVM 2008, which I see as essential for a research virtual
>>>>>>> machine. There is much research that can be done on a single  
>>>>>>> core.
>>>>>>>
>>>>>>> Daniel> This relates to a significant issue that I don't think  
>>>>>>> was
>>>>>>> adequately addressed at the time. The announcement of our  
>>>>>>> initial
>>>>>>> compatibility/support for the Harmony libraries was premature,
>>>>>>> and I
>>>>>>> know made a lot of the core team uncomfortable. Such  
>>>>>>> announcements
>>>>>>> have the potential to significantly affect the credibility of
>>>>>>> Jikes
>>>>>>> RVM, and should not be made without some consultation within the
>>>>>>> core
>>>>>>> team. What we claim to support officially in releases of  
>>>>>>> JikesRVM
>>>>>>> has
>>>>>>> a direct impact on everyone working on the project.
>>>>>>>
>>>>>>> So this is highly irritating. If you have concerns:
>>>>>>> 1) e-mail me, and/or
>>>>>>> 2) open JIRAs, there's a whole place for these in the Harmony
>>>>>>> support, and/or
>>>>>>> 3) mail the core mailing list
>>>>>>> Saying that the Harmony support was announced prematurely should
>>>>>>> really have been stated at the time, when it could have been
>>>>>>> rectified. Saying it now is just ridiculous.
>>>>>>>
>>>>>>> In terms of credibility, it is far far more important for our
>>>>>>> credibility that we run on a class library that will give enough
>>>>>>> support to at a least run major benchmark suites. In this case
>>>>>>> SPEC
>>>>>>> JVM 2008.
>>>>>>>
>>>>>>> I would dearly love help with the Harmony implementation, so far
>>>>>>> Alexey has been the only contributor.
>>>>>>>
>>>>>>> Daniel> I think it is clear that we all want to work toward
>>>>>>> compatibility with Harmony (and possibly OpenJDK) but we need to
>>>>>>> do so
>>>>>>> carefully.
>>>>>>>
>>>>>>> As we were already compatible with Harmony, I don't know why  
>>>>>>> we're
>>>>>>> taking a view that we're not.
>>>>>>>
>>>>>>> Daniel> I have left aside my thoughts on the low level technical
>>>>>>> detail here; it is important to take a step back from that and
>>>>>>> solve
>>>>>>> the strategic issues first.
>>>>>>>
>>>>>>> I agree. We either have a minimalist interface or something
>>>>>>> where we
>>>>>>> have multiple configurations. I need multiple configurations and
>>>>>>> if I
>>>>>>> can't do that against the trunk I will have to do my own thing.
>>>>>>>
>>>>>>> In terms of premature decisions, I think merging Fil's code with
>>>>>>> the
>>>>>>> trunk was premature. We should have worked these issues out  
>>>>>>> first.
>>>>>>> We
>>>>>>> should have closed the JIRAs on RVM-91 first. We should have  
>>>>>>> asked
>>>>>>> the
>>>>>>> core list before the merge, first.
>>>>>>>
>>>>>>> Ian
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>>>>> Francisco, CA
>>>>>>> -OSBC tackles the biggest issue in open source: Open Sourcing  
>>>>>>> the
>>>>>>> Enterprise
>>>>>>> -Strategies to boost innovation and cut costs with open source
>>>>>>> participation
>>>>>>> -Receive a $600 discount off the registration fee with the  
>>>>>>> source
>>>>>>> code: SFAD
>>>>>>> http://p.sf.net/sfu/XcvMzF8H
>>>>>>> _______________________________________________
>>>>>>> Jikesrvm-core mailing list
>>>>>>> Jikesrvm-core@...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>>>> Francisco, CA
>>>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>>>> Enterprise
>>>>>> -Strategies to boost innovation and cut costs with open source
>>>>>> participation
>>>>>> -Receive a $600 discount off the registration fee with the source
>>>>>> code: SFAD
>>>>>> http://p.sf.net/sfu/XcvMzF8H
>>>>>> _______________________________________________
>>>>>> Jikesrvm-core mailing list
>>>>>> Jikesrvm-core@...
>>>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>>> Francisco, CA
>>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>>> Enterprise
>>>>> -Strategies to boost innovation and cut costs with open source
>>>>> participation
>>>>> -Receive a $600 discount off the registration fee with the source
>>>>> code: SFAD
>>>>> http://p.sf.net/sfu/XcvMzF8H
>>>>> _______________________________________________
>>>>> Jikesrvm-core mailing list
>>>>> Jikesrvm-core@...
>>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>>> Francisco, CA
>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>>> Enterprise
>>>> -Strategies to boost innovation and cut costs with open source
>>>> participation
>>>> -Receive a $600 discount off the registration fee with the source
>>>> code: SFAD
>>>> http://p.sf.net/sfu/XcvMzF8H
>>>> _______________________________________________
>>>> Jikesrvm-core mailing list
>>>> Jikesrvm-core@...
>>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>>> Francisco, CA
>>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>>> Enterprise
>>> -Strategies to boost innovation and cut costs with open source
>>> participation
>>> -Receive a $600 discount off the registration fee with the source
>>> code: SFAD
>>> http://p.sf.net/sfu/XcvMzF8H
>>> _______________________________________________
>>> Jikesrvm-core mailing list
>>> Jikesrvm-core@...
>>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San  
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the  
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source  
>> participation
>> -Receive a $600 discount off the registration fee with the source  
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San  
> Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the  
> Enterprise
> -Strategies to boost innovation and cut costs with open source  
> participation
> -Receive a $600 discount off the registration fee with the source  
> code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core

------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core