« Return to Thread: [rvm-core] The future of the thread model, interactions with class libraries
Re: [rvm-core] The future of the thread model, interactions with class libraries
by Ian Rogers (nabble)
::
Rate this Message:
2009/2/19 Filip Pizlo <pizlo@...>:
> Here's my position:
>
> - Let's not bring back the old green threads for a 3.0.2 release.
I think not moving would make sense when it is not clear the current
API is where we should be. Even the JIRAs off of RVM-91 don't argue
the API is where we want it to be.
> - I'm fine with any green threads implementation being in there, and
> would be happy to see the RVM support further configurability.
> However, this should be done in a way that: a) preserves the
> simplicity, stability, and performance of the system, and b) results
> in a system that we can all agree upon.
>
> To that end, I tend to think that a green-threads-underneath approach
> is best. The current code is thread-model-agnostic; it just happens
> to make pthreads calls because that 's what we happen to have. I
> don't foresee any big issues in porting that code to use Harmony or
> even Win32 thread APIs. It could call into any thread
> implementation's API, so long as it has some basic notions of locks,
> notification (either semaphores or condition variables), and of course
> a CreateThread call.
So the problem is that we end up replicating pthread locks using Java
like monitors to implement Java like monitors. The current pthread
model in trunk is agnostic in the sense it doesn't assume Linux or
Windows, however, for Harmony the best option is to simply pick up
their work on making an architecture agnostic thread library, not to
implement any low-level thread or locking code of our own. We're
guaranteed a good interaction with the Class library this way and
don't need to implement our own thread compatibility library (which we
could have to with the stacked model where we're building cond locks
out of monitors).
> The interesting questions are:
>
> 1) Can the implementation of threading, which sits beneath the current
> native thread code, be written in Java?
>
> 2) Can such an implementation be done in an orthogonal fashion? I.e.
> can it be done without disturbing native threads?
>
> For (1) we may have to reconsider the notion that all running code
> always has an RVMThread as its context. Introducing some other notion
> of context - why not just call it RVMContext, or just Context - might
> be appropriate. That way, the Context could either be a thread or a
> processor. You might even go further, and add a notion of an
> InterruptContext, which is a special kind of Context not associated
> with a RVMThread, used for interrupt handlers. I've seen cases where
> such a thing would be useful (Ovm, some OS's written in managed
> languages, etc.).
Context sounds like another name for Processor, I think it is easy to
confuse with a thread so I'd stick with Processor. There may be
another use-case for a Context but we've not needed it yet.
Whilst there are lots of things that could be done the expedient way
to get green threads working is to use the green thread code that was
already working (and running the JSR166 TCK as part of the sanity test
runs). Using the old code also means that we are in some way
backwardly compatible, it's just the code has moved around.
> For (2) my main concern is that native threads is what most of us
> use. We shouldn't sacrifice the simplicity of the system to support
> exotic functionality. I tend to think that the native threads code
> is, currently, more stable and robust than green threads ever was. It
> also supports more functionality (granted, some of that functionality
> is "deprecated", like suspend(), but having it is really great for
> implementing, for example, biased locking ... more on that
> later :-)). So, whatever we do to support multiple threading models,
> we should do it in such a way that native threads remains the primary
> production configuration, and its stability, performance, and
> robustness are not sacrificed.
So whilst pthreads are good they don't:
1) run SPEC JVM 2008 - they are assume/rely on Classpath :-( losing
Harmony support is a massive regression from our 3.0.0 and 3.0.1
releases and is a release blocker imo
2) support a lot of existing research implemented on top of the green
thread model
For specific problems there are the open issues on RVM-91.
> I'm sure this is all eminently doable, but I would just like to argue
> strongly for keeping the default configuration clean and sane.
>
> -F
So I think this is the problem. The code as is argues that a sane
configuration consists of a bunch of primitives on top of a cond lock.
That's fine, but it is 1 of 3 ways of writing this code. We can do the
old green thread approach or punt in the Harmony approach. Most of the
current API is identical to the green thread API, however, some of
what were separate concerns previously all got thrown on top of
RVMThread and I think this should be moved back out again. I think
this is a minor issue. As the APIs are largely recycled in the new API
from the green thread API we had before, I'm not sure why such an
approach would be any less clean or sane. My concerns are to get back
green threads, for those of us working on top of it, and fix Harmony.
Ian
> On Feb 19, 2009, at 8:13 AM, Ian Rogers wrote:
>
>> Hi,
>>
>> so the preliminaries:
>>
>> 1) native threads should be the Jikes RVM default!
>> 2) great work Fil!
>>
>> There is a desire to have green threading again (at least from me) and
>> to get Harmony running again, so let's play out some examples of how
>> to say support the sleep operation with different thread models
>> (ignoring cleaning up the arguments and throwing any exceptions that
>> may have been flagged while we were asleep):
>>
>> greentheads - take the running thread, put some time information into
>> it (or a proxy - I forget) and then put it on the waiting queue,
>> schedule other work
>>
>> pthreads - take the thread, get a pthread cond lock, do a syscall to
>> do a timed wait on the pthread cond lock
>>
>> harmony threads - take the thread, call through to a harmony interface
>> that abstracts the sleep
>>
>> so what abstractions do we need: things that are processor local (ie.
>> processors we used to have these, they should probably be an interface
>> to make native threads efficient), a singleton thing to ask global
>> questions to like how many locks are there (ie. the scheduler we used
>> to have), different thread model implementations. I see the packages
>> of:
>>
>> org.jikesrvm.scheduler
>>
>> this is the interface to the rest of the world, nothing externally
>> should be accessing packages beneath here
>>
>> org.jikesrvm.scheduler.greenthreads - the implementation for
>> greenthreads (queues, etc.) that we had for GNU Classpath, to fully
>> support this with Harmony requires creating a DLL wrapper. Support for
>> Classpath requires the syswrap hack.
>>
>> org.jikesrvm.scheduler.nativethreads.pthreads - Fil's work on
>> providing a full pthread model, cond locks won't be exposed above
>> here.
>>
>> org.jikesrvm.scheduler.nativethreads.harmony - As with the original
>> native thread goal, just punt to a native thread library, ie Harmony
>> threads
>>
>> There is likely to be a chance to share things, like queues, between
>> the pthread and green thread code, we can therefore have a utility
>> directory. We should probably not have a lot of different lock or
>> other implementations in the top-level scheduler package, they can be
>> pushed down into other packages.
>>
>> I believe this proposal is controversial as it's bringing back the
>> interface as it was around 3.0.1. I don't propose we copy and paste
>> that code, but the structure will look like it from the outside. Feel
>> free to flame me about why the old interface was the stupidest thing
>> ever as I would like to shake out what are myths and realities (it's
>> not a secret that the old thread model wasn't 100% complete).
>>
>> Can we implement the Harmony code a different way? Yes, I will try
>> this in the short-term. We can implement the Harmony threads using the
>> Apache Portable Runtime. This is diving under the feet of the harmony
>> thread implementation. I believe it is best in the long-term we use
>> the Harmony interface for Harmony, as it means we are abdicating
>> responsibility for threading to Harmony and Harmony can give us
>> compatibility on all the platforms Harmony supports (we are a Java VM,
>> we should run on all platforms as Java is portable). I could also try
>> emulating pthread cond locks with Harmony's Java like monitors, but
>> this would be vile.
>>
>> What will break? AIX ;-) Nothing should break, as this change is just
>> *adding functionality*. The build can be configured to use the models
>> that are complete for the particular architecture. Long-term I'd
>> expect Harmony native threads to be the default. Green threads has a
>> user base. Native pthreads for Classpath will be there for the curious
>> and platforms like AIX where Harmony may not build straight away.
>>
>> What about OpenJDK? Depending on the abstraction it will probably
>> closely resemble either Harmony or pthreads, my belief is probably
>> more like pthreads. Having the flexibility in our model should mean
>> little work in matching OpenJDKs thread model to ours.
>>
>> What does this mean for 3.1? I think the thread model is in a state of
>> flux and it's not in the place it should be. I'd be tempted to make
>> the next release 3.0.2 and bring back the green thread model
>> temporarily (maybe keep native threads on the trunk and revert them on
>> the 3.0.2 branch). I think the current code exposes too many pthread
>> internals and this will be a portability issue for users interfacing
>> with the thread library. Ideally the changes above can be written
>> before the next release and then it could be 3.1, this may be pushing
>> things (there are still many outstanding pthread JIRAs without the
>> addition of native threads for Harmony and incorporating green threads
>> - it would be nice to see more progress on the pthread JIRAs for
>> low-hanging things like method names).
>>
>> So I am off to get all my clothes asbestos lined. This is deliberate,
>> not just flippant, I want everyone who has a complaint about this idea
>> to complain. I feel with the current pthread code going into the trunk
>> I complained and wasn't listened to. I hope I will listen to other
>> people's pov and we can get the abstraction that fits everyone's
>> requirements (or learn why this is a mistake).
>>
>> Regards,
>> Ian
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source
>> participation
>> -Receive a $600 discount off the registration fee with the source
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
> Here's my position:
>
> - Let's not bring back the old green threads for a 3.0.2 release.
I think not moving would make sense when it is not clear the current
API is where we should be. Even the JIRAs off of RVM-91 don't argue
the API is where we want it to be.
> - I'm fine with any green threads implementation being in there, and
> would be happy to see the RVM support further configurability.
> However, this should be done in a way that: a) preserves the
> simplicity, stability, and performance of the system, and b) results
> in a system that we can all agree upon.
>
> To that end, I tend to think that a green-threads-underneath approach
> is best. The current code is thread-model-agnostic; it just happens
> to make pthreads calls because that 's what we happen to have. I
> don't foresee any big issues in porting that code to use Harmony or
> even Win32 thread APIs. It could call into any thread
> implementation's API, so long as it has some basic notions of locks,
> notification (either semaphores or condition variables), and of course
> a CreateThread call.
So the problem is that we end up replicating pthread locks using Java
like monitors to implement Java like monitors. The current pthread
model in trunk is agnostic in the sense it doesn't assume Linux or
Windows, however, for Harmony the best option is to simply pick up
their work on making an architecture agnostic thread library, not to
implement any low-level thread or locking code of our own. We're
guaranteed a good interaction with the Class library this way and
don't need to implement our own thread compatibility library (which we
could have to with the stacked model where we're building cond locks
out of monitors).
> The interesting questions are:
>
> 1) Can the implementation of threading, which sits beneath the current
> native thread code, be written in Java?
>
> 2) Can such an implementation be done in an orthogonal fashion? I.e.
> can it be done without disturbing native threads?
>
> For (1) we may have to reconsider the notion that all running code
> always has an RVMThread as its context. Introducing some other notion
> of context - why not just call it RVMContext, or just Context - might
> be appropriate. That way, the Context could either be a thread or a
> processor. You might even go further, and add a notion of an
> InterruptContext, which is a special kind of Context not associated
> with a RVMThread, used for interrupt handlers. I've seen cases where
> such a thing would be useful (Ovm, some OS's written in managed
> languages, etc.).
Context sounds like another name for Processor, I think it is easy to
confuse with a thread so I'd stick with Processor. There may be
another use-case for a Context but we've not needed it yet.
Whilst there are lots of things that could be done the expedient way
to get green threads working is to use the green thread code that was
already working (and running the JSR166 TCK as part of the sanity test
runs). Using the old code also means that we are in some way
backwardly compatible, it's just the code has moved around.
> For (2) my main concern is that native threads is what most of us
> use. We shouldn't sacrifice the simplicity of the system to support
> exotic functionality. I tend to think that the native threads code
> is, currently, more stable and robust than green threads ever was. It
> also supports more functionality (granted, some of that functionality
> is "deprecated", like suspend(), but having it is really great for
> implementing, for example, biased locking ... more on that
> later :-)). So, whatever we do to support multiple threading models,
> we should do it in such a way that native threads remains the primary
> production configuration, and its stability, performance, and
> robustness are not sacrificed.
So whilst pthreads are good they don't:
1) run SPEC JVM 2008 - they are assume/rely on Classpath :-( losing
Harmony support is a massive regression from our 3.0.0 and 3.0.1
releases and is a release blocker imo
2) support a lot of existing research implemented on top of the green
thread model
For specific problems there are the open issues on RVM-91.
> I'm sure this is all eminently doable, but I would just like to argue
> strongly for keeping the default configuration clean and sane.
>
> -F
So I think this is the problem. The code as is argues that a sane
configuration consists of a bunch of primitives on top of a cond lock.
That's fine, but it is 1 of 3 ways of writing this code. We can do the
old green thread approach or punt in the Harmony approach. Most of the
current API is identical to the green thread API, however, some of
what were separate concerns previously all got thrown on top of
RVMThread and I think this should be moved back out again. I think
this is a minor issue. As the APIs are largely recycled in the new API
from the green thread API we had before, I'm not sure why such an
approach would be any less clean or sane. My concerns are to get back
green threads, for those of us working on top of it, and fix Harmony.
Ian
> On Feb 19, 2009, at 8:13 AM, Ian Rogers wrote:
>
>> Hi,
>>
>> so the preliminaries:
>>
>> 1) native threads should be the Jikes RVM default!
>> 2) great work Fil!
>>
>> There is a desire to have green threading again (at least from me) and
>> to get Harmony running again, so let's play out some examples of how
>> to say support the sleep operation with different thread models
>> (ignoring cleaning up the arguments and throwing any exceptions that
>> may have been flagged while we were asleep):
>>
>> greentheads - take the running thread, put some time information into
>> it (or a proxy - I forget) and then put it on the waiting queue,
>> schedule other work
>>
>> pthreads - take the thread, get a pthread cond lock, do a syscall to
>> do a timed wait on the pthread cond lock
>>
>> harmony threads - take the thread, call through to a harmony interface
>> that abstracts the sleep
>>
>> so what abstractions do we need: things that are processor local (ie.
>> processors we used to have these, they should probably be an interface
>> to make native threads efficient), a singleton thing to ask global
>> questions to like how many locks are there (ie. the scheduler we used
>> to have), different thread model implementations. I see the packages
>> of:
>>
>> org.jikesrvm.scheduler
>>
>> this is the interface to the rest of the world, nothing externally
>> should be accessing packages beneath here
>>
>> org.jikesrvm.scheduler.greenthreads - the implementation for
>> greenthreads (queues, etc.) that we had for GNU Classpath, to fully
>> support this with Harmony requires creating a DLL wrapper. Support for
>> Classpath requires the syswrap hack.
>>
>> org.jikesrvm.scheduler.nativethreads.pthreads - Fil's work on
>> providing a full pthread model, cond locks won't be exposed above
>> here.
>>
>> org.jikesrvm.scheduler.nativethreads.harmony - As with the original
>> native thread goal, just punt to a native thread library, ie Harmony
>> threads
>>
>> There is likely to be a chance to share things, like queues, between
>> the pthread and green thread code, we can therefore have a utility
>> directory. We should probably not have a lot of different lock or
>> other implementations in the top-level scheduler package, they can be
>> pushed down into other packages.
>>
>> I believe this proposal is controversial as it's bringing back the
>> interface as it was around 3.0.1. I don't propose we copy and paste
>> that code, but the structure will look like it from the outside. Feel
>> free to flame me about why the old interface was the stupidest thing
>> ever as I would like to shake out what are myths and realities (it's
>> not a secret that the old thread model wasn't 100% complete).
>>
>> Can we implement the Harmony code a different way? Yes, I will try
>> this in the short-term. We can implement the Harmony threads using the
>> Apache Portable Runtime. This is diving under the feet of the harmony
>> thread implementation. I believe it is best in the long-term we use
>> the Harmony interface for Harmony, as it means we are abdicating
>> responsibility for threading to Harmony and Harmony can give us
>> compatibility on all the platforms Harmony supports (we are a Java VM,
>> we should run on all platforms as Java is portable). I could also try
>> emulating pthread cond locks with Harmony's Java like monitors, but
>> this would be vile.
>>
>> What will break? AIX ;-) Nothing should break, as this change is just
>> *adding functionality*. The build can be configured to use the models
>> that are complete for the particular architecture. Long-term I'd
>> expect Harmony native threads to be the default. Green threads has a
>> user base. Native pthreads for Classpath will be there for the curious
>> and platforms like AIX where Harmony may not build straight away.
>>
>> What about OpenJDK? Depending on the abstraction it will probably
>> closely resemble either Harmony or pthreads, my belief is probably
>> more like pthreads. Having the flexibility in our model should mean
>> little work in matching OpenJDKs thread model to ours.
>>
>> What does this mean for 3.1? I think the thread model is in a state of
>> flux and it's not in the place it should be. I'd be tempted to make
>> the next release 3.0.2 and bring back the green thread model
>> temporarily (maybe keep native threads on the trunk and revert them on
>> the 3.0.2 branch). I think the current code exposes too many pthread
>> internals and this will be a portability issue for users interfacing
>> with the thread library. Ideally the changes above can be written
>> before the next release and then it could be 3.1, this may be pushing
>> things (there are still many outstanding pthread JIRAs without the
>> addition of native threads for Harmony and incorporating green threads
>> - it would be nice to see more progress on the pthread JIRAs for
>> low-hanging things like method names).
>>
>> So I am off to get all my clothes asbestos lined. This is deliberate,
>> not just flippant, I want everyone who has a complaint about this idea
>> to complain. I feel with the current pthread code going into the trunk
>> I complained and wasn't listened to. I hope I will listen to other
>> people's pov and we can get the abstraction that fits everyone's
>> requirements (or learn why this is a mistake).
>>
>> Regards,
>> Ian
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source
>> participation
>> -Receive a $600 discount off the registration fee with the source
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
« Return to Thread: [rvm-core] The future of the thread model, interactions with class libraries
| Free embeddable forum powered by Nabble | Forum Help |
