« Return to Thread: [rvm-core] The future of the thread model, interactions with class libraries
Re: [rvm-core] The future of the thread model, interactions with class libraries
by Ian Rogers (nabble)
::
Rate this Message:
2009/2/20 Filip Pizlo <pizlo@...>:
> Ian,
>
> I've looked at the Harmony threading APIs. The hythread API has
> "monitors". We can just use these as cond locks.
Yep, I've been saying this but it is inherently ugly and would require
changing the interface to fit (why change the interface when it's not
necessary for Classpath/OpenJDK..).
> Or we could just use pthread mutexes and cond's just as we're doing
> now. This doesn't really hurt portability. Even Win32 has cond locks
> now... I don't see anything wrong with having some #ifdefs for code
> like this. It would be ~40 lines of isolated nastiness in sys.C (or
> its equivalent).
Except you need to change all our cond lock code to also acquire and
have behind it a monitor. This ends up being both ugly and
inefficient.
> Or are you suggesting that we use hythread's monitors as our Java
> monitors? And have pointers from the Java head into the C heap? And
> require fat lock acquisition to do a C downcall? That could get quite
> messy, given how our thin locks rely on very specific invariants from
> our fat locks, which essentially forces us to implement fat locks
> ourselves. I guess my whole problem here is that I don't see what
> design you're arguing for.
>
> -F
I am proposing this as the interface between cond lock and the monitor
doesn't fit. If we use cond locks we end up with a stack of:
Java Monitors (on contention, sleeping, etc. require)
Jikes Cond Locks (are implemented by)
Harmony Monitors (are implemented by)
APR Condition Variables, etc. (are implemented by)
Pthread Condition Variables, etc.
I'm proposing:
Java Monitors (on contention, sleeping, etc. require)
Harmony Monitors (are implemented by)
APR Condition Variables, etc. (are implemented by)
Pthread Condition Variables, etc.
The reason is to avoid requiring Jikes Cond Locks to know anything
about Harmony Monitors. In the Classpath case we have a stack of:
Java Monitors (on contention, sleeping, etc. require)
Jikes Cond Locks (are implemented by)
Pthread Condition Variables, etc.
And I agree that's entirely sensible and shouldn't know anything about
Harmony's monitors.
What I propose is that we start from where we are, implement Harmony
support by refactoring RVMThread into an abstraction again. In the
Harmony implementation we go through to Harmony syscalls, in the
native thread implementation we go through to pthread syscalls. We use
an abstraction rather than if-then-else as we're an OO metacircular VM
and that's just how you do things.
I'd go further and say that once we're in this position we're not far
from having an abstraction where green threads would once again fit,
but having processor local is a useful abstraction rather than thread
in that circumstance. So I think the way forward is:
1) start with Classpath and native threads
2) prioritize getting Harmony working by abstracting the current
native threads to support 2 sys call implementations. As this would
mean Jikes Cond Locks were a feature of Classpath/pthread/OpenJDK
native threading it would make sense to move them a package below
scheduler. I think this task should block a major release as it will
get Harmony support back again (an expedient to get a release out in
the mean-time would be to work from before native threading and add in
the half-dozen or so bug fixes, I'd propose this be 3.0.2 - this work
should be off green threads due to Harmony support)
3) as a low-priority for people like me, get green thread support back
by fitting it into this abstraction and extending where appropriate
I think we're all nearly on the same page, so possibly +1s for this
proposal would be useful.
Ian
> On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:
>
>> So firstly here is the Harmony thread API:
>>
>> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>>
>> I have problems with pretty much everything that's been said here:
>>
>> Dave> There's a default trivial implementation where those methods are
>> implemented as sysCalls to C code that wraps pthread calls. That's
>> native threading against GNU Classpath/pthreads."
>>
>> in the planning for 3.1 I stated that I don't see Classpath as our
>> long term class library, assuming it in the thread model, to me,
>> appears wrong.
>>
>> Dave> There's a default trivial implementation where those methods are
>> implemented as sysCalls to C code that wraps Harmony threading APIs.
>> That's native threading against Harmony/pthreads. Perhaps this is very
>> close to (1) and is just some ifdefs in the C code. I haven't looked
>> at the Harmony library enough to know.
>>
>> this doesn't fit Harmony. In the current threading code we assume
>> cond locks.
>>
>> Dave> There's an implementation, most likely written in Java, that is
>> not at all trivial where something functionally like the m-n threading
>> we had before is provided. Perhaps some old green threading code can
>> be re-used. A key question here is do you want n-1 or n-m. n-1 is
>> significantly simpler to implement. This is really a question about
>> use cases and target platforms. In theory we might want to have both
>> n-1 and n-m.
>>
>> So this appears to be creating a task for people working off green
>> threads to create a new green thread implementation. The expedient
>> thing is to work off where we were (sufficient to run all
>> applications, X10 and the majority of the JSR166 TCK).
>>
>> Dave> The main point is that we avoid re-introducing scheduler APIs in
>> the default case where we are running with native threading. Instead,
>> green threading sits below the rest of the JVM and Jikes RVM as a
>> whole can be oblivious to how those APIs are implemented. I believe
>> this should yield a simpler and easier to maintain green threading
>> layer than what we had before (which had 10+ years of evolution and
>> multiple attempts to accommodate hostile native code embedded in it).
>> I'm sure it will yield a simpler and easier to maintain native
>> threading layer.
>>
>> But we inherently get a non-metacircular implementation and have to
>> reinvent the green thread world.
>>
>> Fil, look at the Harmony thread API and you'll see what my point is. I
>> agree that there are changes elsewhere in the scheduler API and with
>> reintroducing green threads and a harmony model they should be adhered
>> to. What I disagree with is exposing the cond locks in the scheduler.
>>
>> Daniel> The previous implementation of Harmony was not ready for
>> production use. From the tests we were running it never supported
>> multiple processors, and when running with a single processor had a
>> significant number of failures. Obviously moving to a system without
>> green threads makes running with a single processor no longer
>> possible. This was a well understood problem, and work on Harmony to
>> fix this issue was deliberately delayed until after native threads was
>> implemented.
>>
>> I agree, we have been running against and leading Classpath
>> development for a while. Harmony support was at a good level. We ran
>> SPEC JVM 2008, which I see as essential for a research virtual
>> machine. There is much research that can be done on a single core.
>>
>> Daniel> This relates to a significant issue that I don't think was
>> adequately addressed at the time. The announcement of our initial
>> compatibility/support for the Harmony libraries was premature, and I
>> know made a lot of the core team uncomfortable. Such announcements
>> have the potential to significantly affect the credibility of Jikes
>> RVM, and should not be made without some consultation within the core
>> team. What we claim to support officially in releases of JikesRVM has
>> a direct impact on everyone working on the project.
>>
>> So this is highly irritating. If you have concerns:
>> 1) e-mail me, and/or
>> 2) open JIRAs, there's a whole place for these in the Harmony
>> support, and/or
>> 3) mail the core mailing list
>> Saying that the Harmony support was announced prematurely should
>> really have been stated at the time, when it could have been
>> rectified. Saying it now is just ridiculous.
>>
>> In terms of credibility, it is far far more important for our
>> credibility that we run on a class library that will give enough
>> support to at a least run major benchmark suites. In this case SPEC
>> JVM 2008.
>>
>> I would dearly love help with the Harmony implementation, so far
>> Alexey has been the only contributor.
>>
>> Daniel> I think it is clear that we all want to work toward
>> compatibility with Harmony (and possibly OpenJDK) but we need to do so
>> carefully.
>>
>> As we were already compatible with Harmony, I don't know why we're
>> taking a view that we're not.
>>
>> Daniel> I have left aside my thoughts on the low level technical
>> detail here; it is important to take a step back from that and solve
>> the strategic issues first.
>>
>> I agree. We either have a minimalist interface or something where we
>> have multiple configurations. I need multiple configurations and if I
>> can't do that against the trunk I will have to do my own thing.
>>
>> In terms of premature decisions, I think merging Fil's code with the
>> trunk was premature. We should have worked these issues out first. We
>> should have closed the JIRAs on RVM-91 first. We should have asked the
>> core list before the merge, first.
>>
>> Ian
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source
>> participation
>> -Receive a $600 discount off the registration fee with the source
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
> Ian,
>
> I've looked at the Harmony threading APIs. The hythread API has
> "monitors". We can just use these as cond locks.
Yep, I've been saying this but it is inherently ugly and would require
changing the interface to fit (why change the interface when it's not
necessary for Classpath/OpenJDK..).
> Or we could just use pthread mutexes and cond's just as we're doing
> now. This doesn't really hurt portability. Even Win32 has cond locks
> now... I don't see anything wrong with having some #ifdefs for code
> like this. It would be ~40 lines of isolated nastiness in sys.C (or
> its equivalent).
Except you need to change all our cond lock code to also acquire and
have behind it a monitor. This ends up being both ugly and
inefficient.
> Or are you suggesting that we use hythread's monitors as our Java
> monitors? And have pointers from the Java head into the C heap? And
> require fat lock acquisition to do a C downcall? That could get quite
> messy, given how our thin locks rely on very specific invariants from
> our fat locks, which essentially forces us to implement fat locks
> ourselves. I guess my whole problem here is that I don't see what
> design you're arguing for.
>
> -F
I am proposing this as the interface between cond lock and the monitor
doesn't fit. If we use cond locks we end up with a stack of:
Java Monitors (on contention, sleeping, etc. require)
Jikes Cond Locks (are implemented by)
Harmony Monitors (are implemented by)
APR Condition Variables, etc. (are implemented by)
Pthread Condition Variables, etc.
I'm proposing:
Java Monitors (on contention, sleeping, etc. require)
Harmony Monitors (are implemented by)
APR Condition Variables, etc. (are implemented by)
Pthread Condition Variables, etc.
The reason is to avoid requiring Jikes Cond Locks to know anything
about Harmony Monitors. In the Classpath case we have a stack of:
Java Monitors (on contention, sleeping, etc. require)
Jikes Cond Locks (are implemented by)
Pthread Condition Variables, etc.
And I agree that's entirely sensible and shouldn't know anything about
Harmony's monitors.
What I propose is that we start from where we are, implement Harmony
support by refactoring RVMThread into an abstraction again. In the
Harmony implementation we go through to Harmony syscalls, in the
native thread implementation we go through to pthread syscalls. We use
an abstraction rather than if-then-else as we're an OO metacircular VM
and that's just how you do things.
I'd go further and say that once we're in this position we're not far
from having an abstraction where green threads would once again fit,
but having processor local is a useful abstraction rather than thread
in that circumstance. So I think the way forward is:
1) start with Classpath and native threads
2) prioritize getting Harmony working by abstracting the current
native threads to support 2 sys call implementations. As this would
mean Jikes Cond Locks were a feature of Classpath/pthread/OpenJDK
native threading it would make sense to move them a package below
scheduler. I think this task should block a major release as it will
get Harmony support back again (an expedient to get a release out in
the mean-time would be to work from before native threading and add in
the half-dozen or so bug fixes, I'd propose this be 3.0.2 - this work
should be off green threads due to Harmony support)
3) as a low-priority for people like me, get green thread support back
by fitting it into this abstraction and extending where appropriate
I think we're all nearly on the same page, so possibly +1s for this
proposal would be useful.
Ian
> On Feb 20, 2009, at 3:47 AM, Ian Rogers wrote:
>
>> So firstly here is the Harmony thread API:
>>
>> http://harmony.apache.org/externals/vm_doc/html/hythread_8c.html
>>
>> I have problems with pretty much everything that's been said here:
>>
>> Dave> There's a default trivial implementation where those methods are
>> implemented as sysCalls to C code that wraps pthread calls. That's
>> native threading against GNU Classpath/pthreads."
>>
>> in the planning for 3.1 I stated that I don't see Classpath as our
>> long term class library, assuming it in the thread model, to me,
>> appears wrong.
>>
>> Dave> There's a default trivial implementation where those methods are
>> implemented as sysCalls to C code that wraps Harmony threading APIs.
>> That's native threading against Harmony/pthreads. Perhaps this is very
>> close to (1) and is just some ifdefs in the C code. I haven't looked
>> at the Harmony library enough to know.
>>
>> this doesn't fit Harmony. In the current threading code we assume
>> cond locks.
>>
>> Dave> There's an implementation, most likely written in Java, that is
>> not at all trivial where something functionally like the m-n threading
>> we had before is provided. Perhaps some old green threading code can
>> be re-used. A key question here is do you want n-1 or n-m. n-1 is
>> significantly simpler to implement. This is really a question about
>> use cases and target platforms. In theory we might want to have both
>> n-1 and n-m.
>>
>> So this appears to be creating a task for people working off green
>> threads to create a new green thread implementation. The expedient
>> thing is to work off where we were (sufficient to run all
>> applications, X10 and the majority of the JSR166 TCK).
>>
>> Dave> The main point is that we avoid re-introducing scheduler APIs in
>> the default case where we are running with native threading. Instead,
>> green threading sits below the rest of the JVM and Jikes RVM as a
>> whole can be oblivious to how those APIs are implemented. I believe
>> this should yield a simpler and easier to maintain green threading
>> layer than what we had before (which had 10+ years of evolution and
>> multiple attempts to accommodate hostile native code embedded in it).
>> I'm sure it will yield a simpler and easier to maintain native
>> threading layer.
>>
>> But we inherently get a non-metacircular implementation and have to
>> reinvent the green thread world.
>>
>> Fil, look at the Harmony thread API and you'll see what my point is. I
>> agree that there are changes elsewhere in the scheduler API and with
>> reintroducing green threads and a harmony model they should be adhered
>> to. What I disagree with is exposing the cond locks in the scheduler.
>>
>> Daniel> The previous implementation of Harmony was not ready for
>> production use. From the tests we were running it never supported
>> multiple processors, and when running with a single processor had a
>> significant number of failures. Obviously moving to a system without
>> green threads makes running with a single processor no longer
>> possible. This was a well understood problem, and work on Harmony to
>> fix this issue was deliberately delayed until after native threads was
>> implemented.
>>
>> I agree, we have been running against and leading Classpath
>> development for a while. Harmony support was at a good level. We ran
>> SPEC JVM 2008, which I see as essential for a research virtual
>> machine. There is much research that can be done on a single core.
>>
>> Daniel> This relates to a significant issue that I don't think was
>> adequately addressed at the time. The announcement of our initial
>> compatibility/support for the Harmony libraries was premature, and I
>> know made a lot of the core team uncomfortable. Such announcements
>> have the potential to significantly affect the credibility of Jikes
>> RVM, and should not be made without some consultation within the core
>> team. What we claim to support officially in releases of JikesRVM has
>> a direct impact on everyone working on the project.
>>
>> So this is highly irritating. If you have concerns:
>> 1) e-mail me, and/or
>> 2) open JIRAs, there's a whole place for these in the Harmony
>> support, and/or
>> 3) mail the core mailing list
>> Saying that the Harmony support was announced prematurely should
>> really have been stated at the time, when it could have been
>> rectified. Saying it now is just ridiculous.
>>
>> In terms of credibility, it is far far more important for our
>> credibility that we run on a class library that will give enough
>> support to at a least run major benchmark suites. In this case SPEC
>> JVM 2008.
>>
>> I would dearly love help with the Harmony implementation, so far
>> Alexey has been the only contributor.
>>
>> Daniel> I think it is clear that we all want to work toward
>> compatibility with Harmony (and possibly OpenJDK) but we need to do so
>> carefully.
>>
>> As we were already compatible with Harmony, I don't know why we're
>> taking a view that we're not.
>>
>> Daniel> I have left aside my thoughts on the low level technical
>> detail here; it is important to take a step back from that and solve
>> the strategic issues first.
>>
>> I agree. We either have a minimalist interface or something where we
>> have multiple configurations. I need multiple configurations and if I
>> can't do that against the trunk I will have to do my own thing.
>>
>> In terms of premature decisions, I think merging Fil's code with the
>> trunk was premature. We should have worked these issues out first. We
>> should have closed the JIRAs on RVM-91 first. We should have asked the
>> core list before the merge, first.
>>
>> Ian
>>
>> ------------------------------------------------------------------------------
>> Open Source Business Conference (OSBC), March 24-25, 2009, San
>> Francisco, CA
>> -OSBC tackles the biggest issue in open source: Open Sourcing the
>> Enterprise
>> -Strategies to boost innovation and cut costs with open source
>> participation
>> -Receive a $600 discount off the registration fee with the source
>> code: SFAD
>> http://p.sf.net/sfu/XcvMzF8H
>> _______________________________________________
>> Jikesrvm-core mailing list
>> Jikesrvm-core@...
>> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
>
> ------------------------------------------------------------------------------
> Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
> -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
> -Strategies to boost innovation and cut costs with open source participation
> -Receive a $600 discount off the registration fee with the source code: SFAD
> http://p.sf.net/sfu/XcvMzF8H
> _______________________________________________
> Jikesrvm-core mailing list
> Jikesrvm-core@...
> https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
>
------------------------------------------------------------------------------
Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA
-OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise
-Strategies to boost innovation and cut costs with open source participation
-Receive a $600 discount off the registration fee with the source code: SFAD
http://p.sf.net/sfu/XcvMzF8H
_______________________________________________
Jikesrvm-core mailing list
Jikesrvm-core@...
https://lists.sourceforge.net/lists/listinfo/jikesrvm-core
« Return to Thread: [rvm-core] The future of the thread model, interactions with class libraries
| Free embeddable forum powered by Nabble | Forum Help |
