stateful closures
|
View:
New views
18 Messages
—
Rating Filter:
Alert me
|
 |
stateful closures
by Alex Tkachman
::
Rate this Message:
Hi guys!
Here is example of useful annotation I've implemented for one of my project - http://groovyconsole.appspot.com/view.groovy?id=33003 I think it provides useful functionality to be included to groovy core. Please let me know what do you think about it Alex --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Robert Fischer
::
Rate this Message:
This is much closer in concept to "inject" than "each". In fact, it's pretty much the same idea as
"inject", if implemented differently. Another option would just be to close over external variables. That can be abstracted to a method to hide the variable scope...which, if you're doing this kind of complicated work, is probably a good idea. I'd almost be sold on an API like this: (0..10).each(sum:0, index:0) { index++ sum = it + index } That could be implemented as: Collection.each = { Map m, Closure c -> def x = new Expando() m.each { k,v -> x."$k" = v } c.delegate = x delegate.each(c) } If you want to do something with the return value of the closure, you're back in inject land, and taking a closure with two arguments. Otherwise, it's pretty much the same. ~~ Robert Fischer, Smokejumper IT Consulting. Enfranchised Mind Blog http://EnfranchisedMind.com/blog Grails Expert Retainer Services http://smokejumperit.com/grails-retainer/ Alex Tkachman wrote: > Hi guys! > > Here is example of useful annotation I've implemented for one of my > project - http://groovyconsole.appspot.com/view.groovy?id=33003 > I think it provides useful functionality to be included to groovy core. > Please let me know what do you think about it > > Alex > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Alex Tkachman
::
Rate this Message:
Robert,
It seems I explained myself incorrectly. It is not about 1001st way to iterate collection. It's about universal way to make any closure stateful object without traditional ugly out of scope variables, which became especially dangerous when you send your closure to different thread or even serialize to another machine.. Alex On Thu, Nov 5, 2009 at 1:34 AM, Robert Fischer <robert.fischer@...> wrote: > This is much closer in concept to "inject" than "each". In fact, it's > pretty much the same idea as "inject", if implemented differently. Another > option would just be to close over external variables. That can be > abstracted to a method to hide the variable scope...which, if you're doing > this kind of complicated work, is probably a good idea. > > I'd almost be sold on an API like this: > (0..10).each(sum:0, index:0) { > index++ > sum = it + index > } > > That could be implemented as: > > Collection.each = { Map m, Closure c -> > def x = new Expando() > m.each { k,v -> x."$k" = v } > c.delegate = x > delegate.each(c) > } > > If you want to do something with the return value of the closure, you're > back in inject land, and taking a closure with two arguments. Otherwise, > it's pretty much the same. > > ~~ Robert Fischer, Smokejumper IT Consulting. > Enfranchised Mind Blog http://EnfranchisedMind.com/blog > > Grails Expert Retainer Services > http://smokejumperit.com/grails-retainer/ > > > Alex Tkachman wrote: >> >> Hi guys! >> >> Here is example of useful annotation I've implemented for one of my >> project - http://groovyconsole.appspot.com/view.groovy?id=33003 >> I think it provides useful functionality to be included to groovy core. >> Please let me know what do you think about it >> >> Alex >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Robert Fischer
::
Rate this Message:
Closures serialize?
~~ Robert Fischer, Smokejumper IT Consulting. Enfranchised Mind Blog http://EnfranchisedMind.com/blog Grails Expert Retainer Services http://smokejumperit.com/grails-retainer/ Alex Tkachman wrote: > Robert, > > It seems I explained myself incorrectly. It is not about 1001st way to > iterate collection. It's about universal way to make any closure > stateful object without traditional ugly out of scope variables, which > became especially dangerous when you send your closure to different > thread or even serialize to another machine.. > > Alex > > On Thu, Nov 5, 2009 at 1:34 AM, Robert Fischer > <robert.fischer@...> wrote: >> This is much closer in concept to "inject" than "each". In fact, it's >> pretty much the same idea as "inject", if implemented differently. Another >> option would just be to close over external variables. That can be >> abstracted to a method to hide the variable scope...which, if you're doing >> this kind of complicated work, is probably a good idea. >> >> I'd almost be sold on an API like this: >> (0..10).each(sum:0, index:0) { >> index++ >> sum = it + index >> } >> >> That could be implemented as: >> >> Collection.each = { Map m, Closure c -> >> def x = new Expando() >> m.each { k,v -> x."$k" = v } >> c.delegate = x >> delegate.each(c) >> } >> >> If you want to do something with the return value of the closure, you're >> back in inject land, and taking a closure with two arguments. Otherwise, >> it's pretty much the same. >> >> ~~ Robert Fischer, Smokejumper IT Consulting. >> Enfranchised Mind Blog http://EnfranchisedMind.com/blog >> >> Grails Expert Retainer Services >> http://smokejumperit.com/grails-retainer/ >> >> >> Alex Tkachman wrote: >>> Hi guys! >>> >>> Here is example of useful annotation I've implemented for one of my >>> project - http://groovyconsole.appspot.com/view.groovy?id=33003 >>> I think it provides useful functionality to be included to groovy core. >>> Please let me know what do you think about it >>> >>> Alex >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Alex Tkachman
::
Rate this Message:
Of course, it should be done carefully (sometimes with very deep
serialization tricks) but as you can see in sources Closure implements Serializable and it done with purpose (imagine use case when closure represent piece of work to be executed on other machine) On Thu, Nov 5, 2009 at 2:14 AM, Robert Fischer <robert.fiecutedscher@...> wrote: > Closures serialize? > > ~~ Robert Fischer, Smokejumper IT Consulting. > Enfranchised Mind Blog http://EnfranchisedMind.com/blog > > Grails Expert Retainer Services > http://smokejumperit.com/grails-retainer/ > > > Alex Tkachman wrote: >> >> Robert, >> >> It seems I explained myself incorrectly. It is not about 1001st way to >> iterate collection. It's about universal way to make any closure >> stateful object without traditional ugly out of scope variables, which >> became especially dangerous when you send your closure to different >> thread or even serialize to another machine.. >> >> Alex >> >> On Thu, Nov 5, 2009 at 1:34 AM, Robert Fischer >> <robert.fischer@...> wrote: >>> >>> This is much closer in concept to "inject" than "each". In fact, it's >>> pretty much the same idea as "inject", if implemented differently. >>> Another >>> option would just be to close over external variables. That can be >>> abstracted to a method to hide the variable scope...which, if you're >>> doing >>> this kind of complicated work, is probably a good idea. >>> >>> I'd almost be sold on an API like this: >>> (0..10).each(sum:0, index:0) { >>> index++ >>> sum = it + index >>> } >>> >>> That could be implemented as: >>> >>> Collection.each = { Map m, Closure c -> >>> def x = new Expando() >>> m.each { k,v -> x."$k" = v } >>> c.delegate = x >>> delegate.each(c) >>> } >>> >>> If you want to do something with the return value of the closure, you're >>> back in inject land, and taking a closure with two arguments. Otherwise, >>> it's pretty much the same. >>> >>> ~~ Robert Fischer, Smokejumper IT Consulting. >>> Enfranchised Mind Blog http://EnfranchisedMind.com/blog >>> >>> Grails Expert Retainer Services >>> http://smokejumperit.com/grails-retainer/ >>> >>> >>> Alex Tkachman wrote: >>>> >>>> Hi guys! >>>> >>>> Here is example of useful annotation I've implemented for one of my >>>> project - http://groovyconsole.appspot.com/view.groovy?id=33003 >>>> I think it provides useful functionality to be included to groovy core. >>>> Please let me know what do you think about it >>>> >>>> Alex >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe from this list, please visit: >>>> >>>> http://xircles.codehaus.org/manage_email >>>> >>>> >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Ted Dunning
::
Rate this Message:
Is it now possible to serialize stateless closures to other machines? On Wed, Nov 4, 2009 at 4:12 PM, Alex Tkachman <alex.tkachman@...> wrote:
-- Ted Dunning, CTO DeepDyve |
|
|
Re: stateful closures
by Ted Dunning
::
Rate this Message:
This would be huge news for the hadoop community. Python has Dumbo ( http://blog.last.fm/2008/05/29/python-hadoop-flying-circus-elephant ) but last time I tried to do this with groovy, it wasn't possible in a nice way. Is it now possible? On Wed, Nov 4, 2009 at 4:21 PM, Alex Tkachman <alex.tkachman@...> wrote: Of course, it should be done carefully (sometimes with very deep -- Ted Dunning, CTO DeepDyve |
|
|
Re: stateful closures
by HamletDRC
::
Rate this Message:
> I think it provides useful functionality to be included to groovy core.
What is the criteria for including a feature in groovy core versus leaving it in a groovy module? I see benefits to having this in either. But is it a library or part of the language? -- Hamlet D'Arcy hamletdrc@... On Wed, Nov 4, 2009 at 1:33 PM, Alex Tkachman <alex.tkachman@...> wrote: > Hi guys! > > Here is example of useful annotation I've implemented for one of my > project - http://groovyconsole.appspot.com/view.groovy?id=33003 > I think it provides useful functionality to be included to groovy core. > Please let me know what do you think about it > > Alex > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email --
Hamlet D'Arcy |
|
|
Re: stateful closures
by Alex Tkachman
::
Rate this Message:
For me it answers for so fundamental question that it should be groovy
core. It also too small for separate module. On Thu, Nov 5, 2009 at 5:38 AM, Hamlet D'Arcy <hamletdrc@...> wrote: >> I think it provides useful functionality to be included to groovy core. > > What is the criteria for including a feature in groovy core versus > leaving it in a groovy module? > > I see benefits to having this in either. But is it a library or part > of the language? > > -- > Hamlet D'Arcy > hamletdrc@... > > > On Wed, Nov 4, 2009 at 1:33 PM, Alex Tkachman <alex.tkachman@...> wrote: >> Hi guys! >> >> Here is example of useful annotation I've implemented for one of my >> project - http://groovyconsole.appspot.com/view.groovy?id=33003 >> I think it provides useful functionality to be included to groovy core. >> Please let me know what do you think about it >> >> Alex >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Alex Tkachman
::
Rate this Message:
Closures are serializable. Period.
But one should remember that defualt serialization of closure involves serialization of owner and delegate, which potentially lead to serialization of huge object graph, so good care and thinking is required when you choose use it. On Thu, Nov 5, 2009 at 3:07 AM, Ted Dunning <ted.dunning@...> wrote: > > Is it now possible to serialize stateless closures to other machines? > > On Wed, Nov 4, 2009 at 4:12 PM, Alex Tkachman <alex.tkachman@...> > wrote: >> >> you send your closure to different >> thread or even serialize to another machine.. > > > -- > Ted Dunning, CTO > DeepDyve > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Graeme Rocher-3
::
Rate this Message:
This is awesome and I would be +1 for it being in core. I would prefer
if we came up with a keyword, like for example instead of @Field we used private: (0..10).smartEach { // accumulated sum // becomes field in closure class private int sum = 0 // index of itearted element // becomes field in closure class private int index = 0 index++ sum += it + index } That makes sense to me since the variables are like private fields of the closure. Other suggestions welcome. Cheers On Thu, Nov 5, 2009 at 8:40 AM, Alex Tkachman <alex.tkachman@...> wrote: > For me it answers for so fundamental question that it should be groovy > core. It also too small for separate module. > > On Thu, Nov 5, 2009 at 5:38 AM, Hamlet D'Arcy <hamletdrc@...> wrote: >>> I think it provides useful functionality to be included to groovy core. >> >> What is the criteria for including a feature in groovy core versus >> leaving it in a groovy module? >> >> I see benefits to having this in either. But is it a library or part >> of the language? >> >> -- >> Hamlet D'Arcy >> hamletdrc@... >> >> >> On Wed, Nov 4, 2009 at 1:33 PM, Alex Tkachman <alex.tkachman@...> wrote: >>> Hi guys! >>> >>> Here is example of useful annotation I've implemented for one of my >>> project - http://groovyconsole.appspot.com/view.groovy?id=33003 >>> I think it provides useful functionality to be included to groovy core. >>> Please let me know what do you think about it >>> >>> Alex >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > -- Graeme Rocher Head of Grails Development SpringSource - A Division of VMware http://www.springsource.com --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Alex Tkachman
::
Rate this Message:
private sounds like a cool idea and if we go this route the next
logical step would be to expect static variables (as C-static) On Thu, Nov 5, 2009 at 9:45 AM, Graeme Rocher <graeme.rocher@...> wrote: > This is awesome and I would be +1 for it being in core. I would prefer > if we came up with a keyword, like for example instead of @Field we > used private: > > (0..10).smartEach { > // accumulated sum > // becomes field in closure class > private int sum = 0 > > // index of itearted element > // becomes field in closure class > private int index = 0 > > index++ > sum += it + index > } > > That makes sense to me since the variables are like private fields of > the closure. Other suggestions welcome. > > Cheers > > On Thu, Nov 5, 2009 at 8:40 AM, Alex Tkachman <alex.tkachman@...> wrote: >> For me it answers for so fundamental question that it should be groovy >> core. It also too small for separate module. >> >> On Thu, Nov 5, 2009 at 5:38 AM, Hamlet D'Arcy <hamletdrc@...> wrote: >>>> I think it provides useful functionality to be included to groovy core. >>> >>> What is the criteria for including a feature in groovy core versus >>> leaving it in a groovy module? >>> >>> I see benefits to having this in either. But is it a library or part >>> of the language? >>> >>> -- >>> Hamlet D'Arcy >>> hamletdrc@... >>> >>> >>> On Wed, Nov 4, 2009 at 1:33 PM, Alex Tkachman <alex.tkachman@...> wrote: >>>> Hi guys! >>>> >>>> Here is example of useful annotation I've implemented for one of my >>>> project - http://groovyconsole.appspot.com/view.groovy?id=33003 >>>> I think it provides useful functionality to be included to groovy core. >>>> Please let me know what do you think about it >>>> >>>> Alex >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe from this list, please visit: >>>> >>>> http://xircles.codehaus.org/manage_email >>>> >>>> >>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > > > -- > Graeme Rocher > Head of Grails Development > SpringSource - A Division of VMware > http://www.springsource.com > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Robert Fischer
::
Rate this Message:
I'm still concerned that this is revealing a bit too much of the implementation details of closures.
But I suppose that ship has sailed. Definitely +1 for the "private" keyword instead of the annotation. ~~ Robert Fischer, Smokejumper IT Consulting. Enfranchised Mind Blog http://EnfranchisedMind.com/blog Grails Expert Retainer Services http://smokejumperit.com/grails-retainer/ Graeme Rocher wrote: > This is awesome and I would be +1 for it being in core. I would prefer > if we came up with a keyword, like for example instead of @Field we > used private: > > (0..10).smartEach { > // accumulated sum > // becomes field in closure class > private int sum = 0 > > // index of itearted element > // becomes field in closure class > private int index = 0 > > index++ > sum += it + index > } > > That makes sense to me since the variables are like private fields of > the closure. Other suggestions welcome. > > Cheers > > On Thu, Nov 5, 2009 at 8:40 AM, Alex Tkachman <alex.tkachman@...> wrote: >> For me it answers for so fundamental question that it should be groovy >> core. It also too small for separate module. >> >> On Thu, Nov 5, 2009 at 5:38 AM, Hamlet D'Arcy <hamletdrc@...> wrote: >>>> I think it provides useful functionality to be included to groovy core. >>> What is the criteria for including a feature in groovy core versus >>> leaving it in a groovy module? >>> >>> I see benefits to having this in either. But is it a library or part >>> of the language? >>> >>> -- >>> Hamlet D'Arcy >>> hamletdrc@... >>> >>> >>> On Wed, Nov 4, 2009 at 1:33 PM, Alex Tkachman <alex.tkachman@...> wrote: >>>> Hi guys! >>>> >>>> Here is example of useful annotation I've implemented for one of my >>>> project - http://groovyconsole.appspot.com/view.groovy?id=33003 >>>> I think it provides useful functionality to be included to groovy core. >>>> Please let me know what do you think about it >>>> >>>> Alex >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe from this list, please visit: >>>> >>>> http://xircles.codehaus.org/manage_email >>>> >>>> >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe from this list, please visit: >>> >>> http://xircles.codehaus.org/manage_email >>> >>> >>> >> --------------------------------------------------------------------- >> To unsubscribe from this list, please visit: >> >> http://xircles.codehaus.org/manage_email >> >> >> > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Jochen Theodorou
::
Rate this Message:
Alex Tkachman schrieb:
> Hi guys! > > Here is example of useful annotation I've implemented for one of my > project - http://groovyconsole.appspot.com/view.groovy?id=33003 > I think it provides useful functionality to be included to groovy core. > Please let me know what do you think about it I hold back a bit to see where the discussion will lead to. Before releasing the last beta of 1.7 I suggested to Guillaume a new kind of variable scope. One that can be declared and used, but the declaration will be done on first visit and further visits will just read it. The could not remember which language had this concept, but someone here mentioned C for local variables this is exactly what I am thinking of. As implementation I was thinking of using a field. Now your @Field looks extremely like that concept. Two fools, one idea? The concerns we had with this implementation are basically about concurrency. Handling that visit once concept is not concurrency friendly, not if you want to set new values. Also I wouldn't like to expose the implementation so much, which means @Field, if it would stay as annotation, should be renamed. But I think for now discussing the concurrency issue is more important. What do you have in mind for the implementation side here? Or is your plan to make the field final? bye blackdrag -- Jochen "blackdrag" Theodorou The Groovy Project Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Alex Tkachman
::
Rate this Message:
1) I very much like Graeme's suggestion of using 'private' instead of annotation
2) I don't think that final field is such a great idea because what we try to solve is making closures stateful and of course we want to make state mutable (see my example). So in my concept the field is assigned in constructor but can be mutated 3) Usually you don't call closure concurrently. At least in gpars we didn't find good uses for that. Main reason is that you usually have delegate which thread dependent. So what we do is clone basic closure and set right delegate in it and such delegate takes care for concurrency concerns anyhow. 4) I think what would be great if for such closure clone () would be implemented by creating the new object of the same class, so stateful fields of cloned closure would be in clear state On Thu, Nov 5, 2009 at 3:30 PM, Jochen Theodorou <blackdrag@...> wrote: > Alex Tkachman schrieb: >> >> Hi guys! >> >> Here is example of useful annotation I've implemented for one of my >> project - http://groovyconsole.appspot.com/view.groovy?id=33003 >> I think it provides useful functionality to be included to groovy core. >> Please let me know what do you think about it > > I hold back a bit to see where the discussion will lead to. Before releasing > the last beta of 1.7 I suggested to Guillaume a new kind of variable scope. > One that can be declared and used, but the declaration will be done on first > visit and further visits will just read it. The could not remember which > language had this concept, but someone here mentioned C for local variables > this is exactly what I am thinking of. As implementation I was thinking of > using a field. > > > Now your @Field looks extremely like that concept. Two fools, one idea? > > The concerns we had with this implementation are basically about > concurrency. Handling that visit once concept is not concurrency friendly, > not if you want to set new values. Also I wouldn't like to expose the > implementation so much, which means @Field, if it would stay as annotation, > should be renamed. But I think for now discussing the concurrency issue is > more important. What do you have in mind for the implementation side here? > Or is your plan to make the field final? > > bye blackdrag > > -- > Jochen "blackdrag" Theodorou > The Groovy Project Tech Lead (http://groovy.codehaus.org) > http://blackdragsview.blogspot.com/ > > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Jochen Theodorou
::
Rate this Message:
Alex Tkachman schrieb:
[...] > 3) Usually you don't call closure concurrently. At least in gpars we > didn't find good uses for that. Main reason is that you usually have > delegate which thread dependent. So what we do is clone basic closure > and set right delegate in it and such delegate takes care for > concurrency concerns anyhow. I wonder about the cloning part a bit. For example: def count = 0 list.each { it.each { count++ }} to count the number of elements in a list of lists. If gpars would now parallelize the outer closure and the inner closure processing, then we get concurrent access to index. IMHO cloning does not help here. In fact parallelizing this looks wrong. Of course then @Field does not make it better, but maybe it is then not the best example. The fundamental principle here is: no side effect. If a closure or any of its subclosures does a write access to a shared variable, it is not to be parallelized. @Field probably removes the shared variable, but not the restriction. So you have an usage for this stuff that is not based on counting? bye blackdrag -- Jochen "blackdrag" Theodorou The Groovy Project Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Alex Tkachman
::
Rate this Message:
Let me try to sketch something where closure takes care both for
internal state and concurrent consistency // general method to schedule call of closure for each element in the collection concurrently // the method will wait for end of execution and return given closure def inParallel (def iter, Closure op) { Executor executor = Executors.newFixedThreadPoolExecutor(10) def cdl = new CountDownLatch () def count = new AtomicInteger(1) // we don't mind here that we use shared variables as everything is obviously under our control iter.each { count.incrementAndGet () executor.execute { op(it) if (count.decrementAndGet () == 0) cdl.countDouwn () } } if (count.decrementAndGet () == 0) cdl.countDouwn () cdl.await () op } // calculate maximum def max = inParalell (0..1000000) { value -> @Fleld def max = new AtomicInteger (Integer.MIN_INTEGER) for(;;) def m = max.get () if (value <= m) break; if (max.compareAndSet (m, value) break; } }.max On Thu, Nov 5, 2009 at 11:50 PM, Jochen Theodorou <blackdrag@...> wrote: > Alex Tkachman schrieb: > [...] >> >> 3) Usually you don't call closure concurrently. At least in gpars we >> didn't find good uses for that. Main reason is that you usually have >> delegate which thread dependent. So what we do is clone basic closure >> and set right delegate in it and such delegate takes care for >> concurrency concerns anyhow. > > I wonder about the cloning part a bit. For example: > > def count = 0 > list.each { it.each { count++ }} > > to count the number of elements in a list of lists. If gpars would now > parallelize the outer closure and the inner closure processing, then we get > concurrent access to index. IMHO cloning does not help here. In fact > parallelizing this looks wrong. Of course then @Field does not make it > better, but maybe it is then not the best example. > > The fundamental principle here is: no side effect. If a closure or any of > its subclosures does a write access to a shared variable, it is not to be > parallelized. @Field probably removes the shared variable, but not the > restriction. > > So you have an usage for this stuff that is not based on counting? > > > bye blackdrag > > -- > Jochen "blackdrag" Theodorou > The Groovy Project Tech Lead (http://groovy.codehaus.org) > http://blackdragsview.blogspot.com/ > > > --------------------------------------------------------------------- > To unsubscribe from this list, please visit: > > http://xircles.codehaus.org/manage_email > > > --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: stateful closures
by Jochen Theodorou
::
Rate this Message:
Alex Tkachman schrieb:
> Let me try to sketch something where closure takes care both for > internal state and concurrent consistency >[...] > > // calculate maximum > def max = inParalell (0..1000000) { value -> > @Fleld def max = new AtomicInteger (Integer.MIN_INTEGER) > > for(;;) > def m = max.get () > > if (value <= m) > break; > > if (max.compareAndSet (m, value) > break; > } > }.max What I take from this example is: * the max field could be final, since the object reference is not changed * the object handles concurrency This is quite different from the simple counter, because a counter with counter++ would require a non final reference and also there is no handling of concurrency Additionally here it would have made sense to have max outside, not inside, since in the end you will use the outside reference anyway.Instead you need to access the field using .max, which requires inParallel to return the closure or else it won't work. That I see as a problem for usage in a normal "each" for example. Also normally I would have made the field, so this .max would requires access to a private field. I think the current way is good enough if the state is exposed to the outside later anyway. That means it is not so good if the state is supposed to be local to the closure only. Here @Field or something similar would help. I just wish there would be more than counting examples bye blackdrag -- Jochen "blackdrag" Theodorou The Groovy Project Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ --------------------------------------------------------------------- To unsubscribe from this list, please visit: http://xircles.codehaus.org/manage_email |
| Free embeddable forum powered by Nabble | Forum Help |
