Boo Blocks anyone?

[Rafael Teixeira]

After reading this: http://www.linux-mag.com/id/7526 , I've come to think of making it happen in the managed-land, and obviously that is a better fit for an extensible language like Boo.

Boo closures are almost there as "Blocks" with the exception that perhaps it would be needed to add some type and  flow checking to guarantee that captured state is: (A) serializable (B) readonly.

Some macros and/or attributes around the closures would allow to integrate those blocks into the managed queuing engine.

Integration with a lower-level engine like the cited Apple's GCD (libdispatch) would be a lot harder and maybe it would need to be supported directly in the runtime what would point to Mono, as MS probably would not be interested in following such path.

Any other thoughts?


Rafael "Monoman" Teixeira
---------------------------------------
"To be creative means to be in love with life. You can be creative only if you love life enough that you want to enhance its beauty, you want to bring a little more music to it, a little more poetry to it, a little more dance to it."
Osho

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Justin Chase-2]

In .net 4 there is something that sounds similar to this called PLinq (or Parallel LINQ). I'm not sure when / if mono will implement that so creating a library over this GCD library would probably be useful. If you could make it look similar to PLinq so much the better when mono implements it the porting could be pretty easy, or maybe even mono would just call into the GCD under the hood. Anyway, PLinq is pretty nice for the .net world. It doesn't do any guarantees for read-only-ness of your lambdas but that might be a limitation in general. It would have been nice to have a immutable concept built-into the runtime from the beginning... but oh well here we are now.

Basically you get a static class called Parallel with some methods for queueing enumerations of tasks as well as the AsParallel() extension method for IEnumerable<T>. Under the hood it's simply dispatching a call to the lambda foreach task item using the ThreadPool. Here is some code I wrote that uses it a litte:

http://metasharp.codeplex.com/SourceControl/changeset/view/25093#352346

specifically:

this.Input.AsParallel<IServiceProvider>()
.ForAll(p => { ... });

Where input is IEnumerable<IServiceProvider>.

Do you really need to guarantee that the state is serializable and readonly? It would be nice but probably not necessary and you'd essentially be fighting against the runtime to make it happen. You'd have to have functional language end-to-end to really pull that off I think.

On the other hand maybe you could have a specific delegate Type that tells the compiler to ensure that everything is read-only in a supplied lambda (i.e. Expressions only on closure objects)? If you used that delegate type as a parameter for your calls maybe it could tell the compiler to check for assignments on closures and forbid them? But even then you probably couldn't guarantee anything, since callers could be compiled with different compilers. To some extent that's just a buyer-beware situation where the developer has to know enough about parallel programming to at least know you can't modify something in parallel.

On Fri, Sep 18, 2009 at 9:57 AM, Rafael Teixeira <monoman@...> wrote:

After reading this: http://www.linux-mag.com/id/7526 , I've come to think of making it happen in the managed-land, and obviously that is a better fit for an extensible language like Boo.

Boo closures are almost there as "Blocks" with the exception that perhaps it would be needed to add some type and  flow checking to guarantee that captured state is: (A) serializable (B) readonly.

Some macros and/or attributes around the closures would allow to integrate those blocks into the managed queuing engine.

Integration with a lower-level engine like the cited Apple's GCD (libdispatch) would be a lot harder and maybe it would need to be supported directly in the runtime what would point to Mono, as MS probably would not be interested in following such path.

Any other thoughts?


Rafael "Monoman" Teixeira
---------------------------------------
"To be creative means to be in love with life. You can be creative only if you love life enough that you want to enhance its beauty, you want to bring a little more music to it, a little more poetry to it, a little more dance to it."
Osho

--
Justin Chase
http://www.justnbusiness.com

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Jeffery Olson]

the whole argument for mandatory serialization is because of the aim
to support transparent inter-process communication. My understanding
is (and please correct me if I'm wrong) metasharp is alot like scala
actors and other "hello world" actor model impls in that its targetted
to work within the context of a single process out-of-the-box and, in
a very important twist that many people overlook when learning about
many actor model impls, DOES NOT transparently target multiple
processes on multiple platforms the way that (most notably) erlang
does (there is some documentation out there on integrating scala
actors with a messaging backend to acheive inter-process communication
and i can't vouch for metasharp).

in messaging/distributed systems in general, it is basically a
ubiquiotous idea (and essential to a sane implementation) that you
only want to pass serializable/"dumb data" over the wire anyways,
especially across the process/AppDomain boundary (if you've ever
looked into how the .NET Remoting system released in 2.0 works, it's
really a pretty poorly conceived idea with alot of dangerous
implications for performance and from a design perspective).

as you said, it would largely be the compiler responsibility to
enforce those rules as far as serialization goes, though. f# (a purely
functional language) emits IL like other languages, so I think you'd
be fighting the langauge/compiler and not the runtime itself, per se,
but I digress...

On Fri, Sep 18, 2009 at 9:02 AM, Justin Chase <justin.m.chase@...> wrote:

> In .net 4 there is something that sounds similar to this called PLinq (or
> Parallel LINQ). I'm not sure when / if mono will implement that so creating
> a library over this GCD library would probably be useful. If you could make
> it look similar to PLinq so much the better when mono implements it the
> porting could be pretty easy, or maybe even mono would just call into the
> GCD under the hood. Anyway, PLinq is pretty nice for the .net world. It
> doesn't do any guarantees for read-only-ness of your lambdas but that might
> be a limitation in general. It would have been nice to have a immutable
> concept built-into the runtime from the beginning... but oh well here we are
> now.
> Basically you get a static class called Parallel with some methods for
> queueing enumerations of tasks as well as the AsParallel() extension method
> for IEnumerable<T>. Under the hood it's simply dispatching a call to the
> lambda foreach task item using the ThreadPool. Here is some code I wrote
> that uses it a litte:
> http://metasharp.codeplex.com/SourceControl/changeset/view/25093#352346
> specifically:
>
> this.Input.AsParallel<IServiceProvider>()
> .ForAll(p => { ... });
>
> Where input is IEnumerable<IServiceProvider>.
> Do you really need to guarantee that the state is serializable and readonly?
> It would be nice but probably not necessary and you'd essentially be
> fighting against the runtime to make it happen. You'd have to have
> functional language end-to-end to really pull that off I think.
> On the other hand maybe you could have a specific delegate Type that tells
> the compiler to ensure that everything is read-only in a supplied lambda
> (i.e. Expressions only on closure objects)? If you used that delegate type
> as a parameter for your calls maybe it could tell the compiler to check for
> assignments on closures and forbid them? But even then you probably couldn't
> guarantee anything, since callers could be compiled with different
> compilers. To some extent that's just a buyer-beware situation where the
> developer has to know enough about parallel programming to at least know you
> can't modify something in parallel.
>
> On Fri, Sep 18, 2009 at 9:57 AM, Rafael Teixeira <monoman@...> wrote:
>>
>> After reading this: http://www.linux-mag.com/id/7526 , I've come to think
>> of making it happen in the managed-land, and obviously that is a better fit
>> for an extensible language like Boo.
>>
>> Boo closures are almost there as "Blocks" with the exception that perhaps
>> it would be needed to add some type and  flow checking to guarantee that
>> captured state is: (A) serializable (B) readonly.
>>
>> Some macros and/or attributes around the closures would allow to integrate
>> those blocks into the managed queuing engine.
>>
>> Integration with a lower-level engine like the cited Apple's GCD
>> (libdispatch) would be a lot harder and maybe it would need to be supported
>> directly in the runtime what would point to Mono, as MS probably would not
>> be interested in following such path.
>>
>> Any other thoughts?
>>
>>
>> Rafael "Monoman" Teixeira
>> ---------------------------------------
>> "To be creative means to be in love with life. You can be creative only if
>> you love life enough that you want to enhance its beauty, you want to bring
>> a little more music to it, a little more poetry to it, a little more dance
>> to it."
>> Osho
>>
>>
>
>
>
> --
> Justin Chase
> http://www.justnbusiness.com
>
> >
>

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Cedric Vivier]

On Sat, Sep 19, 2009 at 12:02 AM, Justin Chase <justin.m.chase@...> wrote:

It would have been nice to have a immutable concept built-into the runtime from the beginning... but oh well here we are now.

(...)

On the other hand maybe you could have a specific delegate Type that tells the compiler to ensure that everything is read-only in a supplied lambda (i.e. Expressions only on closure objects)? If you used that delegate type as a parameter for your calls maybe it could tell the compiler to check for assignments on closures and forbid them? But even then you probably couldn't guarantee anything, since callers could be compiled with different compilers.

Above is probably 'solved'/mitigated by .NET 4 Contracts, in particular PureAttribute, the compiler (or, rather the rewriter) would be able to guarantee that I think.

 

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Justin Chase-2]

Yeah good point, I can definitely see the importance of serialization. In this case, you're right, metasharp would almost always simply be run in a single process. Though I'm not entirely sure that PLinq in general suffers that constraint necessarily. Being that it's linq you could probably create a PLinq2Cloud provider (probably), where the parallel query was farmed out to distributed machines. The code to write the queries would be very provider agnostic but I'm sure it would not be trivial to write that kind of provider. Though it would be pretty effin' sweet.

I was not aware of the PureAttribute. I'll have to read up on that. How would you apply that to lambdas though?

On Fri, Sep 18, 2009 at 11:18 AM, Jeffery Olson <olson.jeffery@...> wrote:

the whole argument for mandatory serialization is because of the aim
to support transparent inter-process communication. My understanding
is (and please correct me if I'm wrong) metasharp is alot like scala
actors and other "hello world" actor model impls in that its targetted
to work within the context of a single process out-of-the-box and, in
a very important twist that many people overlook when learning about
many actor model impls, DOES NOT transparently target multiple
processes on multiple platforms the way that (most notably) erlang
does (there is some documentation out there on integrating scala
actors with a messaging backend to acheive inter-process communication
and i can't vouch for metasharp).

in messaging/distributed systems in general, it is basically a
ubiquiotous idea (and essential to a sane implementation) that you
only want to pass serializable/"dumb data" over the wire anyways,
especially across the process/AppDomain boundary (if you've ever
looked into how the .NET Remoting system released in 2.0 works, it's
really a pretty poorly conceived idea with alot of dangerous
implications for performance and from a design perspective).

as you said, it would largely be the compiler responsibility to
enforce those rules as far as serialization goes, though. f# (a purely
functional language) emits IL like other languages, so I think you'd
be fighting the langauge/compiler and not the runtime itself, per se,
but I digress...

On Fri, Sep 18, 2009 at 9:02 AM, Justin Chase <justin.m.chase@...> wrote:
> In .net 4 there is something that sounds similar to this called PLinq (or
> Parallel LINQ). I'm not sure when / if mono will implement that so creating
> a library over this GCD library would probably be useful. If you could make
> it look similar to PLinq so much the better when mono implements it the
> porting could be pretty easy, or maybe even mono would just call into the
> GCD under the hood. Anyway, PLinq is pretty nice for the .net world. It
> doesn't do any guarantees for read-only-ness of your lambdas but that might
> be a limitation in general. It would have been nice to have a immutable
> concept built-into the runtime from the beginning... but oh well here we are
> now.
> Basically you get a static class called Parallel with some methods for
> queueing enumerations of tasks as well as the AsParallel() extension method
> for IEnumerable<T>. Under the hood it's simply dispatching a call to the
> lambda foreach task item using the ThreadPool. Here is some code I wrote
> that uses it a litte:
> http://metasharp.codeplex.com/SourceControl/changeset/view/25093#352346
> specifically:
>
> this.Input.AsParallel<IServiceProvider>()
> .ForAll(p => { ... });
>
> Where input is IEnumerable<IServiceProvider>.
> Do you really need to guarantee that the state is serializable and readonly?
> It would be nice but probably not necessary and you'd essentially be
> fighting against the runtime to make it happen. You'd have to have
> functional language end-to-end to really pull that off I think.
> On the other hand maybe you could have a specific delegate Type that tells
> the compiler to ensure that everything is read-only in a supplied lambda
> (i.e. Expressions only on closure objects)? If you used that delegate type
> as a parameter for your calls maybe it could tell the compiler to check for
> assignments on closures and forbid them? But even then you probably couldn't
> guarantee anything, since callers could be compiled with different
> compilers. To some extent that's just a buyer-beware situation where the
> developer has to know enough about parallel programming to at least know you
> can't modify something in parallel.
>
> On Fri, Sep 18, 2009 at 9:57 AM, Rafael Teixeira <monoman@...> wrote:
>>
>> After reading this: http://www.linux-mag.com/id/7526 , I've come to think
>> of making it happen in the managed-land, and obviously that is a better fit
>> for an extensible language like Boo.
>>
>> Boo closures are almost there as "Blocks" with the exception that perhaps
>> it would be needed to add some type and  flow checking to guarantee that
>> captured state is: (A) serializable (B) readonly.
>>
>> Some macros and/or attributes around the closures would allow to integrate
>> those blocks into the managed queuing engine.
>>
>> Integration with a lower-level engine like the cited Apple's GCD
>> (libdispatch) would be a lot harder and maybe it would need to be supported
>> directly in the runtime what would point to Mono, as MS probably would not
>> be interested in following such path.
>>
>> Any other thoughts?
>>
>>
>> Rafael "Monoman" Teixeira
>> ---------------------------------------
>> "To be creative means to be in love with life. You can be creative only if
>> you love life enough that you want to enhance its beauty, you want to bring
>> a little more music to it, a little more poetry to it, a little more dance
>> to it."
>> Osho
>>
>>
>
>
>
> --

> Justin Chase
> http://www.justnbusiness.com
>
> >
>

--
Justin Chase
http://www.justnbusiness.com

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Rafael Teixeira]

On Fri, Sep 18, 2009 at 1:54 PM, Justin Chase <justin.m.chase@...> wrote:
>
> I was not aware of the PureAttribute. I'll have to read up on that. How would you apply that to lambdas though?

Lambdas in the end become methods, so the compiler could help with that.

What I was thinking was to have something like (swiping the tutorial example):

a = 0 # declare a new variable
getter = pure { return a }
setter = pure { value | a = value } # <-- issues error as it has sideeffects

assert 0 == getter()

For the parallel execution scenario we can think of:

list = ["first.model", "second.model", "third.model"]
parallel for modelName in list # 'parallel' implicitly means a 'pure'
block must follow
  imageName = SwitchExtension(modelName) # valid just a local change
  DoRender(modelName, imageName) # SwitchExtension and DoRender would
need to be 'pure' methods

For parallel and sequential dependency we could define Pipes like in F#

list = ["first.model", "second.model", "third.model"]
pipe ValidateModel(modelName as string)
    model = Model(modelName)
    return model if model.IsValid() else null

pipe RenderModel(model as Model)
    if model != null
        renderer = Renderer(model)
        renderer.DoRender()

list -> ValidateModel  -> RenderModel



Rafael "Monoman" Teixeira
---------------------------------------
"To be creative means to be in love with life. You can be creative
only if you love life enough that you want to enhance its beauty, you
want to bring a little more music to it, a little more poetry to it, a
little more dance to it."
Osho

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Justin Chase-2]

That looks pretty damn good to me.

On Fri, Sep 18, 2009 at 1:26 PM, Rafael Teixeira <monoman@...> wrote:

On Fri, Sep 18, 2009 at 1:54 PM, Justin Chase <justin.m.chase@...> wrote:
>
> I was not aware of the PureAttribute. I'll have to read up on that. How would you apply that to lambdas though?

Lambdas in the end become methods, so the compiler could help with that.

What I was thinking was to have something like (swiping the tutorial example):

a = 0 # declare a new variable
getter = pure { return a }
setter = pure { value | a = value } # <-- issues error as it has sideeffects

assert 0 == getter()

For the parallel execution scenario we can think of:

list = ["first.model", "second.model", "third.model"]
parallel for modelName in list # 'parallel' implicitly means a 'pure'
block must follow
 imageName = SwitchExtension(modelName) # valid just a local change
 DoRender(modelName, imageName) # SwitchExtension and DoRender would
need to be 'pure' methods

For parallel and sequential dependency we could define Pipes like in F#

list = ["first.model", "second.model", "third.model"]
pipe ValidateModel(modelName as string)
   model = Model(modelName)
   return model if model.IsValid() else null

pipe RenderModel(model as Model)
   if model != null
       renderer = Renderer(model)
       renderer.DoRender()

list -> ValidateModel  -> RenderModel

Rafael "Monoman" Teixeira
---------------------------------------
"To be creative means to be in love with life. You can be creative
only if you love life enough that you want to enhance its beauty, you
want to bring a little more music to it, a little more poetry to it, a
little more dance to it."
Osho

--
Justin Chase
http://www.justnbusiness.com

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Luciano Ramalho]

On Sep 18, 11:57 am, Rafael Teixeira <mono...@...> wrote:
> Boo closures are almost there as "Blocks" with the exception that perhaps it
> would be needed to add some type and  flow checking to guarantee that
> captured state is: (A) serializable (B) readonly.

Why the requirement for readonly?

--
Luciano Ramalho

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Rodrigo B. de Oliveira]

I second that.

On Fri, Sep 18, 2009 at 3:38 PM, Justin Chase <justin.m.chase@...> wrote:
> That looks pretty damn good to me.
>
> On Fri, Sep 18, 2009 at 1:26 PM, Rafael Teixeira <monoman@...> wrote:
>> ...

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

[Justin Chase-2]

By read-only I think  he means immutability. Which means that you cannot change the state of objects used in your parallel methods. Changing state is also known as "side-effects" which is the root of all evil in parallel land.

On Sat, Sep 19, 2009 at 5:42 AM, Luciano Ramalho <ramalho@...> wrote:

On Sep 18, 11:57 am, Rafael Teixeira <mono...@...> wrote:
> Boo closures are almost there as "Blocks" with the exception that perhaps it
> would be needed to add some type and  flow checking to guarantee that
> captured state is: (A) serializable (B) readonly.

Why the requirement for readonly?

--
Luciano Ramalho

--
Justin Chase
http://www.justnbusiness.com

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---