Thought for the future of Io

[Jeremy Tregunna-2]

Hello folks;

I've been thinking about the future of Io. As some of you know, I have  
my own language project, but the more I work on it, the more it's  
becoming just an implementation of Io. So I'm thinking about  
refocusing my efforts. Instead, what I'm contemplating is writing Io  
in Io. That is to say, trim down the current implementation (not right  
away, but eventually); so it's only effectively a bootstrap. That is,  
have the bootstrap generate LLVM IR, and then the backend on the  
bootstrap compiler generate machine code for whatever platform you're  
on. This should help some of our portability issues out a bit  
(coroutines come to mind).

Doing this would give us an implementation that could even be embedded  
into a kernel running on an embedded device, so long as the kernel  
implements a JIT that translated LLVM IR into something the system can  
execute. But that's just a side note (where I want to eventually take  
this for my own greedy wants and desires).

This would also help us in the long term I'm thinking, in terms of  
contribution to the language. Let's face it, not everyone who learns  
Io knows C, but everyone who learns Io, gets to know Io.

So effectively, I'm talking about breaking out the language into two  
parts:

A build bootstrap staging compiler, which generates LLVM IR (doesn't  
need to know anything about Io itself actually, just needs to generate  
LLVM IR for the language you're defining, in this case Io). This gives  
us a second stage compiler, which knows Io, and in turn can be used  
for AOT translation (the language would need modifications here, to  
restrict some things, like AST manipulation, which severely limits the  
usefulness of writing the language in itself, but that's a later  
discussion) or the stock walking of the tree at runtime ala an  
interpreter.

This second stage need not know anything about LLVM if AOT isn't being  
used.

I'd like to get some opinions before I start to work on this beast.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Guy Hulbert]

On Wed, 2009-28-10 at 13:14 -0400, Jeremy Tregunna wrote:
> I'd like to get some opinions before I start to work on this beast.

Parrot ?

--
--gh

[Jeremy Tregunna-2]

I thought Parrot was dead? Seriously though, it seems to be going  
nowhere in terms of acceptance.

On 2009-10-28, at 1:18 PM, Guy Hulbert wrote:

> On Wed, 2009-28-10 at 13:14 -0400, Jeremy Tregunna wrote:
>> I'd like to get some opinions before I start to work on this beast.
>
> Parrot ?
>
> --
> --gh
>
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Samuel A. Falvo II]

On Wed, Oct 28, 2009 at 10:14 AM, Jeremy Tregunna
<jeremy.tregunna@...> wrote:
> This second stage need not know anything about LLVM if AOT isn't being
> used.
>
> I'd like to get some opinions before I start to work on this beast.

This reminds me somewhat of Squeak Smalltalk's use of a restricted
subset of Smalltalk itself to code the Squeak VM in, which is then
compiled to straight C, whose result is itself compiled again to
native code.

Not only do I find nothing wrong with this, I believe it's a great way
to shake down a language, regardless of what kind of language it is.
I'm all in favor of this.  I just wish I had the time and energy to
contribute to the project.

I did, once upon a time, write a simple Io syntax parser (sans syntax
sugar) in Python.  I'm sure this is one thing you don't need, though.
;)

--
Samuel A. Falvo II

[Guy Hulbert]

On Wed, 2009-28-10 at 13:20 -0400, Jeremy Tregunna wrote:
> I thought Parrot was dead? Seriously though, it seems to be going
> nowhere in terms of acceptance.

They have funding and are making monthly releases with substantial
additions each month.  Several months ago someone reported benchmarks
which were about 100 times slower than they would like.  I would guess
that some of those things have been fixed.

Releases:
http://www.parrot.org/category/news/news

I'm not sure what acceptance means.  They have milestones, which they
are meeting, but I don't think they are at the point where the
performance is sufficient.  The design of parrot was influenced by Io so
I think the plumbing that you would need is/will be there.

Recent Article:
http://www.linux-mag.com/cache/7373/1.html

Start here:
http://dev.perl.org/

--
--gh

[Jeremy Tregunna-2]

On 2009-10-28, at 2:31 PM, Guy Hulbert wrote:

> On Wed, 2009-28-10 at 13:20 -0400, Jeremy Tregunna wrote:
>> I thought Parrot was dead? Seriously though, it seems to be going
>> nowhere in terms of acceptance.
>
> I'm not sure what acceptance means.  They have milestones, which they
> are meeting, but I don't think they are at the point where the
> performance is sufficient.  The design of parrot was influenced by  
> Io so
> I think the plumbing that you would need is/will be there.

Well it's worth a look I guess. That said, I really do like where LLVM  
is at, but I'm not going to rule out one in favour of the other right  
now. But since I'm hoping for some other enthused developers help in  
flushing out this system, whatever this right now fictitious group  
would like to use, is I guess what we'll use :) Right now, it's just me.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Guy Hulbert]

On Wed, 2009-28-10 at 14:34 -0400, Jeremy Tregunna wrote:
> On 2009-10-28, at 2:31 PM, Guy Hulbert wrote:
>
> > On Wed, 2009-28-10 at 13:20 -0400, Jeremy Tregunna wrote:
> >> I thought Parrot was dead? Seriously though, it seems to be going
[snip]
> > I think the plumbing that you would need is/will be there.
>
> Well it's worth a look I guess. That said, I really do like where LLVM
> is at, but I'm not going to rule out one in favour of the other right
> now.

This would not be a problem for Parrot:
http://llvm.org/docs/FAQ.html#source

but in the short term llvm looks like a better bet (if I understand it
correctly).

--
--gh

[Jeremy Tregunna-2]

Well, I have begun work on the io in io using a code generator that  
emits LLVM code. Work on the frontend will start after the backend is  
taking shape. However, I'm running into a bit of a conundrum. Hoping  
to get some people's opinion on it before I tackle it in any way.

Method calls/Blocks.

The way they are in Io right now is such:

Objects which have a mutable scope, a block of code attached (as a  
message argument) and a locals object that gets created. That's the  
crux of it anyway. However, can you tell the difference between:

thisMessage(with, arguments)

and

otherMessage(with, arguments)

?

It's not possible to pick out if one is of type Message or one is of  
type Block until the evaluator is hit; and by this time, it's too late.

Now, what I'm thinking about doing is this:

All messages are function calls as far as the code generator is  
concerned. This however, introduces other problems. How do you know if:

thisName

is actually an unary method/block, or if it holds some other state?  
The simple answer is you don't.

So, what I'm thinking is actually doing this: any message that  
contains parens, is assumed to be a method/block (actually a block for  
the restricted subset of the language I'm working on), and things like  
"thisName" are always assumed to point to some other kind of state.

So what does this mean exactly? Well, it means that all calls to  
methods/blocks need explicit parenthesis. i.e.,

Object clone()

which can be quite a nuisance real quick. However, since this is only  
going to be used to implement the interpreter, or by anyone who really  
likes to write code in a restricted fashion, it's probably not a big  
deal.

Thoughts?

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Jeremy Tregunna-2]

On 2009-10-29, at 5:21 PM, Jeremy Tregunna wrote:

> Objects which have a mutable scope, a block of code attached (as a
> message argument) and a locals object that gets created. That's the
> crux of it anyway. However, can you tell the difference between:
>
> thisMessage(with, arguments)
>
> and
>
> otherMessage(with, arguments)
>
> ?
>
> It's not possible to pick out if one is of type Message or one is of
> type Block until the evaluator is hit; and by this time, it's too  
> late.

Correction, should not have read "Message" there, but rather, "some  
non-Block object".

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[Steve Dekorte]

On 2009-10-29, at 2:21 PM, Jeremy Tregunna wrote:
> Object clone()
>
> which can be quite a nuisance real quick. However, since this is only
> going to be used to implement the interpreter, or by anyone who really
> likes to write code in a restricted fashion, it's probably not a big
> deal.
>
> Thoughts?

It's valid Io code and for a subset of Io semantics - seems like a  
reasonable restriction.

Have you looked at OOC? It looks like they may have done a good job  
designing their object model. Might be a good source of ideas.

[dennisf486]

Just out of curiosity, how would the C binding work in that case?  I assume actually launching an addon would not be a problem - you're just loading a DLL, which is fundementally an OS-supported call, but what about when the DLL attempts to manipulate Io objects using C code?  Once you no longer use any C code to implement the Io interpreter, where will C functions like "UArray* UArray_newWithData_type_encoding_size_copy_(...)" be defined?

One thing you might do, which seems to be in the spirit of what you're trying to accomplish here, is make a thin compatibility/shim library (in C) so that you can still compile the addons.  Instead of the C functions in the shim library doing work on Io data structures directly, they would all be stub functions that call doString() on Io code that does the work.  After all - now that you have Io written in Io, there's no need to keep the C version of the code around; and you probably wouldn't want to, because if you change the organization of your internal data structures the C version of Io would become out of date with respect to it.

It's something I've debated on doing in my Io to C++ binding library - to replace most of my Io C API calls with calls to doString() & Io code (where possible).

[Jeremy Tregunna-2]

Well, one wouldn't need the C addons anymore. Instead, one could  
implement the addons in pure Io code using the FFI mechanism, which as  
I stated before would be needed, and would need to be fairly robust.

On 2009-11-01, at 12:16 AM, dennisf486 wrote:

> Just out of curiosity, how would the C binding work in that case?  I  
> assume actually launching an addon would not be a problem - you're  
> just loading a DLL, which is fundementally an OS-supported call, but  
> what about when the DLL attempts to manipulate Io objects using C  
> code?  Once you no longer use any C code to implement the Io  
> interpreter, where will C functions like "UArray*  
> UArray_newWithData_type_encoding_size_copy_(...)" be defined?
>
> One thing you might do, which seems to be in the spirit of what  
> you're trying to accomplish here, is make a thin compatibility/shim  
> library (in C) so that you can still compile the addons.  Instead of  
> the C functions in the shim library doing work on Io data structures  
> directly, they would all be stub functions that call doString() on  
> Io code that does the work.  After all - now that you have Io  
> written in Io, there's no need to keep the C version of the code  
> around; and you probably wouldn't want to, because if you change the  
> organization of your internal data structures the C version of Io  
> would become out of date with respect to it.
>
> It's something I've debated on doing in my Io to C++ binding library  
> - to replace most of my Io C API calls with calls to doString() & Io  
> code (where possible).
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

[dennisf486]

--- In iolanguage@..., Jeremy Tregunna <jeremy.tregunna@...> wrote:
>
> Well, one wouldn't need the C addons anymore. Instead, one could  
> implement the addons in pure Io code using the FFI mechanism, which as  
> I stated before would be needed, and would need to be fairly robust.

I've been digging around a lot through all the addons' code recently and I wouldn't glibly assume they would all be so simple to replace.  Certainly they could be reimplemented though.

Fair enough; but maybe I posed my question poorly - I mainly brought up the addons as an example of C code that in turn calls Io VM functions.  What I'm mainly concerned about is what about if one is embedding the Io VM in a C/C++ program.  I assume the FFI would allow Io to call C functions; what I'm asking is will C functions be able to call compiled Io functions?

[Jeremy Tregunna-2]

Yes they will. The way I'm thinking is that methods will be c  
functions when compiled down. This would permit one to link a c  
program against an io lib

Regards,

Jeremy Tregunna
Mobile: [phone]

Sent from my iPhone

On 2009-11-01, at 11:14 AM, dennisf486 <dennisf486@...> wrote:

> --- In iolanguage@..., Jeremy Tregunna  
> <jeremy.tregunna@...> wrote:
>>
>> Well, one wouldn't need the C addons anymore. Instead, one could
>> implement the addons in pure Io code using the FFI mechanism, which  
>> as
>> I stated before would be needed, and would need to be fairly robust.
>
> I've been digging around a lot through all the addons' code recently  
> and I wouldn't glibly assume they would all be so simple to  
> replace.  Certainly they could be reimplemented though.
>
> Fair enough; but maybe I posed my question poorly - I mainly brought  
> up the addons as an example of C code that in turn calls Io VM  
> functions.  What I'm mainly concerned about is what about if one is  
> embedding the Io VM in a C/C++ program.  I assume the FFI would  
> allow Io to call C functions; what I'm asking is will C functions be  
> able to call compiled Io functions?
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

[Steve Dekorte]

Looking to alternative implementations of Io, at ~1/3 the speed of C,  
LuaJIT is looking pretty good:

http://lua-users.org/lists/lua-l/2009-10/msg01098.html

A Lua implementation of Io would remove the complexity of having to  
implement a garbage collector and would allow us to use Lua's more  
extensive set of bindings. It might also pave the way toward a  
Javascript implementation at some point.

[Guy Hulbert]

Currently only x86 CPUs are supported. x64 support is in the works.
Other architectures will follow with sufficient demand and/or
sponsoring.

http://luajit.org/faq.html

On Mon, 2009-02-11 at 04:53 -0800, Steve Dekorte wrote:
> Looking to alternative implementations of Io, at ~1/3 the speed of C,
> LuaJIT is looking pretty good:
>
> http://lua-users.org/lists/lua-l/2009-10/msg01098.html
>
> A Lua implementation of Io would remove the complexity of having to
> implement a garbage collector and would allow us to use Lua's more
> extensive set of bindings. It might also pave the way toward a
> Javascript implementation at some point.

--
--gh

[Friedrich]

Steve Dekorte <steve@...> writes:

> Looking to alternative implementations of Io, at ~1/3 the speed of C,  
> LuaJIT is looking pretty good:
>
> http://lua-users.org/lists/lua-l/2009-10/msg01098.html
>
> A Lua implementation of Io would remove the complexity of having to  
> implement a garbage collector and would allow us to use Lua's more  
> extensive set of bindings. It might also pave the way toward a  
> Javascript implementation at some point.
Have you tried it?

Regards
Friedrich

[Steve Dekorte]

On 2009-11-02, at 5:41 AM, Friedrich Dominicus wrote:
> Have you tried it?

Are you referring to LuaJIT 2, implementing Io in Lua, or implementing  
Io In Javascript?

[Jeremy Tregunna-2]

I looked at lua as an implementation language even before c or c++.  
couldn't get the llvm bindings to build. :(

Regards,

Jeremy Tregunna
Mobile: [phone]

Sent from my iPhone

On 2009-11-02, at 7:53 AM, Steve Dekorte <steve@...> wrote:

>
> Looking to alternative implementations of Io, at ~1/3 the speed of C,
> LuaJIT is looking pretty good:
>
> http://lua-users.org/lists/lua-l/2009-10/msg01098.html
>
> A Lua implementation of Io would remove the complexity of having to
> implement a garbage collector and would allow us to use Lua's more
> extensive set of bindings. It might also pave the way toward a
> Javascript implementation at some point.
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

[Steve Dekorte]

On 2009-11-02, at 6:29 AM, Jeremy Tregunna wrote:
> I looked at lua as an implementation language even before c or c++.
> couldn't get the llvm bindings to build. :(

What's the goal that llvm is needed for? If you want small size and/or  
high speed, Lua's vm+jit seems to fit the bill.