Under the hood.

[tobeythorn]

I've been trying to understand how Io works and what goes on under the hood. I'm becoming more and more impressed with it, but am a little confused about a a few things.

What are the statements 123 and "this is a word" and TRUE, FALSE, and NIL? Are they messages, or objects, or messages routed to a special built in hidden object that constructs these objects?

[Jeremy Tregunna-2]

On 4-Jun-09, at 10:31 AM, tobeythorn wrote:

> I've been trying to understand how Io works and what goes on under  
> the hood. I'm becoming more and more impressed with it, but am a  
> little confused about a a few things.
>
> What are the statements 123 and "this is a word" and TRUE, FALSE,  
> and NIL? Are they messages, or objects, or messages routed to a  
> special built in hidden object that constructs these objects?

I don't understand what you mean with "123" and "this is a word"  
please clarify that.

However, TRUE, FALSE and NIL are just constants that point at  
singleton objects that are created in the runtime on startup.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[tobeythorn]

--- In iolanguage@..., Jeremy Tregunna <jeremy.tregunna@...> wrote:

>
>
> On 4-Jun-09, at 10:31 AM, tobeythorn wrote:
>
> > I've been trying to understand how Io works and what goes on under  
> > the hood. I'm becoming more and more impressed with it, but am a  
> > little confused about a a few things.
> >
> > What are the statements 123 and "this is a word" and TRUE, FALSE,  
> > and NIL? Are they messages, or objects, or messages routed to a  
> > special built in hidden object that constructs these objects?
>
> I don't understand what you mean with "123" and "this is a word"  
> please clarify that.
>
> However, TRUE, FALSE and NIL are just constants that point at  
> singleton objects that are created in the runtime on startup.
>
> Regards,
>
> Jeremy Tregunna
> jeremy.tregunna@...
>

What I mean is the construction of numbers and strings.

[Jeremy Tregunna-2]

On 4-Jun-09, at 11:31 AM, tobeythorn wrote:

> --- In iolanguage@..., Jeremy Tregunna  
> <jeremy.tregunna@...> wrote:
>>
>>
>> On 4-Jun-09, at 10:31 AM, tobeythorn wrote:
>>
>>> I've been trying to understand how Io works and what goes on under
>>> the hood. I'm becoming more and more impressed with it, but am a
>>> little confused about a a few things.
>>>
>>> What are the statements 123 and "this is a word" and TRUE, FALSE,
>>> and NIL? Are they messages, or objects, or messages routed to a
>>> special built in hidden object that constructs these objects?
>>
>> I don't understand what you mean with "123" and "this is a word"
>> please clarify that.
>>
>> However, TRUE, FALSE and NIL are just constants that point at
>> singleton objects that are created in the runtime on startup.
>>
>
> What I mean is the construction of numbers and strings.

That's part of the parsing subsystem. When the lexer matches certain  
patterns (like numbers or strings), it will create a token of that  
particular type. When the parser gets it, it'll create an object of  
that type with that value. They are still messages, as everything the  
lexer processes (except comments, though that could be changed) are  
messages.

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[tobeythorn]

--- In iolanguage@..., Jeremy Tregunna <jeremy.tregunna@...> wrote:

>
>
> On 4-Jun-09, at 11:31 AM, tobeythorn wrote:
>
> > --- In iolanguage@..., Jeremy Tregunna  
> > <jeremy.tregunna@> wrote:
> >>
> >>
> >> On 4-Jun-09, at 10:31 AM, tobeythorn wrote:
> >>
> >>> I've been trying to understand how Io works and what goes on under
> >>> the hood. I'm becoming more and more impressed with it, but am a
> >>> little confused about a a few things.
> >>>
> >>> What are the statements 123 and "this is a word" and TRUE, FALSE,
> >>> and NIL? Are they messages, or objects, or messages routed to a
> >>> special built in hidden object that constructs these objects?
> >>
> >> I don't understand what you mean with "123" and "this is a word"
> >> please clarify that.
> >>
> >> However, TRUE, FALSE and NIL are just constants that point at
> >> singleton objects that are created in the runtime on startup.
> >>
> >
> > What I mean is the construction of numbers and strings.
>
> That's part of the parsing subsystem. When the lexer matches certain  
> patterns (like numbers or strings), it will create a token of that  
> particular type. When the parser gets it, it'll create an object of  
> that type with that value. They are still messages, as everything the  
> lexer processes (except comments, though that could be changed) are  
> messages.
>
> Regards,
>
> Jeremy Tregunna
> jeremy.tregunna@...
>

Jeremy,
But a message (or tree of messages) must, at the end of the day, be sent to real objects (right?). If primitive objects like numbers and strings are not the tails of trees of messages, what are? To which object would the message "Hello world" or 12345 be sent?

Thanks for your insight,
-Tobey

[Jeremy Tregunna-2]

On 4-Jun-09, at 3:25 PM, tobeythorn wrote:

> --- In iolanguage@..., Jeremy Tregunna  
> <jeremy.tregunna@...> wrote:
>>
>> That's part of the parsing subsystem. When the lexer matches certain
>> patterns (like numbers or strings), it will create a token of that
>> particular type. When the parser gets it, it'll create an object of
>> that type with that value. They are still messages, as everything the
>> lexer processes (except comments, though that could be changed) are
>> messages.
>>
>> Regards,
>>
>> Jeremy Tregunna
>> jeremy.tregunna@...
>>
> Jeremy,
> But a message (or tree of messages) must, at the end of the day, be  
> sent to real objects (right?). If primitive objects like numbers and  
> strings are not the tails of trees of messages, what are? To which  
> object would the message "Hello world" or 12345 be sent?

Strings and Numbers *ARE* objects just like any other. They just  
happen to be created in C code rather than Io code. You can manipulate  
number objects just like any other.

Literals are sent to the Lobby, which is unlike other objects in that  
given: foo 5 <-- 5 won't be executed in the context of foo, like: foo  
bar <-- would, 5 will be evaluated against the Lobby. Special case,  
but that's how things are.

>
> Thanks for your insight,
> -Tobey
>
>
>
> ------------------------------------
>
> Yahoo! Groups Links
>
>
>

Regards,

Jeremy Tregunna
jeremy.tregunna@...

[tobeythorn]

--- In iolanguage@..., Jeremy Tregunna <jeremy.tregunna@...> wrote:

>
>
> On 4-Jun-09, at 3:25 PM, tobeythorn wrote:
>
> > --- In iolanguage@..., Jeremy Tregunna  
> > <jeremy.tregunna@> wrote:
> >>
> >> That's part of the parsing subsystem. When the lexer matches certain
> >> patterns (like numbers or strings), it will create a token of that
> >> particular type. When the parser gets it, it'll create an object of
> >> that type with that value. They are still messages, as everything the
> >> lexer processes (except comments, though that could be changed) are
> >> messages.
> >>
> >> Regards,
> >>
> >> Jeremy Tregunna
> >> jeremy.tregunna@
> >>
> > Jeremy,
> > But a message (or tree of messages) must, at the end of the day, be  
> > sent to real objects (right?). If primitive objects like numbers and  
> > strings are not the tails of trees of messages, what are? To which  
> > object would the message "Hello world" or 12345 be sent?
>
> Strings and Numbers *ARE* objects just like any other. They just  
> happen to be created in C code rather than Io code. You can manipulate  
> number objects just like any other.
>
> Literals are sent to the Lobby, which is unlike other objects in that  
> given: foo 5 <-- 5 won't be executed in the context of foo, like: foo  
> bar <-- would, 5 will be evaluated against the Lobby. Special case,  
> but that's how things are.
>
> >
> > Thanks for your insight,
> > -Tobey
> >
> >
> >
> > ------------------------------------
> >
> > Yahoo! Groups Links
> >
> >
> >
>
> Regards,
>
> Jeremy Tregunna
> jeremy.tregunna@...
>

Jeremy,
Thanks, that makes sense now!

[Steve Dekorte]

On 2009-06-04, at 7:31 AM, tobeythorn wrote:

> I've been trying to understand how Io works and what goes on under  
> the hood. I'm becoming more and more impressed with it, but am a  
> little confused about a a few things.
>
> What are the statements 123 and "this is a word" and TRUE, FALSE,  
> and NIL? Are they messages, or objects, or messages routed to a  
> special built in hidden object that constructs these objects?

Hi Tobey,

Everything compiles to a tree composed of messages. Literals such as  
numbers and strings in the source become instances of number and  
string objects attached to a Message object's "cachedResult", which  
short circuits the message send and just returns the cachedResult when  
that message is evaled.

true, false, and nil are just objects found in slots in the Lobby  
Protos Core. IIRC, we don't put those in cachedResult, but it would  
probably speed up the code (but slightly alter the semantics) if we did.

Cheers,
- Steve

[tobeythorn]

--- In iolanguage@..., Steve Dekorte <steve@...> wrote:

>
>
> On 2009-06-04, at 7:31 AM, tobeythorn wrote:
>
> > I've been trying to understand how Io works and what goes on under  
> > the hood. I'm becoming more and more impressed with it, but am a  
> > little confused about a a few things.
> >
> > What are the statements 123 and "this is a word" and TRUE, FALSE,  
> > and NIL? Are they messages, or objects, or messages routed to a  
> > special built in hidden object that constructs these objects?
>
> Hi Tobey,
>
> Everything compiles to a tree composed of messages. Literals such as  
> numbers and strings in the source become instances of number and  
> string objects attached to a Message object's "cachedResult", which  
> short circuits the message send and just returns the cachedResult when  
> that message is evaled.
>
> true, false, and nil are just objects found in slots in the Lobby  
> Protos Core. IIRC, we don't put those in cachedResult, but it would  
> probably speed up the code (but slightly alter the semantics) if we did.
>
> Cheers,
> - Steve
>

Steve,
Thanks for your previous reply.
I know it has been a while since I first started this thread, but I have a related inquiry. In the following Io code.

55 myMethod := method("hello" print)
55 myMethod

the output is "hello", which I find quite remarkable, since there are nearly infinite number of numbers and it doesn't seem that 55 is made a slot of lobby by the first line. It isn't obvious to me how 55 and its slot myMethod are remembered.

Furthermore, the 55 doesn't appear to have any protos or slots, while Number does have slots but no protos. Somehow though, 55 can still be printed, or can have arithmetic operators called on it, as if it has Number as a prototype.

The same peculiarities apply to strings as well.

Again,
Thanks for you insight!

[Friedrich Weber-3]

Hi Tobey,

I try to answer some of your questions here :)

`55` *has* a proto, the `Number` object:

Io> 55 protos
==> list(0)
Io> 55 proto == Number
==> true

`0` is just the string representation of the `Number` object (the return
value of `Number asString`).
`55` has no slots because Io uses the concept of differential
inheritance. `Number` itself has a proto, too, the root `Object` object.

Sorry I can't help for the other questions.

Hope that does help you,

Friedrich

[tobeythorn]

--- In iolanguage@..., Friedrich Weber <fred.reichbier@...> wrote:

>
> Hi Tobey,
>
> I try to answer some of your questions here :)
>
> `55` *has* a proto, the `Number` object:
>
> Io> 55 protos
> ==> list(0)
> Io> 55 proto == Number
> ==> true
>
> `0` is just the string representation of the `Number` object (the return
> value of `Number asString`).
> `55` has no slots because Io uses the concept of differential
> inheritance. `Number` itself has a proto, too, the root `Object` object.
>
> Sorry I can't help for the other questions.
>
> Hope that does help you,
>
> Friedrich
>

Friedrich,
That certainly does help. I was reading 0 as meaning the list size was zero... The inheritance hierarchy now makes sense. Thanks man!

[klr_home]

> Literals are sent to the Lobby, which is unlike other objects in that  
> given: foo 5 <-- 5 won't be executed in the context of foo, like: foo  
> bar <-- would, 5 will be evaluated against the Lobby. Special case,  
> but that's how things are.

Really? Is there any reason why literals must be sent to Lobby? I'm new with Io, but deeply interested in language design and implementation.

My understanding of literals is that *conceptually* they are messages, so that they can be represented grammatically by the message node (they are not special cases, conceptually). But all objects respond to such messages by simply returning the literal value. In this view, there's no reason why literals must be evaled against Lobby as a special case. Am I wrong about this?

(That's the conceptual view, as distinguished from implementation. Probably no slot lookup occurs; the literal value is simply used, but the end result is the same.)

Oh wait! Rereading the quote above, I realized another interpretation of what the author wrote. Does that mean that a literal is treated by the parser as always beginning a new message context. I.e., that (foo 5) is equivalent to (foo; 5) [whereas (foo bar) and (foo; bar) are clearly different]. Is that the correct interpretation?  TIA.

[Kevin Edwards-2]

On 7/17/2009 12:58 PM, tobeythorn wrote:

> --- In iolanguage@..., Steve Dekorte <steve@...> wrote:
>> Everything compiles to a tree composed of messages. Literals such as  
>> numbers and strings in the source become instances of number and  
>> string objects attached to a Message object's "cachedResult", which  
>> short circuits the message send and just returns the cachedResult when  
>> that message is evaled.
> [...]
>
> 55 myMethod := method("hello" print)
> 55 myMethod
>
> the output is "hello", which I find quite remarkable, since there
> are nearly infinite number of numbers and it doesn't seem that 55
> is made a slot of lobby by the first line. It isn't obvious to me
> how 55 and its slot myMethod are remembered.
>
> Furthermore, the 55 doesn't appear to have any protos or slots,
> while Number does have slots but no protos. Somehow though, 55
> can still be printed, or can have arithmetic operators called on
> it, as if it has Number as a prototype.
>
> The same peculiarities apply to strings as well.
>

All Number objects between MIN_CACHED_NUMBER (-10) and
MAX_CACHED_NUMBER (256) [IoState_symbols.c] are cached in
IoState->cachedNumbers [IoState.h].  e.g. your example doesn't work
with number literals over 256.

All message strings are also cached in IoState->symbols.

These caches are consulted when constructing a Message's
cachedResult, so the same exact object is used.

Aside from all the hopping around between files, which probably
can't be helped all that much, Steve has designed the code cleanly,
so I tend to just look through it when I'm curious and put together
the chains of calls to make things more obvious.  For example, in
the case of your question:

------
IoMessage_parseName(IoMessage *self, IoLexer *lexer)
- IoMessage_ifPossibleCacheToken_(IoMessage *self, IoToken *p)
-- IONUMBER(IoSeq_asDouble(method))
--- IoState_numberWithDouble_((IoState*)IOSTATE, (double)num)

IoObject *IoState_numberWithDouble_(IoState *self, double n)
{
  long i = (long)n;

  if (self->cachedNumbers && i == n && i >= MIN_CACHED_NUMBER && i
<= MAX_CACHED_NUMBER)
  {
    return List_at_(self->cachedNumbers, i - MIN_CACHED_NUMBER);
  }

  return IoNumber_newWithDouble_(self, n);
}
------

HTH,

Kevin

[Kevin Edwards-2]

On 7/28/2009 9:33 PM, klr_home wrote:
>> Literals are sent to the Lobby, which is unlike other objects in that  
>> given: foo 5 <-- 5 won't be executed in the context of foo, like: foo  
>> bar <-- would, 5 will be evaluated against the Lobby. Special case,  
>> but that's how things are.
>
> Really? Is there any reason why literals must be sent to Lobby?
> I'm new with Io, but deeply interested in language design and
> implementation.

No, not really. :)  Sending messages representing literals to Lobby
is one solution (in theory, allowing interception at the Lobby), but
that's not how Io does it.

From Steve's reply:
"""
Everything compiles to a tree composed of messages. Literals such as
numbers and strings in the source become instances of number and
string objects attached to a Message object's "cachedResult", which
short circuits the message send and just returns the cachedResult
when that message is evaled.
"""

In other words, during parsing, an IoMessage tree is built.  If a
message represents a literal, that literal is used to create/get a
corresponding IoObject (e.g. an IoNumber) and that object is cached
in the message's cachedResult.  When messages with a cached result
are evaluated, they are not sent to a target; instead they just
evaluate to their cachedResult.  They are basically self-evaluating.

If you're interested in Io's design, I suggest you delve into the C
code.  Years ago, I wrote a brief introduction to Io internals but
it's probably out of date by now.

Kevin

[klr_home]

--- In iolanguage@... <mailto:iolanguage@...> ,
Kevin Edwards <edwakev@...> wrote:
> In other words, during parsing, an IoMessage tree is built.  If a
> message represents a literal, that literal is used to create/get a
> corresponding IoObject (e.g. an IoNumber) and that object is cached
> in the message's cachedResult.  When messages with a cached result
> are evaluated, they are not sent to a target; instead they just
> evaluate to their cachedResult.

Thanks for your reply. Actually, I understood that. You're addressing
the implementation view, how the interpreter is written.

My question
<http://tech.groups.yahoo.com/group/iolanguage/message/11757>  concerns
the conceptual view, what the language says a program is supposed to
mean, independently of any implementation. The implementer is free to
take shortcuts as long as it matches the language semantics. You
described the shortcut; I'm interested in the required semantic.

[Steve Dekorte]

On 2009-07-29, at 10:20 AM, Kevin Edwards wrote:
 From Steve's reply:
> """
> Everything compiles to a tree composed of messages. Literals such as
> numbers and strings in the source become instances of number and
> string objects attached to a Message object's "cachedResult", which
> short circuits the message send and just returns the cachedResult
> when that message is evaled.
> """

I don't know if I mentioned this, but this model makes it possible to  
both make anything or nothing a literal as you can add/remove literals  
from the message tree at any time.

For example, you could remove all string and number literals and add a  
forward method to the Lobby that intercepted such messages (like '2'  
or '"foo"') and then returned number and string objects.

Conversely, you could add literals for maps and lists to the message  
tree. For example, right now, list() is just a method. You could  
override it to examine it's contents, and if they are literals, set  
the cacheMessage slot of the calling message object to be the value.  
This probably wouldn't be advisable for mutable objects though.

[Kevin Edwards-2]

On 7/29/2009 5:20 PM, klr_home wrote:

> Kevin Edwards <edwakev@...> wrote:
>> In other words, during parsing, an IoMessage tree is built.  If a
>> message represents a literal, that literal is used to create/get a
>> corresponding IoObject (e.g. an IoNumber) and that object is cached
>> in the message's cachedResult.  When messages with a cached result
>> are evaluated, they are not sent to a target; instead they just
>> evaluate to their cachedResult.
>
> Thanks for your reply. Actually, I understood that. You're addressing
> the implementation view, how the interpreter is written.
>
> My question
> <http://tech.groups.yahoo.com/group/iolanguage/message/11757> concerns
> the conceptual view, what the language says a program is supposed to
> mean, independently of any implementation. The implementer is free to
> take shortcuts as long as it matches the language semantics. You
> described the shortcut; I'm interested in the required semantic.

I see.  But keep in mind that Io's semantics are informal and
ultimately defined by its implementation (excluding bugs), which is
why I responded as I did.

The default semantics of Io does not entail sending literals to
objects for evaluation.  If it did, the receiving object could
interpret literals as it sees fit.  This is not the case.

I believe the correct conceptual interpretation of literals is that
they are self-evaluating messages, or, more accurately, messages
whose results are cached.

Kevin