things

Hello concatenators,

All those talks about concatenativity let me "dream" about a tiny language designed only to exhibit some properties -- or the lack of them. An attempt to have a base of concrete ;-) facts to reason.
Here a kind of informal description I wish you experimented language designers will comment & criticize; before I really start a foolish implementation. It may be written in Io (using a modified version of IoPEG, see http://code.google.com/p/iopeg/), for I'm learning this language.
Then I intend to explore how modifications will let the language get or lose this one or that one property.

======================================
-1- stack based

-2- reverse polish notation

-3- single data type: "thing"
This is a kind of tree which nodes can have any number of leaf or branch children. Or a nestable list. (If someone would try to explain me, outside the list, the difference between a tree and a recursive list -- thanks a lot ;-). Meaning something like

(x (1 (u v) 2) (a b))

or

x
<branch> # or <sequence>
  1
  <branch>
    u
    v
  2
<branch>
  a
  b

-4- plain text data
Basically, things are internally stored as plain text the same way they are literally expressed in source code (thinking at TCL). Wich should be the Lisp-like (...) expression above.

-5- one unique "operational" primitive word: "write"
Write will take the topmost item on the stack and write it out into a tree view like shown above.

        (x (1 2 (u v)) (a b)) write
        ==> tree view

-5- code is thing
Code is data of the type thing, too. There may be some syntactic sugar:

* To define a word -- meaning binding name to code block -- the format

        name : block
maps to
        (name (block))
or in Forth to
        :name block;

* End of lines mark the end of a word definition. They are thus a top-level nesting separator inside a thing literal. So that

        (a b c
        d e f)
maps to
        ((abc)(def))

And
        name1 : block1
        name2 : block2
maps to
        ((name1 (block1) (name2 (block2))

* As a consequence, the stack itself could be stored as an EOL-separated text of thing literals -- instead of a list of things (?!). Pop reads the last line; push appends a new line.

-6- every action is explicitely word call
There are some "meta-primitives", first of all push and pop that must be expressed. Things read (or produced by possible future words) go into an accumulator. Push moves them from the accumulator to the top of the stack. Pop removes the topmost item. Do "does" code that should be on the stack (if the code is a defined word, it may be referenced instead).

        :writeOne 1 write
in Forth could written
        writeOne : 1 push write push do pop pop

If the stack is empty at start, it's empty at the end. This only to show that there is no magic -- only actions -- then I could get rid of that expliciteness. Onother options could be that words are implicitely pop-ed when read on the stack to be done (else the argument is the second item from top of the stack):

        writeOne : 1 push write push do pop

Note that words can be written out like any other thing (this I like very much -- it's like in Io "myMethod code println"):

        showCode : writeOne push write push do pop

-7- the interpreter can be fed with code things
As code is thing, then it can be expressed and "done" at runtime. Maybe something like (untested ;-):

        writeOneTwice : (one push dup write push dup do do pop2) push do pop

Here the whole parenthesized thing is pushed as a thing, then done as code -- if ever it's valid ;-)
========================================

Only when processed with do or write or any future word, a thing will be parsed to exhibit its structure (into a list, probably). But this is then interpreter level implementation.

I would like to add primitives to access and manipulate a thing's internal data and structure. This would also allow Lisp-like or Io-like manipulation of code.

An intriguing idea is to change Thing's structure into a kind of nested record (or dict), or named tree, where nodes have keys/names/indexes. Like for instance:

(point:(kind:center position:(33 99) color(127 0 255)))

point
  kind
    center
  position
    33
    99
  color
    127
    0
    255

[I do not know how to call such a mix of record and list.]
Then a word definition written as
        name:(block)
directly maps to a thing literal without any word-name data item: the word's name is the top-level key of the thing.
Unnamed items would implicitely get an ordinal key:
        (a:x y c:z) --> (a:x 1:y c:z)
The strange thing here is that if ever the stack is still stored and represented as a thing, then it would be a kind of record object -- wich items may be accessible by name... where does this lead ? To a kind of object forth at the base???

Still further, I could add thing parsing ability to the language itself. Then it could parse code, the interpreter could be written in itself -- except for basic primitive such as IO -- and it could even interpret itself?

denis


------
la vita e estrany

[John Nowak]

On Mar 4, 2009, at 5:21 AM, spir wrote:

> It may be written in Io (using a modified version of IoPEG, see http://code.google.com/p/iopeg/)
> , for I'm learning this language.

Why not write it in Factor using Factor's PEG implementation?  You'd  
learn how to use a concatenative language in the process.

> -3- single data type: "thing"
> This is a kind of tree which nodes can have any number of leaf or  
> branch children.

Factor and Joy already allow first-class stacks and arbitrary nesting  
of data types.

> -4- plain text data

Why would you want plain text when you could have structured data?

> -5- one unique "operational" primitive word: "write"

Aka Factor and Joy's '.' word.

> -5- code is thing
> Code is data of the type thing, too.

Joy is the king here.

> comment & criticize; before I really start a foolish implementation.

To be honest, it looks like a waste of time. I'd encourage you to look  
closer at Joy and use Factor before you start thinking of things to  
change. I don't mean do discourage experimentation of course, but you  
asked for criticism, and what you're proposing looks like a (slow)  
dead end.

- John

Le Wed, 4 Mar 2009 05:32:58 -0500,
John Nowak <john@...> s'exprima ainsi:

> > -5- one unique "operational" primitive word: "write"  

> Aka Factor and Joy's '.' word.
???

> > -5- code is thing
> > Code is data of the type thing, too.  

> Joy is the king here.
???

[...]

> To be honest, it looks like a waste of time. I'd encourage you to look  
> closer at Joy and use Factor before you start thinking of things to  
> change. I don't mean do discourage experimentation of course, but you  
> asked for criticism, and what you're proposing looks like a (slow)  
> dead end.

Sorry, I was not clear enough: I do not mean making anything useful. Just to construct some toy language myself, and play with its design, in order to better understand what all of you are talking about.
[From this point of view, comparing with existing languages that have this or that feature makes no sense by itself. I'm not competing...]

Denis
------
la vita e estrany

[William Tanksley, Jr]

spir wrote:
> All those talks about concatenativity let me "dream" about a tiny
> language designed only to exhibit some properties -- or the lack of
> them. An attempt to have a base of concrete ;-) facts to reason.
> Here a kind of informal description I wish you experimented language
> designers will comment & criticize; before I really start a foolish
> implementation. It may be written in Io (using a modified version of
> IoPEG, see http://code.google.com/p/iopeg/), for I'm learning this
> language.

Sounds like fun; I hope you enjoy it.

> Then I intend to explore how modifications will let the language get
> or lose this one or that one property.

I agree that you'll learn something from this. Nowak is right that
you'll also learn something valuable from using a good concatenative
language, so you may want to start by coding something in a known-good
concatenative language -- perhaps it might be worthwhile to code your
new language in Factor or Joy. I'd say that Factor is the more practical
choice, given the amount of documentation and libraries.

> -3- single data type: "thing"
> This is a kind of tree which nodes can have any number of leaf or
> branch children. Or a nestable list. (If someone would try to explain
> me, outside the list, the difference between a tree and a recursive
> list -- thanks a lot ;-). Meaning something like

There's no difference. Some trees can have limitations that lists don't
have; but every list is also a tree.

> -4- plain text data
> Basically, things are internally stored as plain text the same way
> they are literally expressed in source code (thinking at TCL). Wich
> should be the Lisp-like (...) expression above.

I hate this. It's your choice, but I don't think this is one of the
pleasant features of TCL.

> -5- one unique "operational" primitive word: "write"
> Write will take the topmost item on the stack and write it out into a
> tree view like shown above.

Not unique. Most interactive languages have a "show top stack item"
function. Traditionally it's named "." (Forth, I believe, started that
tradition).

> -6- every action is explicitely word call
> There are some "meta-primitives", first of all push and pop that must
> be expressed. Things read (or produced by possible future words) go
> into an accumulator. Push moves them from the accumulator to the top
> of the stack. Pop removes the topmost item. Do "does" code that should
> be on the stack (if the code is a defined word, it may be referenced
> instead).

Why do you have an accumulator? Why not just use the stack directly?

Many Forth processors DO have accumulator registers, but they
automatically merge them with the stack. I don't see why you'd use one.

> :writeOne 1 write
> in Forth could written
> writeOne : 1 push write push do pop pop

In Forth that would be

: writeOne 1 . ;

> An intriguing idea is to change Thing's structure into a kind of
> nested record (or dict), or named tree, where nodes have
> keys/names/indexes. Like for instance:
> (point:(kind:center position:(33 99) color(127 0 255)))

I heard of one language that does this (I lack the time to search, but
it's a derivative of LISP that processed dictionaries instead of lists).
It's interesting to try to imagine what would happen if you attempted to
execute a dictionary -- what would named elements of the dictionary do?

> [I do not know how to call such a mix of record and list.]

It looks like a record that maps from symbol/string to "item"s. By the
way, I would recommend you name "item" something more standard and
descriptive -- if it's always a list, you might as well call it a list.

> Then a word definition written as
> name:(block)
> directly maps to a thing literal without any word-name data item: the
> word's name is the top-level key of the thing.

That's one thing you could do, but it doesn't tell you what happens if
someone types:

name:(some thing with:(something else))

I don't know if you care. I don't, except for idle curiosity.

> denis

-Wm

[Don Groves]

On Mar 4, 2009, at 2:21 AM, spir wrote:

> Hello concatenators,
>
> All those talks about concatenativity let me "dream" about a tiny  
> language designed only to exhibit some properties -- or the lack of  
> them. An attempt to have a base of concrete ;-) facts to reason.
> Here a kind of informal description I wish you experimented language  
> designers will comment & criticize; before I really start a foolish  
> implementation. It may be written in Io (using a modified version of  
> IoPEG, see http://code.google.com/p/iopeg/), for I'm learning this  
> language.
> Then I intend to explore how modifications will let the language get  
> or lose this one or that one property.

You are now officially hooked! No other programming paradigm
in existence (that I've heard of anyway) has created as many new
forms of itself as this one. It's like a virus that takes up residence
in a person's brain and can never be driven out -- it can only be
placated by writing one of your own. This is the source of the old
statement: "When you've seen one Forth, you've seen one Forth."

This is a good thing, however, as it is what got us to this point.
--
don

Le Wed, 4 Mar 2009 12:05:45 -0800,
Don Groves <dgroves@...> s'exprima ainsi:

> On Mar 4, 2009, at 2:21 AM, spir wrote:
>
> > Hello concatenators,
> >
> > All those talks about concatenativity let me "dream" about a tiny  
> > language designed only to exhibit some properties -- or the lack of  
> > them. An attempt to have a base of concrete ;-) facts to reason.
> > Here a kind of informal description I wish you experimented language  
> > designers will comment & criticize; before I really start a foolish  
> > implementation. It may be written in Io (using a modified version of  
> > IoPEG, see http://code.google.com/p/iopeg/), for I'm learning this  
> > language.
> > Then I intend to explore how modifications will let the language get  
> > or lose this one or that one property.
>
> You are now officially hooked! No other programming paradigm
> in existence (that I've heard of anyway) has created as many new
> forms of itself as this one. It's like a virus that takes up residence
> in a person's brain and can never be driven out -- it can only be
> placated by writing one of your own. This is the source of the old
> statement: "When you've seen one Forth, you've seen one Forth."

Maybe I was born with such a virus... Have always loved exploring and designing things. Languages are a wonderful playing field, imo, for they have this exciting complexity level: complex enough to be a world worth exploring and still "catchable" by human brains.
[Natural languages are much more complex indeed, still we all know at least one of them -- cf. Chomsky's competence) but this happens implicitely ; PLs require expliciteness which leads to abstract difficulty instead.]

> --
> don

denis
------
la vita e estrany

Le Wed, 04 Mar 2009 11:47:22 -0800,
William Tanksley <wtanksleyjr@...> s'exprima ainsi:

> spir wrote:
> > All those talks about concatenativity let me "dream" about a tiny
> > language designed only to exhibit some properties -- or the lack of
> > them. An attempt to have a base of concrete ;-) facts to reason.
> > Here a kind of informal description I wish you experimented language
> > designers will comment & criticize; before I really start a foolish
> > implementation. It may be written in Io (using a modified version of
> > IoPEG, see http://code.google.com/p/iopeg/), for I'm learning this
> > language.
>
> Sounds like fun; I hope you enjoy it.
>
> > Then I intend to explore how modifications will let the language get
> > or lose this one or that one property.
>
> I agree that you'll learn something from this. Nowak is right that
> you'll also learn something valuable from using a good concatenative
> language, so you may want to start by coding something in a known-good
> concatenative language -- perhaps it might be worthwhile to code your
> new language in Factor or Joy. I'd say that Factor is the more practical
> choice, given the amount of documentation and libraries.

Yes, I think John Nowak is right, too -- thanks for his sensible advice -- and I'm considering using either Joy or Factor. The point is making something practicle (such as a parser) requires a minimum knowledge: I spent some weeks reaching that point with Io. My choice will probably depend on how fast I may feel "gemütlich" (comfortable, easy, at home) with one or the other language.

> > -4- plain text data
> > Basically, things are internally stored as plain text the same way
> > they are literally expressed in source code (thinking at TCL). Wich
> > should be the Lisp-like (...) expression above.
>
> I hate this. It's your choice, but I don't think this is one of the
> pleasant features of TCL.

I understand this point of view and even agree it's stupid (I'll have to parse them to construct their structure anyway before doing anything). The purpose here is only to have data and code _really_ be the same thing. Then I may get rid of that: parse textual representations (of code or real data) as soon as encountered in source to store them into tree data structures.

> > -5- one unique "operational" primitive word: "write"
> > Write will take the topmost item on the stack and write it out into a
> > tree view like shown above.
>
> Not unique. Most interactive languages have a "show top stack item"
> function. Traditionally it's named "." (Forth, I believe, started that
> tradition).

Sorry, I meant "single". I may use '.' for 'write' -- but it will actually show a tree view.

By the way, I have another issue with "code is data". Code is always valid data -- but is the converse statement true?

> > -6- every action is explicitely word call
> > There are some "meta-primitives", first of all push and pop that must
> > be expressed. Things read (or produced by possible future words) go
> > into an accumulator. Push moves them from the accumulator to the top
> > of the stack. Pop removes the topmost item. Do "does" code that should
> > be on the stack (if the code is a defined word, it may be referenced
> > instead).
>
> Why do you have an accumulator? Why not just use the stack directly?

> Many Forth processors DO have accumulator registers, but they
> automatically merge them with the stack. I don't see why you'd use one.

Again the purpose is primarily self-pedagogical: to make it explicit that everything is an action/word/function:
        (1)
literal alone does nothing: (1) (the value) will briefly live inside the accumulator
        (1) push
is an action that lets 1 available on the stack for further process. I think at John insisting that everything is a function -- a feature I find closely related to concatenativity. May be wrong, but I feel the need to feel it with my guts ;-)

Then again I will get rid of this requirement in favor of a Forth-like behaviour were literals (including code expressions) are implicitely pushed.
One may also read that () means push -- but in fact parens are rather useful for "things" are inherently composite (and nestable): it make clear the difference between eg "(1 2)" and "(1) (2)" while without parens both would read "1 2".

> > An intriguing idea is to change Thing's structure into a kind of
> > nested record (or dict), or named tree, where nodes have
> > keys/names/indexes. Like for instance:
> > (point:(kind:center position:(33 99) color(127 0 255)))
>
> I heard of one language that does this (I lack the time to search, but
> it's a derivative of LISP that processed dictionaries instead of lists).
> It's interesting to try to imagine what would happen if you attempted to
> execute a dictionary -- what would named elements of the dictionary do?
>
> > [I do not know how to call such a mix of record and list.]
>
> It looks like a record that maps from symbol/string to "item"s. By the
> way, I would recommend you name "item" something more standard and
> descriptive -- if it's always a list, you might as well call it a list.

Actually, I could name it node instead. When it happens to be a branch then it is indeed analog to a list -- and may be implemented as a list. Only the outermost thing must be a branch/list, even if it happens to contain only a single leaf node, or even none.
I read something about a data type that gets rid of the distinction between a "1-uple" (how do you call tuples of length 1 in english?) and the single item it holds but I can imagine it practically. Also, a 0-uple should be (equal/identical to) 'nothing'?
 
> > Then a word definition written as
> > name:(block)
> > directly maps to a thing literal without any word-name data item: the
> > word's name is the top-level key of the thing.
>
> That's one thing you could do, but it doesn't tell you what happens if
> someone types:
>
> name:(some thing with:(something else))

[Assuming I get rid of explicite push]
It stores the right hand literal in a dict under the name 'name'. Then what would do "name do"?
some & thing are pushed on the stack. with:(something else) too. If ever it happens to be valid code code, then a combinator (in the sense of joy) could use it as argument. Now, the name "with" is useless, indeed.
After
        name:(some thing with:(something else) do)
then "name do" would push some & thing and should run with:(something else). But it's obviously inconsistent as a code expression because of the name. It seems to me that plain code (the right-hand side of a def) can be only flatly linear, unstructured, except for real data expressions -- I mean intended as data by the programmer.

> I don't know if you care. I don't, except for idle curiosity.

I just realize it builds a syntax for variables ;-)
As code and data have the same form, then
        x : whatever
will store it under the name 'x' even if "whatever" is not *intended* to be the definition of a function/word. The interpreter cannot make the difference at this stage. Only when x is recalled for execution using 'do'.

This is also related to the question I asked above: is data always valid code? Is there any semantic distinction to do? It seems to me that in the worst case it will simply push the whole thing on the stack. It will actually do something when the thing itself contains 'do'. And as I have only one operational primitive (write), then
        x do
will perform something only if x contains tself the expression
        write do
Otherwise there will only be stack changing games.

> > denis
>
> -Wm
>
>


------
la vita e estrany

[John Cowan]

spir scripsit:

> By the way, I have another issue with "code is data". Code is always
> valid data -- but is the converse statement true?

Yes, at least in Joy; any list may be executed.  Joy has a runtime
feature to decide what to do when an unknown symbol is executed; by
default nothing happens, but if the feature is selected Joy will report
an error and jump back to the REPL.

--
John Cowan  cowan@...   http://ccil.org/~cowan
Promises become binding when there is a meeting of the minds and consideration
is exchanged. So it was at King's Bench in common law England; so it was
under the common law in the American colonies; so it was through more than
two centuries of jurisprudence in this country; and so it is today.
       --Specht v. Netscape

[William Tanksley, Jr]

spir wrote:
> William Tanksley <wtanksleyjr@...> s'exprima ainsi:
> By the way, I have another issue with "code is data". Code is always
> valid data -- but is the converse statement true?

Trivially, no. Data will in general error out if you try to execute
it. :-)

More interestingly, it is possible to build a system in which all data
is always valid code, as long as you allow non-termination and data out
of range errors like division by zero or empty-stack in "valid code".
It's necessary to have the code format be entirely flat (non-nested) so
that you can't possibly have syntax errors. It's also essential to
accept the entire input token set, obviously. The simplest way to be
100% sure that every token is always acceptable would be to tokenize
down to bytes or even bits (and then define the meaning of every
possible token), rather than down to words (which limits you to only
words that are listed in some arbitrary dictionary).

In other words, you'd have to build a bytecode interpreter.

> > Why do you have an accumulator? Why not just use the stack directly?

> Again the purpose is primarily self-pedagogical: to make it explicit
> that everything is an action/word/function:

I truly don't understand how making literals _not_ be a function (?)
will help you understand how everything is a function. It looks like
you're designing a language with a deliberate, obvious mistake baked
into it in order to help yourself see why it's a mistake. If that's your
goal, feel free, but I don't see how it's going to teach you anything;
the process of coding anything in the resulting language will be utterly
forbidding, since the trivial mistake of forgetting to push will be
silently ignored for no obvious reason. That mistake (forgetting to
push) seems unrelated to the mistake you're trying to understand (not
making everything a function).

> Then again I will get rid of this requirement in favor of a Forth-like
> behaviour were literals (including code expressions) are implicitely
> pushed.
> One may also read that () means push -- but in fact parens are rather
> useful for "things" are inherently composite (and nestable): it make
> clear the difference between eg "(1 2)" and "(1) (2)" while without
> parens both would read "1 2".

Again, I have no clue whatsoever why you need to have _anything_ "mean"
push. The word "push" has no meaning whatsoever if you lack an
accumulator. I'd need to have some introduction to the value of having
an accumulator; the problems are painfully obvious to me, while the
value remains entirely obscure.

> I read something about a data type that gets rid of the distinction
> between a "1-uple" (how do you call tuples of length 1 in english?)
> and the single item it holds but I can imagine it practically. Also, a
> 0-uple should be (equal/identical to) 'nothing'?

1-tuple and 0-tuple.

Yes, you can implicitly wrap and unwrap 1-tuples if you want (there are
consequences, but I think you'd learn more by experiencing them), but
you can't automatically unwrap a 0-tuple -- what would fall out of it?
In Lisp, a 0-tuple is special. In NIAL there are actually two entirely
different languages that depend entirely on what type a 0-size array is
(is it an array containing nothing of any type, or is it an array
containing nothing of a specific type?).

-Wm

[eas lab]

Dennis wrote:-
* To define a word -- meaning binding name to code block -- the format

> name : block
> maps to
> (name (block))
> or in Forth to
> :name block;

It might help you to first know a bit how forth DOES work:
* tokens in the source are separated by white-spaces.
So ':name' is ONE token.
But ': name' is TWO tokens meaning:
':' =I'm starting a definition to be entered into the dictionary
& the next token [ignore possible exceptions] will be the ID
of this new definition.

> * End of lines mark the end of a word definition.
> They are thus a top-level nesting separator inside a
> thing literal. So that
>
> (a b c
> d e f)
> maps to
> ((abc)(def))

 eol is a unique, very-valuable char for human recognition.
Why do you want to waste it to do what ")" seems to be
capable of doing.
--snip--
> Do "does" code that should be on the stack (if the code
> is a defined word, it may be referenced instead).
>
> :writeOne 1 write
> in Forth could written
> writeOne : 1 push write push do pop pop

This seems to do:
1 = create '1', perhaps like forth: by recognising that
it's an integer, which is VERY different from a word
in the dictionary.
push = is a reserved-word/meta-primitives
  which pushes the previous input-token.

forth automatically pushes any number.
Apparenty you shouldn't 'mention' it unless you
intend to push it.

If you don't want to push the "1", how else
would you want to use it ?
If you don't want to 'use/do' "write" why do
you evoke it ?

But actually, you're OK, since I'm talking about
optimising, which you should not do too early.
So OK, '2 push'  '3 push' '+ do' 'write do';
you would later optimise to '2 3 + .', as you got
to better understand the 'structure'.

What you seem to be doing is interesting:
avoiding reading what the forth inner-interpreter
does and making your own inner-interpreter.

Apparently you are describing the parser/inner-
interpreter. Which you're merging with the next stage.

In computing we normally separate the layers.
Not like also below where you merge the
stack & the accumulator conceptually.

With current technology, data transformation
eg. addition can ONLY be done in an ALU. And
the stack is only passive memory. So when the
'operations are done to the TOS' this is only
virtually.  In order to handle the mental complexity,
we separate the process into different [isolated]
conceptual layers.   You are merging the layers.

If I see mickey mouse dancing on my screen, the
underlying pixels and electronic signals are at
different conceptual levels.

== Chris Glur.