Spawn proposal strawman

[kevin curtis]

Just an idea -  () seems natural for AST's. jsonML is more than good.

I am _definitely_ not suggesting building a jsonML object graph if the
programmer does not require it. eg sax vs dom.

Is your approach:

// jsonML AST string
var js_str = "2 + 2"
var jsonML_ast_str = eval.parse(js_str)
print(jsonML_ast_str) // ["+", 2, 2] or [ast.plus, 2, 2]
var res = eval.execute(jsonML_ast_str)
print(res) // 4

// if the programmer wants to manipulate the jsonML AST in memory
var js_lst = ["+", 2]
js_lst[2] = 2
var jsonML_ast_str = JSON.stringify(js_lst)
... then same as above

The eval.execute function (implemented in c/c++) would take the
jsonML_ast_str and build the native c/c++ ast tree. Which is then
walked to generate bytecode (and maybe machine code) for execution.


In the original code snippet which prompted the discussion:
eval.hermetic(program :(string | AST), thisValue, optBindings) :any

I wasn't sure if the AST type meant a string (in jsonML) or a
reference to the native AST object (in c++) which could be be
manipulated via an api.

On Tue, May 12, 2009 at 8:31 AM, Brendan Eich <brendan@...> wrote:

> On May 12, 2009, at 12:24 AM, kevin curtis wrote:
>
>> JsonML looks good for for an AST handling:
>>
>> ["||",
>>  ["||",
>>    ["Id", "X"],
>>    ["Id", "Y"]],
>>  ["Id", "Z"]]
>
> Yes.
>
>
>> Maybe the 'canonical' AST serialized string format could actually be
>> more scheme-y:
>>
>> (or (or X Y) Z)
>>
>> JsonML could be used for building pure js in-memory AST graphs which
>> could then be easily stringified to the 'canonical' format.
>
> JsonML wouldn't be used to build object graphs -- the JSON decoder would do
> that given JsonML in a string, from the AST encoder. That's the point I made
> in the words you bottom-cite about not mandating a big fat object graph if
> the use-case doesn't need the graph, just the string containing the AST
> serialization.
>
>
>> The benefit is that a scheme-y format could help the thinking on the
>> semantics for es6/harmony.
>
> That seems like no benefit in memory use or cycles, only in thinking. If you
> squint, don't the []s turn into ()s? :-P
>
>
>> (Downside compared to a JSON  canonical format is that with JSON the
>> parsing/stringifying is free via the JSON api in es5).
>
> This is a big downside.
>
>
>> For convenience JSON could remain JSON in this scheme-y format:
>> var x = [1,4,5]
>> becomes:
>> (var x [1,4,5])
>
> I don't see why we'd invent a third language.
>
> /be
>
>

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Kris Kowal-2]

On Mon, May 11, 2009 at 4:21 PM, Brendan Eich <brendan@...> wrote:

> On May 11, 2009, at 4:10 PM, Kris Kowal wrote:
>> Perhaps I'm behind on the times, but I'm under the impression that
>> presently the behavior of this function "foo" declaration has no
>> standard behavior:
>>
>> (function () {
>>  function foo() {
>>  }
>> })();
>
> No, that's fully specified by ES3.

Once again, I've been chasing a JS phantom.  That cuts the complexity
of the options tree roughly in half.  Let's consider:

let asts = {};
let memos = {};

// loading
if (Object.prototype.hasOwnProperty.call(asts, id))
   return id;
let ast = parse(moduleText);
asts[id] = ast;

// executing
if (Object.prototype.hasOwnProperty.call(memo, id))
   return id;
let ast = asts[id];
let exports = memo[id] = {};
let require = Require(id);
execute(ast, {require, exports});

Furthermore, let's assume that "execute" enforces "use lexical scope"
and "use strict".

These are the ramifications if I understand correctly:

 * free assignment becomes an error at run time.
 * free variable access, apart from primordials, require, and exports
throw reference errors.
 * the object bearing the primordials has no name.
 * global object has no name.
 * the bottom scope has no name.
 * default "this" for functions and the bottom-scope is "undefined".
 * function statements are local to the module and only accessible lexically.
 * var and let declarations in the bottom scope.
 * "require" and "exports" get injected into the bottom scope.

What scope contains primordials?  Should primordials be injected into
the bottom scope before "require" and "exports" or should their be two
scopes (global, local) in a module?  I see several potential
definitions of execute:

execute(program:AST|String, scope);
 // wherein we create a new global scope and add require and exports
for each program

execute(program:AST|String, local);
 // wherein a shared frozen global scope is implied and a local scope
is pushed above it

execute(program:AST|String, global, local);
 // wherein we reuse the global scope frame and put require and
exports in a scope right above it

In both of these cases, Mark's comments about copying members to a
scope frame instead of using the object itself might apply.  I presume
that this is to avoid following the prototype chain when resolving a
variable.  If globals are shallowly copied into the local/bottom scope
of the module, some things get simpler.  For one, we can freeze the
global object without freezing the bottom of the scope chain, which
would impair module local declarations.  We also wouldn't need two
scopes initially.  However, it would be nominally slower.  I think
that execute should take two arguments either way, since it would be
inconvenient and slower than necessary to do this for every module
execution:

 var scope = copy(global);
 scope.require = Require(id);
 scope.exports = memo[exports] = {};
 execute(ast, scope);
  // scope chain is [Frame(scope)]

…since execute effectively hides an implicit copy to the scope frame,
making the explicit copy superfluous, as opposed to:

 var local = {
   require: Require(id),
   exports: memo[exports] = {}
 };
 execute(ast, global, local);
  // scope chain is [Frame(global), Frame(local)]

Is this on the right line of reasoning?

Kris Kowal
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Eugene Lazutkin]

I think JsonML (with []) is perfect:

- it doesn't introduce new concepts like the other proposal
- working with it doesn't require new APIs
- it is simple but not more restrictive than the other proposal
- we can add new "tokens" if we need to without introducing new objects/APIs

Given these positives and the lack of notable negatives I give +1 for
JsonML. If we need more user-friendly facade we can always implement it
with a library that uses JsonML-based facilities under the hood.

Eugene Lazutkin
Dojo Toolkit, Committer
http://lazutkin.com/

On 05/12/2009 02:31 AM, Brendan Eich wrote:

> On May 12, 2009, at 12:24 AM, kevin curtis wrote:
>
>> JsonML looks good for for an AST handling:
>>
>> ["||",
>>   ["||",
>>     ["Id", "X"],
>>     ["Id", "Y"]],
>>   ["Id", "Z"]]
>
> Yes.
>
>
>> Maybe the 'canonical' AST serialized string format could actually be
>> more scheme-y:
>>
>> (or (or X Y) Z)
>>
>> JsonML could be used for building pure js in-memory AST graphs which
>> could then be easily stringified to the 'canonical' format.
>
> JsonML wouldn't be used to build object graphs -- the JSON decoder would
> do that given JsonML in a string, from the AST encoder. That's the point
> I made in the words you bottom-cite about not mandating a big fat object
> graph if the use-case doesn't need the graph, just the string containing
> the AST serialization.
>
>
>> The benefit is that a scheme-y format could help the thinking on the
>> semantics for es6/harmony.
>
> That seems like no benefit in memory use or cycles, only in thinking. If
> you squint, don't the []s turn into ()s? :-P
>
>
>> (Downside compared to a JSON  canonical format is that with JSON the
>> parsing/stringifying is free via the JSON api in es5).
>
> This is a big downside.
>
>
>> For convenience JSON could remain JSON in this scheme-y format:
>> var x = [1,4,5]
>> becomes:
>> (var x [1,4,5])
>
> I don't see why we'd invent a third language.
>
> /be

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Kris Kowal-2]

>> kevin curtis wrote:
>>> Is a 'canonical' AST part of the plans for ecmascript 6/harmony.

> On May 9, 2009, at 9:19 AM, David-Sarah Hopwood wrote:
>> I hope so; that would be extremely useful. I would like to see an
>> ECMAScript source -> AST parser (as well as an AST evaluator) in the
>> Harmony standard library.

On Sat, May 9, 2009 at 11:57 AM, Brendan Eich <brendan@...> wrote:
> We've wanted this since early in ES4 days. It would help many projects and
> experimental extensions (type checkers, template systems, macro processors,
> etc.) to have a standard AST, which could be serialized to JSON.

Other neat uses for the AST would potentially include comment scraping
for automated documentation tools and minification, which have their
own requirements beyond those for execution, optimization, and static
analysis.

Upon further reflection, I'm not sure that parse(program:String):AST
would serve the purpose of fast sandboxing.  The intent of splitting
parse and execute is to reduce the cost of execution, so that modules
can be reused in many small sandboxes.  Having parse produce a
(mutable) AST, and then leaving execute to translate the AST for the
interpreter might constrain our options for producing the desired
performance.  It might be better to have a compile() routine that
returns an opaque, engine-specific Program object that can in turn be
executed multiple times.

Kris Kowal
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[Brendan Eich-3]

On May 21, 2009, at 4:15 PM, Kris Kowal wrote:

> Upon further reflection, I'm not sure that parse(program:String):AST
> would serve the purpose of fast sandboxing.  The intent of splitting
> parse and execute is to reduce the cost of execution, so that modules
> can be reused in many small sandboxes.

Premature optimization?

Implementations can do ccache(1)-style compiled-results-caching.


> Having parse produce a
> (mutable) AST, and then leaving execute to translate the AST for the
> interpreter might constrain our options for producing the desired
> performance.  It might be better to have a compile() routine that
> returns an opaque, engine-specific Program object that can in turn be
> executed multiple times.

SpiderMonkey used to have a Script object you could create via s = new  
Script(source), or s = Script.compile(source) -- then you'd s.exec()  
to run the script. It was eval split in two. That made it pure evil,  
since the implementation couldn't analyze for it easily to deoptimize  
the calling code so that its scope could be used. We had code to  
deoptimize at runtime, but that was many years ago, when we didn't  
optimize much.

I don't think we want any such primitives split from eval's guts,  
although if we were to have these, the eval-as-container idea Mark  
started and you ran with does have appeal, since eval is already a  
quasi-keyword that implementations must recognize: eval.
{parse,compile,exec}.

Anyway, AST serialization is what I'm working on (back burner this  
week). Beyond that is a cliff....

/be
_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss

[kevin curtis]

I initially thought that:
parse(program:String):AST

would return a handle to the native c/c++ AST tree.Then the question
was if this native AST tree should be manipulatable by javascript ie
should there be a wrapper javascript api which updates the native
tree. Python says no - its native asdl c AST tree is not manipulatable
by Python. Rather a copy is made into pure Python objects which can be
manipulated and then converted back to a new native adl ast. So the
native AST is immutable.

I was thinking along the lines of:

var astObj = parse(js_source) // returns a handle to a immutable
native c/c++ AST object
var jsonML = astObj.toJson() // the ast object has a single method -
toJson() which returns a jsonML string

//the jsonML is manipulated either as as a string or via JSON.parse
var jsonAstObj = JSON.parse(jsonML)
.... processing
var jsonML2 = JSON.stringify(jsonASTObj)

var ast2 = parseJson(jsonML2) // create a handle to new immutable
native ast object
execute(ast2)



This would cover the 2 use cases of:
- efficient processing re processing the AST when no javascript
manipulation of the AST is required, as in fast sandboxing.  ie no use
of ast.toJson and parseJson methods.
- manipulating the AST in javascript for templating etc via jsonML

There could be issues re resource usage/garbage collection of native
AST object. Maybe the ast object could have a method - release().


On Fri, May 22, 2009 at 12:15 AM, Kris Kowal <kris.kowal@...> wrote:

>>> kevin curtis wrote:
>>>> Is a 'canonical' AST part of the plans for ecmascript 6/harmony.
>
>> On May 9, 2009, at 9:19 AM, David-Sarah Hopwood wrote:
>>> I hope so; that would be extremely useful. I would like to see an
>>> ECMAScript source -> AST parser (as well as an AST evaluator) in the
>>> Harmony standard library.
>
> On Sat, May 9, 2009 at 11:57 AM, Brendan Eich <brendan@...> wrote:
>> We've wanted this since early in ES4 days. It would help many projects and
>> experimental extensions (type checkers, template systems, macro processors,
>> etc.) to have a standard AST, which could be serialized to JSON.
>
> Other neat uses for the AST would potentially include comment scraping
> for automated documentation tools and minification, which have their
> own requirements beyond those for execution, optimization, and static
> analysis.
>
> Upon further reflection, I'm not sure that parse(program:String):AST
> would serve the purpose of fast sandboxing.  The intent of splitting
> parse and execute is to reduce the cost of execution, so that modules
> can be reused in many small sandboxes.  Having parse produce a
> (mutable) AST, and then leaving execute to translate the AST for the
> interpreter might constrain our options for producing the desired
> performance.  It might be better to have a compile() routine that
> returns an opaque, engine-specific Program object that can in turn be
> executed multiple times.
>
> Kris Kowal
> _______________________________________________
> es-discuss mailing list
> es-discuss@...
> https://mail.mozilla.org/listinfo/es-discuss
>

_______________________________________________
es-discuss mailing list
es-discuss@...
https://mail.mozilla.org/listinfo/es-discuss