ECMAScript Harmony

[Brendan Eich-2]

It's no secret that the JavaScript standards body, Ecma's Technical  
Committee 39, has been split for over a year, with some members  
favoring ES4, a major fourth edition to ECMA-262, and others  
advocating ES3.1 based on the existing ECMA-262 Edition 3 (ES3)  
specification. Now, I'm happy to report, the split is over.

The Ecma TC39 meeting in Oslo at the end of July was very productive,  
and if we keep working together, it will be seen as seminal when we  
look back in a couple of years. Before this meeting, I worked with  
John Neumann, TC39 chair, and ES3.1 and ES4 principals, especially  
Lars Hansen (Adobe), Mark Miller (Google), and Allen Wirfs-Brock  
(Microsoft), to unify the committee around shared values and a common  
roadmap. This message is my attempt to announce the main result of  
the meeting, which I've labeled "Harmony".

Executive Summary

The committee has resolved in favor of these tasks and conclusions:

1. Focus work on ES3.1 with full collaboration of all parties, and  
target two interoperable implementations by early next year.
2. Collaborate on the next step beyond ES3.1, which will include  
syntactic extensions but which will be more modest than ES4 in both  
semantic and syntactic innovation.
3. Some ES4 proposals have been deemed unsound for the Web, and are  
off the table for good: packages, namespaces and early binding. This  
conclusion is key to Harmony.
4. Other goals and ideas from ES4 are being rephrased to keep  
consensus in the committee; these include a notion of classes based  
on existing ES3 concepts combined with proposed ES3.1 extensions.

Detailed Statement

A split committee is good for no one and nothing, least of all any  
language specs that might come out of it. Harmony was my proposal  
based on this premise, but it also required (at least on the part of  
key ES4 folks) intentionally dropping namespaces.

This is good news for everyone, both those who favor smaller changes  
to the language and those who advocate ongoing evolution that  
requires new syntax if not new semantics. It does mean that some of  
the ideas going back to the first ES4 proposals in 1999, implemented  
variously in JScript.NET and ActionScript, won't make it into any ES  
standard. But the benefit is collaboration on unified successor  
specifications to follow ES3, starting with ES3.1 and continuing  
after it with larger changes and improved specification techniques.

One of the use-cases for namespaces in ES4 was early binding (use  
namespace intrinsic), both for performance and for programmer  
comprehension -- no chance of runtime name binding disagreeing with  
any earlier binding. But early binding in any dynamic code loading  
scenario like the web requires a prioritization or reservation  
mechanism to avoid early versus late binding conflicts.

Plus, as some JS implementors have noted with concern, multiple open  
namespaces impose runtime cost unless an implementation works  
significantly harder.

For these reasons, namespaces and early binding (like packages before  
them, this past April) must go. This is final, they are not even a  
future possibility. To achieve harmony, we have to focus not only on  
nearer term improvements -- on "what's in" or what could be in --  we  
must also strive to agree on what's out.

Once namespaces and early binding are out, classes can desugar to  
lambda-coding + Object.freeze and friends from ES3.1. There's no need  
for new runtime semantics to model what we talked about in Oslo as a  
harmonized class proposal (I will publish wiki pages shortly to show  
what was discussed).

We talked about desugaring classes in some detail in Oslo. During  
these exchanges, we discussed several separable issues, including  
classes, inheritance, like patterns, and type annotations. I'll avoid  
writing more here, except to note that there were clear axes of  
disagreement and agreement, grounds for hope that the committee could  
reach consensus on some of these ideas, and general preference for  
starting with the simplest proposals and keeping consensus as we go.

We may add runtime helpers if lambda-coding is too obscure for the  
main audience of the spec, namely implementors who aim to achieve  
interoperation, but who may not be lambda-coding gurus. But we will  
try to avoid extending the runtime semantic model of the 3.1 spec, as  
a discipline to guard against complexity.

One possible semantic addition to fill a notorious gap in the  
language, which I sketched with able help from Mark Miller: a way to  
generate new Name objects that do not equate as property identifiers  
to any string. I also showed some sugar, but that is secondary at  
this point. Many were in favor of this new Name object idea.

There remain challenges, in particular getting off of the untestable  
and increasingly unwieldy ES1-3.x spec formalism. I heard some  
generally agree, and no one demur, about the ES4 approach of using an  
SML + self-hosted built-ins reference implementation (RI).

We are going to look into stripping the RI of namespaces and early  
binding (which it uses to ensure normative self-hosted behavior, not  
susceptible to "user code" modifying the meaning of built-ins),  
simplifying it to implement ES3.1plus or minus (self-hosted built-ins  
may require a bit more magic). More on that effort soon.

ES3.1 standardizes getters and setters that were first implemented at  
Mozilla and copied by Apple and Opera. More such de-facto  
standardization is on the table for a successor edition in the  
harmonized committee.

I heard good agreement on low-hanging "de-facto standard" fruit,  
particularly let as the new var, to match block-scoped const as still  
proposed (IIRC) in 3.1. Also some favorable comments about simple  
desugarings such as expression closures and destructuring assignment,  
and other changes in JS1.7 and 1.8 that do not require new runtime  
semantic models.

Obviously, these require new syntax, which is appropriate for a major  
post-3.1 "ES-harmony" edition. Syntax is user interface, there's no  
reason to avoid improving it. What's more, the intersection semantics  
of extended ES3 implementations conflict and choke off backward-
compatible *semantics* for syntax that may even parse in all top four  
browsers (e.g., functions in blocks).

Both the appropriateness of new syntax, and the need to make  
incompatible (with ES3 extensions) semantic changes, motivate opt-in  
versioning of harmonized successor edition. I believe that past  
concerns about opt-in versioning requiring server file suffix to MIME  
type mapping maintenance were assuaged (browsers in practice, and  
HTML5 + RFC 4329, do not consider server-sent Content-Type -- the web  
page author can write version parameters directly in script tag type  
attributes).

Some expressed interest in an in-language pragma to select version;  
this would require immediate version change during parsing. It's a  
topic for future discussions.

The main point, as important as cutting namespaces in my view, is  
that the committee has a vision for extending the language  
syntactically, not trying to fit new semantics entirely within some  
combination of existing "three of four top browsers" syntax and  
standard library extensions.

As Waldemar Horwat (Google) said on the final day, the meeting was  
seminal, and one of the most productive in a long while. Much work  
remains on 3.1 and Harmony, but we are now on a good footing to make  
progress as a single committee.

/be

_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Brendan Eich-2]

In light of Harmony, and the recurrent over- and under-cross-posting,  
I'd like to merge the es3.x-discuss@... and es4-
discuss@... lists into es-discuss@.... The old  
archives will remain available via the web, and the old aliases will  
forward to the new list. Any objections?

/be
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Douglas Crockford]

Brendan Eich wrote:
> In light of Harmony, and the recurrent over- and under-cross-posting,  
> I'd like to merge the es3.x-discuss@... and es4-
> discuss@... lists into es-discuss@.... The old  
> archives will remain available via the web, and the old aliases will  
> forward to the new list. Any objections?

Yes. I'd like to keep ES3.x to discuss ES3.1 business until we get that finalized.
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Brendan Eich-2]

On Aug 13, 2008, at 2:36 PM, Douglas Crockford wrote:

> Brendan Eich wrote:
>> In light of Harmony, and the recurrent over- and under-cross-posting,
>> I'd like to merge the es3.x-discuss@... and es4-
>> discuss@... lists into es-discuss@.... The old
>> archives will remain available via the web, and the old aliases will
>> forward to the new list. Any objections?
>
> Yes. I'd like to keep ES3.x to discuss ES3.1 business until we get  
> that finalized.


I think Allen Wirfs-Brock was in favor of merging the lists, but we  
could just rename es4-discuss to es-discuss. Is it worth dropping  
that 4?

/be
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[P T Withington]

[Trimmed recipients]

On 2008-08-13, at 17:26EDT, Brendan Eich wrote:

> One possible semantic addition to fill a notorious gap in the
> language, which I sketched with able help from Mark Miller: a way to
> generate new Name objects that do not equate as property identifiers
> to any string. I also showed some sugar, but that is secondary at
> this point. Many were in favor of this new Name object idea.

Is this "Name object" what we old Lispers would call a SYMBOL?
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Brendan Eich-2]

On Aug 13, 2008, at 2:50 PM, P T Withington wrote:

> [Trimmed recipients]
>
> On 2008-08-13, at 17:26EDT, Brendan Eich wrote:
>
>> One possible semantic addition to fill a notorious gap in the
>> language, which I sketched with able help from Mark Miller: a way to
>> generate new Name objects that do not equate as property identifiers
>> to any string. I also showed some sugar, but that is secondary at
>> this point. Many were in favor of this new Name object idea.
>
> Is this "Name object" what we old Lispers would call a SYMBOL?

I said "gensym" in Oslo and had it in a draft, but kids today do not  
appreciate how l33t it is.

The inspiration came from several sources, and one idea for sugar  
came from PLT-Scheme's define-local-member-name.

/be
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Allen Wirfs-Brock-2]

> I think Allen Wirfs-Brock was in favor of merging the lists, but we
> could just rename es4-discuss to es-discuss. Is it worth dropping
> that 4?
>

I think that "es-discuss" is a lot more "future proof" and now is as good a time to change as any.

As I've already mentioned to Brendan, I think it may be useful in the long term to separate detailed discussion of the "next" revision from longer term or speculative discussions.  However, for now, we need to rebuild a single community and most of the ES3.x-discuss items are getting (or should be) cross posted to es4-discuss anyway. Given that, I think we should just merge the lists and if we find that 3.1 and post 3.1 items are getting intermingled in a confusing manner we can always split it out again.  I suppose one way to preserve that option is for everyone to just stop posting to ES3.x-discuss and only post the new es-discuss.

Allen
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Maciej Stachowiak]

On Aug 13, 2008, at 2:30 PM, Brendan Eich wrote:

> In light of Harmony, and the recurrent over- and under-cross-posting,
> I'd like to merge the es3.x-discuss@... and es4-
> discuss@... lists into es-discuss@.... The old
> archives will remain available via the web, and the old aliases will
> forward to the new list. Any objections?

I support this move.

  - Maciej

_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Peter Michaux]

On Wed, Aug 13, 2008 at 2:26 PM, Brendan Eich <brendan@...> wrote:

[snip]

> We talked about desugaring classes in some detail in Oslo. During
> these exchanges, we discussed several separable issues, including
> classes, inheritance, like patterns, and type annotations. I'll avoid
> writing more here,

Is there more to read elsewhere? I'd like to know concretely what
"desugaring classes" means.

[snip]

Thanks,
Peter
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Kris Zyp-4]

>> We talked about desugaring classes in some detail in Oslo. During
>> these exchanges, we discussed several separable issues, including
>> classes, inheritance, like patterns, and type annotations. I'll avoid
>> writing more here,
>
> Is there more to read elsewhere? I'd like to know concretely what
> "desugaring classes" means.

I would too, I assume this is based on Mark's proposal for Classes as Sugar
(http://www.mail-archive.com/es4-discuss@.../msg01624.html).
However, Mark's proposal seems pretty vague in the area of typing, so I am
curious how this would work as well. Would there be new
functionality/semantics for defining types on properties (via defineProperty
in ES3.1, sugared in ES4) with some type of object structures like I
suggested in
https://mail.mozilla.org/pipermail/es4-discuss/2008-July/003321.html?

Kris

_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Peter Michaux]

On Wed, Aug 13, 2008 at 8:26 PM, Kris Zyp <kris@...> wrote:

>>> We talked about desugaring classes in some detail in Oslo. During
>>> these exchanges, we discussed several separable issues, including
>>> classes, inheritance, like patterns, and type annotations. I'll avoid
>>> writing more here,
>>
>> Is there more to read elsewhere? I'd like to know concretely what
>> "desugaring classes" means.
>
> I would too, I assume this is based on Mark's proposal for Classes as Sugar
> (http://www.mail-archive.com/es4-discuss@.../msg01624.html).

Thanks for the link.

Mark's proposal looks like something that could perhaps be handled
with macros. What I've thought would be a great way to add classes to
ECMAScript is to give ECMAScript macros and then let folks build their
own class macro libraries. That is essentially what has happened now
but the wrapping of the prototype-based system and the lack of macros
has lead to "ugly" code. Folks are going to continue with their own
wrapping of the built-in prototype system anyway even if there is one
sugar wrapping included in the language. Different situations require
different OOP features. Some situations more complex than others. For
example, if the built-in sugar doesn't have mixins and someone needs
mixins then they will need to wrap exactly the same prototype system
that Mark's proposal wraps.

This brings up an interesting post from what is getting close to a year ago.

On Nov 12, 2007 Brendan Eich wrote:

> On Nov 12, 2007 YR Chen wrote:
>
> > Macros have been proposed and discussed before - they're somewhere
> > on the wiki. AFAIK, they've been deferred to a future ES5.
>
> Researchers on the list may be willing to speak to macros with more
> authority than I have. I'll just say that apart from the fun with C-
> like syntax (which can be handled; we're close to specifying standard
> ASTs for ES4), sound hygiene theory is still a research topic. I
> expect good results from academia in a year or so, and hope that TG1
> will take advantage of them.
>
> In the mean time, I think it would be very wrong to defer syntactic
> conveniences until after macros are done. With macros, conveniences
> that de-sugar in ES4 could be re-specified (and implemented at the
> same time) cheaply, for sure -- but users deserve convenient syntax
> sooner than "after macros".

Given the time lines, will hygienic macros now be ready before or in
time for ES4?

Perhaps classes don't need a hygienic macro system and hygienic macros
can come later.

[snip]

Peter
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Garrett]

On Wed, Aug 13, 2008 at 2:26 PM, Brendan Eich <brendan@...> wrote:
> It's no secret that the JavaScript standards body, Ecma's Technical
> Committee 39, has been split for over a year, with some members
> favoring ES4, a major fourth edition to ECMA-262, and others
> advocating ES3.1 based on the existing ECMA-262 Edition 3 (ES3)
> specification. Now, I'm happy to report, the split is over.
>

That is good to know.

> A split committee is good for no one and nothing,

It led to contrasting ideas.

>
> There remain challenges, in particular getting off of the untestable
> and increasingly unwieldy ES1-3.x spec formalism.

Unit tests:
 * help developer understanding
 * provide examples for discussion
 * provide conformance suite for implementors
 * define boundaries on what is and what is not supported*

The tests could use a custom test runner or use an existing one. It
would be useful if the testrunner could be run in a browser.

It would be helpful to have code comments in the tests, indicating
exactly which rules of the spec are being tested, so that the reader
can quickly go to that section of the spec (spec in HTML).

Real code examples in the spec would be useful, too. I can easily see
a bookmarklet used to capture and execute the example selection of the
spec.



* defining "not supported" could be as simple as having an assertion
test that should be ignored, and should be marked ignore, along with a
comment as to why the test is ignored (use of Host object, et c).
Ignored tests can be a useful form of documentation.

>
> As Waldemar Horwat (Google) said on the final day, the meeting was
> seminal,

I like it.

Garrett

>
> /be
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss

[Peter Michaux]

On Fri, Aug 15, 2008 at 1:40 PM, Garrett Smith <dhtmlkitchen@...> wrote:

[snip]

>> There remain challenges, in particular getting off of the untestable
>> and increasingly unwieldy ES1-3.x spec formalism.
>
> Unit tests:
>  * help developer understanding
>  * provide examples for discussion
>  * provide conformance suite for implementors
>  * define boundaries on what is and what is not supported*
>
> The tests could use a custom test runner or use an existing one. It
> would be useful if the testrunner could be run in a browser.
>
> It would be helpful to have code comments in the tests, indicating
> exactly which rules of the spec are being tested, so that the reader
> can quickly go to that section of the spec (spec in HTML).
>
> Real code examples in the spec would be useful, too. I can easily see
> a bookmarklet used to capture and execute the example selection of the
> spec.

This seems like "literate programming" to me where the program is a
set of unit tests.

[snip]

Peter
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss