Generator use-cases

This is a long-ish case for including generators in ES4 as proposed.  
I offered, to several Ecma colleagues, to mail pointers to examples  
of how useful the Python-inspired generators in JS1.7 and JS1.8 in  
Firefox, and proposed for inclusion in the ES4 standard, are in real-
world code. But I figured that folks on es4-discuss@... might  
like to see these few examples too.

I ported Peter Norvig's Sudoku solver:

http://norvig.com/sudoku.html

from Python to JS1.8:

https://bugzilla.mozilla.org/attachment.cgi?id=266577

This code uses not only generators and array comprehensions (e.g. the  
cross function), but also generator expressions -- which are sugar  
for generator functions immediately applied, and therefore lazy,  
unlike array comprehensions. This laziness is important to avoid  
using exponential amounts of memory.

My Mozilla colleague Igor Bukanov rewrote this code in more  
straightforward JS-functional-programming style in JS1.8 (so using  
expression closures, e.g. function add(x,y) x + y; but not  
generators) -- see here:

https://bugzilla.mozilla.org/attachment.cgi?id=266938

Opinions vary on which version is better, but the generator-based one  
is significantly shorter, and also faster in SpiderMonkey. And the  
main thing is that it lets the code focus on the essentials of the  
search algorithm and minimize the bookkeeping, which Peter's Python  
code did very well (Python has lighter syntax, unburdened by the C  
heritage, but JS can't disown curly braces and parens; other than  
that the JS and Python versions are close).

Another example is a static analysis script (one of many by Dave  
Mandelin) for Mozilla's "TreeHydra" GCC plugin (developed by Taras  
Glek and Dave):

https://bugzilla.mozilla.org/attachment.cgi?id=311698

Notice the yield usage, and also the array comprehensions returned,  
e.g., by rectify_attribute_args.

Rewriting these to use iterators, or ES3-style functional  
programming, adds a lot of source boilerplate that again obscures the  
essential code, and tends to perform not as well to boot. To pick one  
example, here is rewrite from the flatten_chain generator:

function flatten_chain(chain_head) {
   for (let o = chain_head; o; o = TREE_CHAIN(o)) {
     yield o;
   }
}

to this roughly equivalent iterator coded using pure ES3:

function flatten_chain(chain_head) {
   return {
     next: function () {
       var o = chain_head;
       if (o) {
         chain_head = TREE_CHAIN(chain_head);
         return o;
       }
       throw StopIteration;
     }
   }
}

Note that this rewrite loses integrity since next is mutable, which  
ES3 can't control (the ES4 methods, also in JS1.7 and higher versions  
in Firefox 2-3, are DontDelete and ReadOnly). And of course it leaves  
out the rest of the generator suite, send/throw/close, which come for  
free in the ES4 Generator class instantiated by a call to a function  
containing yield.

Here is the expansion of that array-comprehension-returning  
flatten_chain caller, rectify_attribute_args, that I mentioned above,  
from:

function rectify_attribute_args(tree) {
   return [ TREE_STRING_POINTER1(TREE_VALUE(a)) for (a in  
flatten_chain(tree)) ];
}

to this ES3 code:

function rectify_attribute_args(tree) {
   var r = [];
   var i = flatten_chain(tree);
   for (;;) {
     var a;
     try {
       a = i.next();
     } catch (e if e instanceof StopIteration) {
       break;
     }
     r.push(TREE_STRING_POINTER1(TREE_VALUE(a)));
   }
   return r;
}

Again I omitted the finally clause to call i.close and other bits of  
the general generator mechanism. Of course one could specialize the  
termination technique and other details to re-optimize, but why  
should this be necessary?

The Python-based syntax is subject to criticism for changing the  
meaning of a function once yield is used in the body of the function,  
but we are hitching wagons to Python and reusing community brain-
print and design experience (also giving feedback to simplify future  
versions of Python based on our experience, specifically by  
eliminating the GeneratorExit exception).

And as far as I know from the experience in Firefox 2 and 3, we've  
had no problems with the potential confusion caused by this extension  
to function syntax -- it has been painless.

The ES4 iteration protocol is proposed here:

http://wiki.ecmascript.org/doku.php?
id=proposals:iterators_and_generators

The iteration protocol underlies for-in and for-each-in constructs in  
loops and comprehensions. It hides iterator-specific implementation  
details such as StopIteration, while providing uniform looping syntax  
that can be customized to improve (I would say restore) the utility  
of the built-in JS for-in syntax.

Generators, besides supporting one level of coroutine suspending and  
(re-)calling, are the cheapest way to implement an iterator. Unlike  
general coroutines, they do not break functional abstraction by  
jumping over multiple (possibly native) stack frames, which would  
have to be saved, and then restoring the saved stack frames later.  
They're simple, for better and for worse.

In my opinion, and to borrow from Tallyrand (or whoever said it  
first), if ES4 has iterators but not generators, it would be  
something worse than a crime -- it would be a mistake.

/be
_______________________________________________
Es4-discuss mailing list
Es4-discuss@...
https://mail.mozilla.org/listinfo/es4-discuss