Nabble - E Lang

Re: [Caja] Functional auditor for Cajita: EQ vs referential transparency

2009-12-13T18:37:14Z

David-Sarah Hopwood wrote:
> 1. Prevent pure [Cajita] code from using identity-revealing operations.
> This includes both primitives like === and !==,

I should also have mentioned == and != here.

> and APIs such as
> cajita.identical, Array.prototype.indexOf/lastIndexOf or the
> proposed EphemeronTable -- these would simply not be whitelisted
> as deep-frozen.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita: EQ vs referential transparency

2009-12-13T18:24:41Z

David-Sarah Hopwood wrote:

> David Wagner wrote:
>> David-Sarah Hopwood wrote:
>>> In fact I'm wrong [in claiming that @pure implies referentially transparent
>>> in my original proposal], as shown by this example:
>>>
>>> /*@pure*/ function f() { return cajita.deepFreeze({}); }
>>>
>>> const a = f();
>>> const b = f();
>>>
>>> /*@pure*/ function g(x) { return x === a; }
>>>
>>> g(a); // true
>>> g(b); // false
>>>
>>> Since a and b are observably different, either f or g must not
>>> be referentially transparent.
>>
>> I'm not sure if I follow this example. Can you elaborate
>> what it illustrates?
>>
>> Here's my understanding of the bit I think you may be getting at:
>> conceptually, a referentially transparent function should satisfy the
>> following property: whenever x=y, then f(x)=f(y). However, to fill out this
>> definition, we must define what we mean by "=": what form of equality we
>> have in mind.
>
> Yes. But whatever we mean by "=", it's clear that we can't reasonably
> have true = false. In the above example, we can apply referential
> transparency once to conclude a = b (since the calls to f() have no
> arguments -- or they could have been passed the same argument), and
> again to conclude g(a) = g(b). Therefore either true = false, or
> one of f or g is not referentially transparent. But both were declared
> as @pure, therefore @pure does not imply referentially transparent for
> any definition of = satisfying not(true = false).

It appears that the Functional auditor in the design described in
<http://www.erights.org/elang/kernel/auditors/> suffers from essentially
the same issue.

Consider the E equivalent of the above example (note that 'def obj {}' has
no mutable state):

def f() :DeepFrozen { return def obj {} }
f :Functional # passes

def a := f()
def b := f()

def g(x) :boolean { return x == a }
g :Functional # passes

g(a) # true
g(b) # false

This is because each 'obj' returned by f is DeepFrozen but not DeepSelfless
(see Figure 3 of <http://www.erights.org/elang/kernel/auditors/>). If a
Functional object returns an object that is not DeepSelfless, then it fails
to be referentially transparent.

The auditors page doesn't explicitly claim that the Functional guard is
supposed to provide referential transparency. However, it does say that

# if we know that an object is functional, we can cache its output,
# reorder calls, or skip repeated calls ...

Memoization (cacheing) depends on referential transparency.

The problem with memoizing functions that don't return a DeepSelfless
value has been noted before in this thread started by Kevin Reid:
<http://www.eros-os.org/pipermail/e-lang/2006-May/011206.html>

Kevin first suggested requiring the return of a memoized function to
be guarded by DeepPassByCopy (which implies DeepSelfless). In the
specific case of memoization -- not necessarily for other uses of
referential transparency -- there are workarounds that are less
restrictive than this. For example MarkM gave a solution that involved
only adding a non-DeepPassByCopy result to the memo table when it is
seen twice: <http://www.eros-os.org/pipermail/e-lang/2006-May/011207.html>

Alternatively, for some applications it may be possible to use a
different form of cache, call it an "interning memoizer", as described
in <http://www.eros-os.org/pipermail/e-lang/2006-May/011270.html> and
<http://www.eros-os.org/pipermail/e-lang/2006-May/011282.html>.

However, those approaches are specific to memoization. If we want
to avoid the problem more generally, then we have roughly three
options:

1. Prevent pure code from using identity-revealing operations. This
includes both primitives like === and !==, and APIs such as
cajita.identical, Array.prototype.indexOf/lastIndexOf or the
proposed EphemeronTable -- these would simply not be whitelisted
as deep-frozen.

Note that impure code can still observe that a pure function has
returned different objects on different calls. But it is reasonable
to ask impure code not to rely on that -- whereas if pure code itself
could make such an observation, it would defeat the point of auditing
as a method of determining properties of code without having to
review it.

This introduces the problem that pure code no longer has access
to *any* efficient equality predicate. Below is an equality test that
is safe to give to pure code; in the cases where egal would do
structural comparison, it sticks its head in the sand and returns
undefined. (This at least has the advantage of bounding the time
for comparison to be proportional to the number of own properties
in any tested object. It would be O(1) if not for the calls to
Object.isFrozen, and an ES5 implementation *should* cache whether
objects are frozen.)

/**
* If both x and y are frozen objects, return undefined.
* Otherwise give the same result as cajita.identical(x, y).
*/
function ostrichEq(x, y) {
if (x === y) {
if (x === Object(x)) {
// x and y are the same object. If that object is frozen, return
// undefined.
return Object.isFrozen(x) ? undefined : true;
}
// Usually true, but might be a false positive for 0 and -0.
return x !== 0 || 1/x === 1/y;
}
if (x === Object(x)) {
if (y === Object(y) && Object.isFrozen(x) && Object.isFrozen(y)) {
// x and y are different objects, but both are frozen.
return undefined;
}
return false;
}
// Usually false, but might be a false negative for NaN and NaN.
return x !== x && y !== y;
}

2. Force all DeepFrozen objects to be DeepSelfless. Note that if
functions can be DeepFrozen, then this is equivalent to using
the same comparison for function instances as the original egal
(see the definition of 'egal-function' on page 8 of Baker's paper
<http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.23.9999>),
which compares function identities and lexical environments.

However, this may weaken encapsulation for DeepFrozen objects that
wouldn't otherwise have been DeepSelfless.

3. [For E] Define a Pure auditor that is like Functional but also
requires all returns to be guarded by DeepSelfless or DeepPassByCopy.

[For Cajita] Introduce a deep-selfless auditor, and strengthen @pure
so that a rewritten @pure function checks that its return value is
deep-selfless.

> Forbidding use of === and !== in a deep-frozen function solves the
> problem.

This is option 1 above.

> It's similar to Joe-E restricting on == and != to non-value
> types, but we can't do that in Cajita because it is untyped.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-13T00:46:40Z

David-Sarah Hopwood wrote:
> (Nor can we usefully weaken referential transparency to use different
> versions of equality on the left and right of the implication, since
> then it could not be applied compositionally.)

I should have said: we cannot usefully weaken referential transparency
to use a coarser version of equality on the right than on the left.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-12T23:34:28Z

David Wagner wrote:

> David-Sarah Hopwood wrote:
>> In fact I'm wrong here, as shown by this example:
>>
>> /*@pure*/ function f() { return cajita.deepFreeze({}); }
>>
>> const a = f();
>> const b = f();
>>
>> /*@pure*/ function g(x) { return x === a; }
>>
>> g(a); // true
>> g(b); // false
>>
>> Since a and b are observably different, either f or g must not
>> be referentially transparent.
>
> I'm not sure if I follow this example. Can you elaborate
> what it illustrates?
>
> Here's my understanding of the bit I think you may be getting at:
> conceptually, a referentially transparent function should satisfy the
> following property: whenever x=y, then f(x)=f(y). However, to fill out this
> definition, we must define what we mean by "=": what form of equality we
> have in mind.

Yes. But whatever we mean by "=", it's clear that we can't reasonably
have true = false. In the above example, we can apply referential
transparency once to conclude a = b (since the calls to f() have no
arguments -- or they could have been passed the same argument), and
again to conclude g(a) = g(b). Therefore either true = false, or
one of f or g is not referentially transparent. But both were declared
as @pure, therefore @pure does not imply referentially transparent for
any definition of = satisfying not(true = false).

(Nor can we usefully weaken referential transparency to use different
versions of equality on the left and right of the implication, since
then it could not be applied compositionally.)

Forbidding use of === and !== in a deep-frozen function solves the
problem. It's similar to Joe-E restricting on == and != to non-value
types, but we can't do that in Cajita because it is untyped.

If I'm not mistaken, Joe-E *could* also relax its restriction on ==
and !=, and possibly its restrictions on calling nondeterministic APIs,
by only applying them to (I think) methods of Immutable objects.

> I posted earlier about this:
> My post (see the "P.P.S.")
> http://www.eros-os.org/pipermail/e-lang/2009-December/013327.html
> Ihab's response:
> http://www.eros-os.org/pipermail/e-lang/2009-December/013335.html
> The response was that in this context, equality is given by ===.

Ihab only said that Cajita currently exposes ===. That doesn't imply
that = is ===. In my proposal, = is Henry Baker's 'egal'. (I should
write out a definition of egal in Cajita to make this clearer.)

> So that means that we only get guarantees about two invocations to
> g if the arguments in both invocations are equal according to "===";
> otherwise, there are no promises.
>
> You give an example of two invocation to g, namely g(a) and g(b),
> where the arguments in both arguments are not equal according to
> "===", namely, a !== b. So you shouldn't expect the results of those
> two invocations to be related at all; the two results might be
> entirely unrelated.
>
> Does this make sense? Have I missed something?

I think you missed that referential transparency was being applied
twice (probably my fault for not spelling out the argument in more
detail).

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-12T16:51:57Z

David-Sarah Hopwood wrote:

>In fact I'm wrong here, as shown by this example:
>
> /*@pure*/ function f() { return cajita.deepFreeze({}); }
>
> const a = f();
> const b = f();
>
> /*@pure*/ function g(x) { return x === a; }
>
> g(a); // true
> g(b); // false
>
>Since a and b are observably different, either f or g must not
>be referentially transparent.

I'm not sure if I follow this example. Can you elaborate
what it illustrates?

Here's my understanding of the bit I think you may be getting at:
conceptually, a referentially transparent function should satisfy the
following property: whenever x=y, then f(x)=f(y). However, to fill out this
definition, we must define what we mean by "=": what form of equality we
have in mind. The definition of referential transparency is parametrized
by your definition of equality, and to provide a full definition you
must decide what you mean by "=". See Section 3.2 and follow-on
discussion in the Joe-E purity paper (Finifter et al 2008).

Note that we only get guarantees about the result of the function
in case when the arguments in both arguments are equal according
to "=". Otherwise, all bets are off.

I posted earlier about this:
My post (see the "P.P.S.")
http://www.eros-os.org/pipermail/e-lang/2009-December/013327.html
Ihab's response:
http://www.eros-os.org/pipermail/e-lang/2009-December/013335.html
The response was that in this context, equality is given by ===.
So that means that we only get guarantees about two invocations to
g if the arguments in both invocations are equal according to "===";
otherwise, there are no promises.

You give an example of two invocation to g, namely g(a) and g(b),
where the arguments in both arguments are not equal according to
"===", namely, a !== b. So you shouldn't expect the results of those
two invocations to be related at all; the two results might be
entirely unrelated.

Does this make sense? Have I missed something?

In Joe-E, pure functions are useful for relating two invocations
when the arguments are value types that can be compared according
to their value (object identity is not visible). When you talk
about invocations where the arguments have visible object identity,
things get much messier. Object identity introduces a can of
worms to discussion of purity.
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-10T22:15:50Z

David-Sarah Hopwood wrote:

> /**
> * Check that a chain of property accesses starting from obj, and
> * using property names given by subsequent arguments, is guaranteed
> * to give a constant result. No getters are invoked during the check.
> *
> * This can be used to relax the condition that all accesses to captured
> * variables of a deepFrozen function are deepFrozen. For instance,
> * if a @deepFrozen function uses foo only by accessing foo.bar.baz,
> * then rather than requiring foo to be deepFrozen, we can require
> * just that foo.bar.baz gives a constant result.
> * I.e. we verify that foo is statically const, and generate a call to
> * checkConstant(foo, 'bar', 'baz') for each scope in which foo
> * might be different.
> */
> function checkConstant(obj /*, ...*/) {

The implementation of checkConstant was consistent with the comment, but
it was not quite what is needed. It's also necessary to check that the
final result of the chain of property accesses is deepFrozen. This must
be done by 'checkConstant' rather than separately by the rewritten code,
because only checkConstant has access to the 'value' fields obtained from
the data property descriptors. (Calling 'checkDeepFrozen(foo.bar.baz)'
would be incorrect since that might invoke getters.)

So, the first sentence of the comment should say "a constant result value
that is deepFrozen." A small change is also needed to the implementation:

function checkConstantAndDeepFrozen(obj /*, ...*/) {
> var i = 1;
> test: while (true) {
- if (i >= arguments.length) return;
+ if (i >= arguments.length) {
+ checkDeepFrozen(obj);
+ return;
+ }

(The other two early returns are correct since they only occur when the
current value of 'obj' is already known to be deepFrozen.)

Note that the special case checkConstantAndDeepFrozen(obj) is the same as
checkDeepFrozen(obj), so some refactoring might be in order.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-10T16:46:35Z

David-Sarah Hopwood wrote:

> David-Sarah Hopwood wrote:
>> It would be possible to make the argument and result checks optional
>> depending on the function annotation: say, /*@pure*/ includes them but
>> /*@functional*/ doesn't. (The environment checks would be the same,
>> and both /*@pure*/ and /*@functional*/ would mark instances as
>> copacetic.)
>>
>> In that case, /*@pure*/ would imply unconditional determinism
>> (referential transparency), whereas /*@functional*/ would only imply
>> determinism for calls for which all arguments are copacetic.
>
> In fact I'm wrong here, as shown by this example:
>
> /*@pure*/ function f() { return cajita.deepFreeze({}); }
>
> const a = f();
> const b = f();
>
> /*@pure*/ function g(x) { return x === a; }
>
> g(a); // true
> g(b); // false
>
> Since a and b are observably different, either f or g must not
> be referentially transparent.
>
> A Cajita that only exposed an egal operator, as defined in
>
> Henry Baker,
> "Equal Rights for Functional Objects, or The More Things Change,
> The More They Are the Same"
> <http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.23.9999>
>
> would solve this problem. (It might be difficult to tame away all
> indirect access to ===, though.)

A more lenient option is to prohibit access to === and !== only to
copacetic functions. So, the above code would be invalid because
g is annotated as @pure, therefore required to be copacetic.
This would transitively deny access to === and !== as long as no
tamed objects are incorrectly marked as being copacetic.
(The fact that some code has access to === and !== precludes the
optimizations I referred to in my previous post, but those optimizations
would not be available in an ECMAScript implementation anyway.)

@pure and @functional functions would still be able to use egal,
if its implementation were "deemed" copacetic (exempted from the
restriction on === and !==).

From now on, I'll rename copacetic to deep-frozen, since it is close
enough to the E concept that I don't think there will be any confusion.
(Bye bye, copacetic. We'll miss you.)
I'll also rename the @functional annotation to @deepFrozen.

I'll write out the differences between the auditing in E, Joe-E, and
my proposal for Cajita in a separate post. In the meantime, here is most
of the run-time support that could be used to implement deep-frozen
auditing in Cajita:

// Using <http://wiki.ecmascript.org/doku.php?id=strawman:weak_references>.
function makeWeakTable() { return new EphemeronTable(true); }

/**
* An auditDeepFrozen entry has the value:
* false or a sour pumpkin, if the key is known not to be deepFrozen
* true or a tasty pumpkin, if the key is known to be deepFrozen.
*
* The objects traversed in a given checkDeepFrozen run are set
* to a unique pumpkin object, which is used to detect loops.
* A "tasty pumpkin" is an object with a 'tasty' field set to
* true, indicating that its run succeeded. A "sour pumpkin" has
* its 'tasty' field set to false. If a run fails, the entries
* pointing to its pumpkin (which has no 'tasty' field) are left
* in the table but are ignored, i.e. treated as equivalent to a
* missing entry, on subsequent runs.
*/
/*const*/ var auditDeepFrozen = makeWeakTable();

/*const*/ var tastyPumpkin = Object.freeze({ tasty: true });
/*const*/ var sourPumpkin = Object.freeze({ tasty: false });

/**
* A functionMarks entry has the value:
* 'deepFrozen', if marked as @deepFrozen
* 'pure', if marked as @pure.
*/
/*const*/ var functionMarks = makeWeakTable();

/**
* Check that obj satisfies the conditions to be deepFrozen, and throw
* AuditError if not. Memoize any results for non-primitive objects in
* the auditDeepFrozen table.
*/
function checkDeepFrozen(obj) {
if (isPrimitive(obj)) return;

/*const*/ var pumpkin = {};
if (isDeepFrozen(obj, pumpkin)) {
// Mark all visited objects that participated in a loop as deepFrozen.
pumpkin.tasty = true;
Object.freeze(pumpkin);
return;
}
Object.freeze(pumpkin);
throw new AuditError();
}

/**
* Return true if obj is definitely deepFrozen, false if it definitely
* isn't, and pumpkin if we encountered a loop.
*/
function isDeepFrozen(obj, pumpkin) {
var t = auditDeepFrozen.get(obj);
if (t) {
/*const*/ var tasty = t.tasty;
if (tasty !== undefined) return tasty;
if (t === pumpkin) return t;
}

test: {
// Function instances must be marked in order to be deepFrozen.
if (typeof obj === 'function' && !functionMarks.get(obj))
break test;

// Mark that we are checking obj, to avoid infinite loops.
// Progress depends on single-threadedness; multiple threads
// could livelock by treading on each other's pumpkins.
auditDeepFrozen.put(obj, pumpkin);

t = isDeepFrozen(Object.getPrototypeOf(obj), pumpkin);
var loopy = t === pumpkin;
if (!t || !Object.isFrozen(obj)) break test;

/*const*/ var ownprops = Object.getOwnPropertyNames(obj);
for (var i = 0; i < ownprops.length; i++) {
/*const*/ var prop = ownprops[i];
/*const*/ var desc = Object.getOwnPropertyDescriptor(obj, prop);

if ('value' in desc) {
t = isDeepFrozen(desc.value, pumpkin);
loopy = loopy || t === pumpkin;
if (!t) break test;
} else {
t = isDeepFrozen(desc.get, pumpkin);
loopy = loopy || t === pumpkin;

// It would be sufficient to check that the setter is deepFrozen,
// but a setter that doesn't actually set anything is misleading.
if (!t || desc.set !== null) break test;
}
}

// If the traversal from this object did not hit a loop, then we can
// memoize it as deepFrozen even if the overall checkDeepFrozen run
// fails.
if (loopy) return pumpkin;
auditDeepFrozen.put(obj, tastyPumpkin);
return true;
}

// break test:
auditDeepFrozen.put(obj, sourPumpkin);
return false;
}

/**
* Check that a chain of property accesses starting from obj, and
* using property names given by subsequent arguments, is guaranteed
* to give a constant result. No getters are invoked during the check.
*
* This can be used to relax the condition that all accesses to captured
* variables of a deepFrozen function are deepFrozen. For instance,
* if a @deepFrozen function uses foo only by accessing foo.bar.baz,
* then rather than requiring foo to be deepFrozen, we can require
* just that foo.bar.baz gives a constant result.
* I.e. we verify that foo is statically const, and generate a call to
* checkConstant(foo, 'bar', 'baz') for each scope in which foo
* might be different.
*/
function checkConstant(obj /*, ...*/) {
var i = 1;
test: while (true) {
if (i >= arguments.length) return;

/*const*/ var t = auditDeepFrozen.get(obj);
if (t && t.tasty) return; // optimization

/*const*/ var prop = arguments[i];

var desc;
try {
desc = Object.getOwnPropertyDescriptor(obj, prop);
} catch (e) {
// getOwnPropertyDescriptor *should* only throw if Type(obj) is not
// Object. However the ES5 spec doesn't explicitly say that
// [[GetOwnProperty]] for a host object can't throw (even though
// that would break stuff).

if (isPrimitive(obj)) return;
break test;
}

if (desc === undefined) {
// If prop doesn't exist as an own-property, then the object must be
// non-Extensible, and we continue using the object's prototype.
// If the property doesn't exist in the prototype chain and all
// objects in the chain are non-Extensible, then obj will eventually
// be null, which is allowed by the primitive case above.

if (Object.isExtensible(obj)) break test;
obj = Object.getPrototypeOf(obj);
continue;
}

// The property must be non-Configurable. If it is a data property,
// it must also be non-Writable, and we continue checking that
// subsequent property accesses on the value are constant. If it
// is an accessor property, we conservatively require its getter
// to be deepFrozen (note that the getter may be null).

if (desc.configurable !== false) break test;
if ('value' in desc) {
if (desc.writable !== false) break test;
obj = desc.value;
i++; // next access
continue;
} else {
checkDeepFrozen(desc.get);

// Since the getter is deepFrozen, there is no need to check
// subsequent properties in the chain (which is lucky, because
// we can't).
return;
}
}

// break test:
auditDeepFrozen.put(obj, sourPumpkin);
throw new AuditError();
}

function isPrimitive(obj) {
if (obj == null) return true; // null and undefined
/*const*/ var t = typeof obj;
return t === 'number' || t === 'string' || t === 'boolean';
}

/**
* This would be called by the rewritten code that instantiates an
* annotated function.
*/
function markFunc(mark, obj) {
if (typeof obj !== 'function' || functionMarks.get(obj) !== undefined)
throw new TypeError();

Object.freeze(obj);
functionMarks.put(obj, mark);
checkDeepFrozen(obj);
}

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T21:40:47Z

Mike Samuel wrote:

> 2009/12/8 David-Sarah Hopwood <david-sarah@...>:
>> var f = /*@functional*/ function (evil, s) {
>> evil(s);
>> };
>>
>> f is now copacetic.
>>
>>> If passed the arguments (freeze(eval), 'a = 0') it will modify the
>>> lexical environment, and I think both arguments are copacetic
>>> according to the frozen functions definition,
>> In
>>
>> f(cajita.freeze(eval), 'a = 0;');
>>
>> there is a free variable reference 'eval', and the global eval is
>> not accessible to Cajita code. So this program is not valid Cajita.
>
> So it is not a goal for a copacetic program to maintain its properties
> when functions defined in it are called by non-strict non-cajita code.
> So it is the copacetic program parsers responsibility to make sure
> that such functions do not escape to where they could be called that
> way?

Non-Cajita ECMAScript code would not be able to mutate copacetic values,
because copacetic would imply deep-frozen using ES5 Object.freeze.
Also, non-global eval can only mutate variables in the lexical scope
in which 'eval' is referenced. So non-Cajita code could only manipulate
its own scope this way, not Cajita scopes.

This assumes that non-Cajita code can't diddle with the copacetic
marker tags, or otherwise interfere with the security of the Cajita
runtime. Note that any implementation of a marker tag using a property
on the marked object would not work here, because non-Cajita code would
be able to set that property on a new object in the same way (and with
the same attributes) as the Cajita runtime.

However, a Name or Trademark mechanism (which can be implemented in
terms of weak hashtables), could work.
See <http://wiki.ecmascript.org/doku.php?id=strawman:weak_references>,
subsection 'Trademarking', for instance.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T20:29:44Z

Bill Frantz wrote:

> david-sarah@... (David-Sarah Hopwood) on Tuesday, December 8, 2009 wrote:
>
>> "copacetic" [*]
>>
>> ...
>>
>> [*] Please, don't let this provisional term catch on :-) In practice
>> we can probably get away with using "pure", or think of something
>> better.
>
> Here you have coined a technical meaning for a term that is, as far as I
> know, new to computer science, and you want to change it? :-)

Oh dear, I fear I may have opened a Pandora's box ...

# 1919 I. BACHELLER /Man for Ages/ iv. 69 'As to looks I'd call him, as ye
# might say, real copasetic.' Mrs. Lukins expressed this opinion solemnly...
# Its last word stood for nothing more than an indefinite depth of meaning.
-- OED

<http://en.wiktionary.org/wiki/Talk:copacetic>
<http://www.worldwidewords.org/weirdwords/ww-cop1.htm>

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Causeway support for AmbientTalk

2009-12-08T20:17:04Z

Hi all,

I just wanted to share with the E community that AmbientTalk (also a communicating event loop language, directly influenced by E) now also supports Terry Stanley's Causeway debugger. I wrote up a small tutorial on how to use Causeway in AmbientTalk, which I think might be useful for those with a general interest in Causeway:
http://code.google.com/p/ambienttalk/wiki/Debugging

Thanks go to Terry, Mark and Tyler for their support and, of course, for providing Causeway in the first place.

Cheers,
Tom

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T18:13:01Z

2009/12/8 David-Sarah Hopwood <david-sarah@...>:

> Mike Samuel wrote:
>> Is the following copacetic by that definition?
>>
>> (function (eval, s) {
>> eval(s);
>> })
>
> Trivially no, since it isn't annotated as functional nor pure.
> Also, Cajita is a subset of ES5-strict, which prohibits 'eval' from
> being used as a formal parameter name:
>
> # It is a SyntaxError if the identifier eval or arguments appears
> # within a FormalParameterList of a strict mode FunctionDeclaration
> # or FunctionExpression (13.1)
>
> Let's fix these problems first:
>
> var f = /*@functional*/ function (evil, s) {
> evil(s);
> };
>
> f is now copacetic.
>
>> If passed the arguments (freeze(eval), 'a = 0') it will modify the
>> lexical environment, and I think both arguments are copacetic
>> according to the frozen functions definition,
>
> In
>
> f(cajita.freeze(eval), 'a = 0;');
>
> there is a free variable reference 'eval', and the global eval is
> not accessible to Cajita code. So this program is not valid Cajita.

So it is not a goal for a copacetic program to maintain its properties
when functions defined in it are called by non-strict non-cajita code.
So it is the copacetic program parsers responsibility to make sure
that such functions do not escape to where they could be called that
way?

> Suppose that a hypothetical Cajita were extended with a restricted
> cajita.eval function. (It would be frozen so we don't need to explicitly
> freeze it.) Then cajita.eval could not be marked as copacetic. Still,
>
> f(cajita.eval, 'a = 0;');
>
> would succeed up to the point where it calls cajita.eval('a = 0;')
> (because f is only marked as functional, not pure, so its
> cajita.eval argument is not required to be copacetic). But then it
> would fail because 'a = 0;' is not a valid Cajita program (it tries to
> assign to a free variable).
>
> Even if it did not fail for that reason, cajita.eval would presumably
> be modelled on ES5-strict "global eval", which cannot directly modify
> its lexical environment.
>
> OTOH,
>
> f(cajita.eval, 'var a = 0;');
>
> would succeed, but is harmless.
>
> Note that single-argument cajita.eval would have to treat its code as
> being in an empty lexical environment (apart from safe globals).
> There could potentially be a variant that would support evaluation
> in an explicitly specified environment:
>
> var f2 = /*@functional*/ function (evil, s, e) {
> evil(s, e);
> };
>
> var env = {a: undefined};
> f(cajita.eval, 'a = 0;', env);
>
> but since env is not frozen (and neither cajita.eval nor f are pure),
> there is no reason to expect this code to be prevented from mutating
> env.a.
>
> --
> David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com
>
>

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T17:23:45Z

Mike Samuel wrote:
> Is the following copacetic by that definition?
>
> (function (eval, s) {
> eval(s);
> })

Trivially no, since it isn't annotated as functional nor pure.
Also, Cajita is a subset of ES5-strict, which prohibits 'eval' from
being used as a formal parameter name:

# It is a SyntaxError if the identifier eval or arguments appears
# within a FormalParameterList of a strict mode FunctionDeclaration
# or FunctionExpression (13.1)

Let's fix these problems first:

var f = /*@functional*/ function (evil, s) {
evil(s);
};

f is now copacetic.

> If passed the arguments (freeze(eval), 'a = 0') it will modify the
> lexical environment, and I think both arguments are copacetic
> according to the frozen functions definition,

In

f(cajita.freeze(eval), 'a = 0;');

there is a free variable reference 'eval', and the global eval is
not accessible to Cajita code. So this program is not valid Cajita.

Suppose that a hypothetical Cajita were extended with a restricted
cajita.eval function. (It would be frozen so we don't need to explicitly
freeze it.) Then cajita.eval could not be marked as copacetic. Still,

f(cajita.eval, 'a = 0;');

would succeed up to the point where it calls cajita.eval('a = 0;')
(because f is only marked as functional, not pure, so its
cajita.eval argument is not required to be copacetic). But then it
would fail because 'a = 0;' is not a valid Cajita program (it tries to
assign to a free variable).

Even if it did not fail for that reason, cajita.eval would presumably
be modelled on ES5-strict "global eval", which cannot directly modify
its lexical environment.

OTOH,

f(cajita.eval, 'var a = 0;');

would succeed, but is harmless.

Note that single-argument cajita.eval would have to treat its code as
being in an empty lexical environment (apart from safe globals).
There could potentially be a variant that would support evaluation
in an explicitly specified environment:

var f2 = /*@functional*/ function (evil, s, e) {
evil(s, e);
};

var env = {a: undefined};
f(cajita.eval, 'a = 0;', env);

but since env is not frozen (and neither cajita.eval nor f are pure),
there is no reason to expect this code to be prevented from mutating
env.a.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T16:31:39Z

David-Sarah Hopwood wrote:
> It would be possible to make the argument and result checks optional
> depending on the function annotation: say, /*@pure*/ includes them but
> /*@functional*/ doesn't. (The environment checks would be the same,
> and both /*@pure*/ and /*@functional*/ would mark instances as
> copacetic.)
>
> In that case, /*@pure*/ would imply unconditional determinism
> (referential transparency), whereas /*@functional*/ would only imply
> determinism for calls for which all arguments are copacetic.

In fact I'm wrong here, as shown by this example:

/*@pure*/ function f() { return cajita.deepFreeze({}); }

const a = f();
const b = f();

/*@pure*/ function g(x) { return x === a; }

g(a); // true
g(b); // false

Since a and b are observably different, either f or g must not
be referentially transparent.

A Cajita that only exposed an egal operator, as defined in

Henry Baker,
"Equal Rights for Functional Objects, or The More Things Change,
The More They Are the Same"
<http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.23.9999>

would solve this problem. (It might be difficult to tame away all
indirect access to ===, though.)

Unfortunately egal isn't constant-time. Ideally, an implementation
would optimize it by unifying pointers when they are found to be egal;
in that case it still wouldn't be constant-time, but repeated tests
of egality would be efficient, and redundant subgraphs could be
garbage-collected. This would be similar to hash-consing but possibly
less expensive (depending on whether pointers can be efficiently
unified). Unification can also potentially be done on garbage collection,
as suggested in Baker's paper.

[I was disturbed to see that the first Google hit for "egal operator"
is a travel agent, not even with the same spelling. What is the world
coming to -- do people have no appreciation for computer science?]

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T14:26:27Z

On Tue, Dec 8, 2009 at 2:06 PM, Mike Samuel <mikesamuel@...> wrote:
> Is the following copacetic by that definition?
>
> (function (eval, s) {
> eval(s);
> })

It is "E-copacetic" where the E stands for "evil".

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T14:23:42Z

On Tue, Dec 8, 2009 at 2:06 PM, Mike Samuel <mikesamuel@...> wrote:
> Is the following copacetic by that definition?
>
> (function (eval, s) {
> eval(s);
> })
>
> If passed the arguments (freeze(eval), 'a = 0') it will modify the
> lexical environment, and I think both arguments are copacetic
> according to the frozen functions definition,

Aren't we presuming Cajita, in which case this is illegal? If instead
full ES5-strict, then the above "eval(s)" code should be considered a
special form causing all variables in its lexical scope to be
considered accessible and (if assignable) potentially assigned.

>
>
> 2009/12/8 <ihab.awad@...>:
>> On Tue, Dec 8, 2009 at 12:05 PM, David-Sarah Hopwood
>> <david-sarah@...> wrote:
>>> Note that this implies that the lexical environment of each successfully
>>> created instance is observationally immutable, not just that it can't
>>> be directly changed by that instance. I.e. it also can't be changed
>>> indirectly via an operation on an argument ...
>>
>> D'oww, I missed that one! Yes, thank you.
>>
>>> The implementation sketch I gave would preclude that use case because
>>> if the supplied argument values are not copacetic, the object's read()
>>> method would throw. This might have been too conservative -- if those
>>> argument checks were omitted then the property above (that the function
>>> instance can't change its lexical environment) would still hold.
>>
>> Yes.
>>
>>> It would be possible to make the argument and result checks optional
>>> depending on the function annotation: say, /*@pure*/ includes them but
>>> /*@functional*/ doesn't. (The environment checks would be the same,
>>> and both /*@pure*/ and /*@functional*/ would mark instances as
>>> copacetic.)
>>
>> Yeah I think we're talking about two, possibly both useful classes of
>> behavior here.
>>
>>> What a fun discussion! Thanks for starting it.
>>
>> Glad to oblige. :)
>>
>> Ihab
>>
>> --
>> Ihab A.B. Awad, Palo Alto, CA
>>
>

--
Cheers,
--MarkM
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T14:06:53Z

Is the following copacetic by that definition?

(function (eval, s) {
eval(s);
})

If passed the arguments (freeze(eval), 'a = 0') it will modify the
lexical environment, and I think both arguments are copacetic
according to the frozen functions definition,

2009/12/8 <ihab.awad@...>:

> On Tue, Dec 8, 2009 at 12:05 PM, David-Sarah Hopwood
> <david-sarah@...> wrote:
>> Note that this implies that the lexical environment of each successfully
>> created instance is observationally immutable, not just that it can't
>> be directly changed by that instance. I.e. it also can't be changed
>> indirectly via an operation on an argument ...
>
> D'oww, I missed that one! Yes, thank you.
>
>> The implementation sketch I gave would preclude that use case because
>> if the supplied argument values are not copacetic, the object's read()
>> method would throw. This might have been too conservative -- if those
>> argument checks were omitted then the property above (that the function
>> instance can't change its lexical environment) would still hold.
>
> Yes.
>
>> It would be possible to make the argument and result checks optional
>> depending on the function annotation: say, /*@pure*/ includes them but
>> /*@functional*/ doesn't. (The environment checks would be the same,
>> and both /*@pure*/ and /*@functional*/ would mark instances as
>> copacetic.)
>
> Yeah I think we're talking about two, possibly both useful classes of
> behavior here.
>
>> What a fun discussion! Thanks for starting it.
>
> Glad to oblige. :)
>
> Ihab
>
> --
> Ihab A.B. Awad, Palo Alto, CA
>

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T13:33:47Z

On Tue, Dec 8, 2009 at 12:05 PM, David-Sarah Hopwood
<david-sarah@...> wrote:
> Note that this implies that the lexical environment of each successfully
> created instance is observationally immutable, not just that it can't
> be directly changed by that instance. I.e. it also can't be changed
> indirectly via an operation on an argument ...

D'oww, I missed that one! Yes, thank you.

> The implementation sketch I gave would preclude that use case because
> if the supplied argument values are not copacetic, the object's read()
> method would throw. This might have been too conservative -- if those
> argument checks were omitted then the property above (that the function
> instance can't change its lexical environment) would still hold.

Yes.

> It would be possible to make the argument and result checks optional
> depending on the function annotation: say, /*@pure*/ includes them but
> /*@functional*/ doesn't. (The environment checks would be the same,
> and both /*@pure*/ and /*@functional*/ would mark instances as
> copacetic.)

Yeah I think we're talking about two, possibly both useful classes of
behavior here.

> What a fun discussion! Thanks for starting it.

Glad to oblige. :)

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita [correction]

2009-12-08T12:42:29Z

David-Sarah Hopwood wrote:

> Mark S. Miller wrote:
>> On Tue, Dec 8, 2009 at 8:47 AM, <ihab.awad@...> wrote:
>>> Thank you for the writeup; interesting. Just one point of motivation
>>> that perhaps I missed from the original post:
>>>
>>> On Mon, Dec 7, 2009 at 11:17 PM, David-Sarah Hopwood
>>> <david-sarah@...> wrote:
>>>> To dodge this issue, let's provisionally call a function *instance*
>>>> "copacetic" [*] if:
>>>> - that instance has only captured copacetic values, and
>>>> - it has no side effects and is deterministic whenever it is
>>>> only called with copacetic argument values, and
>>>> - it uses no side-effecting or nondeterministic primitives.
>>> I'm getting at something similar but distinct, call it "i-copacetic".
>>> :) Specifically:
>>>
>>> - it has no side effects on its lexical environment regardless
>>> of its argument values
>>>
>>> The motivation is this: An object's state is managed by some
>>> surrounding system. However, it is allowed to expose "read()" services
>>> to the outside world that do not participate in this state management.
>>> Each "read()" service may side-effect the supplied arguments, but it
>>> must not side-effect the lexical environment of the service (i.e., the
>>> object itself).
>> How is this different from E's DeepFrozen or Joe-E's Immutable?
>
> Since E has no nondeterministic ambient operations, DeepFrozen in E
> implies determinism for calls with only DeepFrozen arguments. But in
> general, if a language does have nondeterministic primitives (say,
> nondeterministic floating point arithmetic), then a copacetic function
> would be verified to be implemented in a deterministic sublanguage.
> In that case, DeepFrozen would differ from (copacetic without the
> argument and result checks).

Actually, DeepFrozen necessarily implies non-access to nondeterminism,
since otherwise a "DeepFrozen" value could be observed to change.
So DeepFrozen is the same as (copacetic without the argument and
result checks) -- or at least to what I intended that to mean, despite
some imprecision in the definition.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T12:25:14Z

Mark S. Miller wrote:

> On Tue, Dec 8, 2009 at 8:47 AM, <ihab.awad@...> wrote:
>> Thank you for the writeup; interesting. Just one point of motivation
>> that perhaps I missed from the original post:
>>
>> On Mon, Dec 7, 2009 at 11:17 PM, David-Sarah Hopwood
>> <david-sarah@...> wrote:
>>> To dodge this issue, let's provisionally call a function *instance*
>>> "copacetic" [*] if:
>>> - that instance has only captured copacetic values, and
>>> - it has no side effects and is deterministic whenever it is
>>> only called with copacetic argument values, and
>>> - it uses no side-effecting or nondeterministic primitives.
>> I'm getting at something similar but distinct, call it "i-copacetic".
>> :) Specifically:
>>
>> - it has no side effects on its lexical environment regardless
>> of its argument values
>>
>> The motivation is this: An object's state is managed by some
>> surrounding system. However, it is allowed to expose "read()" services
>> to the outside world that do not participate in this state management.
>> Each "read()" service may side-effect the supplied arguments, but it
>> must not side-effect the lexical environment of the service (i.e., the
>> object itself).
>
> How is this different from E's DeepFrozen or Joe-E's Immutable?

Since E has no nondeterministic ambient operations, DeepFrozen in E
implies determinism for calls with only DeepFrozen arguments. But in
general, if a language does have nondeterministic primitives (say,
nondeterministic floating point arithmetic), then a copacetic function
would be verified to be implemented in a deterministic sublanguage.
In that case, DeepFrozen would differ from (copacetic without the
argument and result checks). But otherwise it is very similar.

I haven't had chance to reread the Joe-E functional purity paper yet
(http://www.cs.berkeley.edu/~daw/papers/pure-ccs08.pdf), so I'll
refrain from commenting on Immutable for the time being.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T12:05:20Z

ihab.awad@... wrote:

You're right, that's a better definition.

The implementation sketch I gave for 'copacetic' actually ensures that
property already, even though the definition above doesn't. That's
because the verification that captured variables are const is done
statically, and the check that captured values are copacetic is done
for each instantiation of a function marked as /*@functional*/, before
any call to that instance.

Note that this implies that the lexical environment of each successfully
created instance is observationally immutable, not just that it can't
be directly changed by that instance. I.e. it also can't be changed
indirectly via an operation on an argument (even if the checks for
arguments being copacetic are dropped as discussed below), or by other
code.

> The motivation is this: An object's state is managed by some
> surrounding system. However, it is allowed to expose "read()" services
> to the outside world that do not participate in this state management.
> Each "read()" service may side-effect the supplied arguments, but it
> must not side-effect the lexical environment of the service (i.e., the
> object itself).

The implementation sketch I gave would preclude that use case because
if the supplied argument values are not copacetic, the object's read()
method would throw. This might have been too conservative -- if those
argument checks were omitted then the property above (that the function
instance can't change its lexical environment) would still hold.

Similar cases occur for higher order functions. For example 'map' or
'fold' functions have no side effects except via their arguments.
With the checks that arguments and result are copacetic, you would need
both /*@functional*/ and non-/*@functional*/ variants of each such
function, with the same implementation. If the argument *and* result
checks were omitted (as David Wagner suggested), the same function
could serve both cases.

Another motivation, of course, is that not doing these checks is more
efficient.

It would be possible to make the argument and result checks optional
depending on the function annotation: say, /*@pure*/ includes them but
/*@functional*/ doesn't. (The environment checks would be the same,
and both /*@pure*/ and /*@functional*/ would mark instances as
copacetic.)

In that case, /*@pure*/ would imply unconditional determinism
(referential transparency), whereas /*@functional*/ would only imply
determinism for calls for which all arguments are copacetic.

What a fun discussion! Thanks for starting it.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita [minor correction]

2009-12-08T11:17:57Z

David-Sarah Hopwood wrote:
> Consider the following example (in ECMAScript syntax since I'll just get
> confused switching between ECMAScript and E):
>
> var f = /*@functional*/ function() {
> return cajita.deepFreeze(function() { return {}; });
> };
> var obj = f(); obj.p = "surprise";

This should have been obj = f()().

(Static typing would have caught that bug :-)

> If this followed equivalent auditor definitions to E, it would pass the
> audit checks, even though 'var obj = f(); obj.p = "surprise";' causes
> an *internal* side-effect (that is, a side effect to an object that does
> not escape).
>
> Note that if we wrap this with another function g:
>
> var g = /*@functional*/ function() {
> var f = /*@functional*/ function() {
> return cajita.deepFreeze(function() { return {}; });
> };
> var obj = f(); obj.p = "surprise";

Same here.

> return cajita.deepFreeze(obj);
> }
>
> the object returned by g (with p:"surprise") is deep-frozen by the
> time it is returned, so f and g arguably *are* functional, even
> though the object f returns is *not* functional.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T10:04:34Z

On Tue, Dec 8, 2009 at 1:08 AM, David-Sarah Hopwood
<david-sarah@...> wrote:
> David Wagner wrote:
>> If I understand correctly, this seems overly strict. It should
>> be safe for a functionally pure function to capture and invoke a
>> reference to a non-copacetic function, shouldn't it?
>
> Yes, provided the return value is deep-frozen. In that case, we
> have almost exactly E's Functional auditor (modulo the lack of
> explicit guards).

Or provided the outer function deep-freezes the return value.

function outer(x) {
return ___.deepFreeze(nonDeepFreezingFunction(x));
}

> No, that was an impressively quick and accurate response (but you
> didn't catch the return value error ;-)

I don't think there *is* one. It might be hard to check all that, but still.

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T09:58:41Z

On Tue, Dec 8, 2009 at 1:06 AM, David Wagner <daw@...> wrote:
> David-Sarah Hopwood wrote:
> P.S. Does Cajita's taming make it a design goal to tame away all access
> to non-determinism? If not, that's something else you'll have to tackle.

We haven't done that yet but are thinking about it at the moment.

> P.P.S. Does Cajita expose object identity?

At the moment, yes, via JS's "===".

> If I know that f is purely functional and I call f(o1) and f(o2),
> do you want to be able to infer that f(o1) is "equivalent" to
> f(o2), or are you OK with the possibility that f(o1) might be
> totally unrelated to f(o2)?

Under the circumstances, we would have to be ok with the "unrelated" case.

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T09:45:15Z

On Tue, Dec 8, 2009 at 12:26 AM, David Wagner <daw@...> wrote:
> Does Cajita have the equivalent of Java's OutOfMemoryError,
> which can be thrown under effectively non-deterministic conditions?
> (The condition under which it is thrown is a deeply non-local
> condition.)

I'm sure there is some interpreter-dependent equivalent. The
"interpreter dependent" part is what makes it thorny though.

> Are we guaranteed in Cajita that whether an exception is thrown at
> any point is a deterministic function of the local state of named
> values involved in a specified computation (not the global state
> of the interpreter; not as a function of values not named explicitly
> in the code)?

I personally don't know; that said, I suspect that, in our world, we
would have to go through each possible state of madness that each JS
embedding in each Web browser could wedge itself into. Nontrivial. :/

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T09:28:07Z

On Tue, Dec 8, 2009 at 9:22 AM, Mark S. Miller <erights@...> wrote:
> How is this different from E's DeepFrozen or Joe-E's Immutable?

Ah I see, it's not. :) In fact:

http://www.eros-os.org/pipermail/e-lang/2005-January/010411.html

and especially Kevin Reid's definition in:

http://www.eros-os.org/pipermail/e-lang/2005-January/010414.html

So functional purity was _not_ what I was after, after all -- _pace_
the very interesting remarks on this thread (thank you posters).

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T09:22:09Z

On Tue, Dec 8, 2009 at 8:47 AM, <ihab.awad@...> wrote:

> Thank you for the writeup; interesting. Just one point of motivation
> that perhaps I missed from the original post:
>
> On Mon, Dec 7, 2009 at 11:17 PM, David-Sarah Hopwood
> <david-sarah@...> wrote:
>> To dodge this issue, let's provisionally call a function *instance*
>> "copacetic" [*] if:
>> - that instance has only captured copacetic values, and
>> - it has no side effects and is deterministic whenever it is
>> only called with copacetic argument values, and
>> - it uses no side-effecting or nondeterministic primitives.
>
> I'm getting at something similar but distinct, call it "i-copacetic".
> :) Specifically:
>
> - it has no side effects on its lexical environment regardless
> of its argument values
>
> The motivation is this: An object's state is managed by some
> surrounding system. However, it is allowed to expose "read()" services
> to the outside world that do not participate in this state management.
> Each "read()" service may side-effect the supplied arguments, but it
> must not side-effect the lexical environment of the service (i.e., the
> object itself).

How is this different from E's DeepFrozen or Joe-E's Immutable?

>
> Ihab
>
> --
> Ihab A.B. Awad, Palo Alto, CA
>

--
Cheers,
--MarkM
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T08:49:07Z

On Mon, Dec 7, 2009 at 11:17 PM, David-Sarah Hopwood
<david-sarah@...> wrote:
> An auditor could however check that a frozen data structure contains
> only copacetic values, and if so mark it as copacetic. That would give
> a consistent soft effect / auditing system with the desired property.

Yes, cool.

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T08:47:42Z

Thank you for the writeup; interesting. Just one point of motivation
that perhaps I missed from the original post:

On Mon, Dec 7, 2009 at 11:17 PM, David-Sarah Hopwood
<david-sarah@...> wrote:
> To dodge this issue, let's provisionally call a function *instance*
> "copacetic" [*] if:
> - that instance has only captured copacetic values, and
> - it has no side effects and is deterministic whenever it is
> only called with copacetic argument values, and
> - it uses no side-effecting or nondeterministic primitives.

I'm getting at something similar but distinct, call it "i-copacetic".
:) Specifically:

- it has no side effects on its lexical environment regardless
of its argument values

The motivation is this: An object's state is managed by some
surrounding system. However, it is allowed to expose "read()" services
to the outside world that do not participate in this state management.
Each "read()" service may side-effect the supplied arguments, but it
must not side-effect the lexical environment of the service (i.e., the
object itself).

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T08:30:52Z

On Mon, Dec 7, 2009 at 10:56 PM, Mike Samuel <mikesamuel@...> wrote:
> Some values that are not representable in JSON but that are
> probably functional: NaN, +/-Infinity, undefined, Dates.

Good point. These too then. :)

> So functional code can't call functions, but can pass function
> objects around?

That's not what I meant - see below.

> What is the definition of functional? That evaluating it has no
> side effect?

No side effect except on the arguments supplied.

> Functions capture the lexical environment in which they're defined and
> capture "this", so it's not pure, as the "purely" in "purely
> functional" above might suggest.

Right, so that's where the compiler would step in and conservatively
annotate certain functions as having no side effects on their lexical
environment.

But it seems this sparked a lot of discussions so off I go to read.... :)

Ihab

--
Ihab A.B. Awad, Palo Alto, CA
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T01:08:08Z

David Wagner wrote:

> David-Sarah Hopwood wrote:
>> First let's decide what property we want from a purely functional auditing
>> system. I think the property we want is:
>>
>> If all values directly referred to by an expression in language L
>> (that is, all captured values and literals, including function literals)
>> pass some auditor, then I know that evaluating the expression in L will
>> have no side effects, and the resulting value or exception will be a
>> deterministic function of those values.
>
> Hmm. I wonder if this is overly strict. Maybe what we care about
> is observational side-effect-freedom: i.e., it's OK if the function
> internally performs some side-effect on an object it allocated itself
> as long as that side effect is not externally visible. Does that
> sound right?
>
> e.g., this is all fine, even though it internally performs
> some side-effecting operations:
>
> /* functional */ function f() {
> var n = 0;
> n++; // OK
> var o = {};
> o.x = 5; // OK
> return o; // OK
> }

Yes, I think you're right. For an imperative or only-mostly-functional
language, there would be a severe loss of expressiveness in not allowing
internal side effects. (I hadn't intended them to be included as
"side effects" in the above definition, but didn't make that sufficiently
clear.)

> Does Cajita have the equivalent of Java's OutOfMemoryError,
> which can be thrown under effectively non-deterministic conditions?
> (The condition under which it is thrown is a deeply non-local
> condition.)

The current failure of the Caja implementation to prevent Cajita
code from catching such exceptions is issue 460, and other causes of
nondeterminism are issue 1175:

<http://code.google.com/p/google-caja/issues/detail?id=460>
<http://code.google.com/p/google-caja/issues/detail?id=1175>

Idealized Cajita, with these bugs fixed (and no other correctness
bugs in the implementation or underlying platform), would be fail-stop
deterministic -- i.e. the computation can stop at an arbitrary point
in case of failure, but is deterministic up to that point.

> Are we guaranteed in Cajita that whether an exception is thrown at
> any point is a deterministic function of the local state of named
> values involved in a specified computation (not the global state
> of the interpreter; not as a function of values not named explicitly
> in the code)?

For idealized Cajita, yes.

>> To dodge this issue, let's provisionally call a function *instance*
>> "copacetic" [*] if:
>> - that instance has only captured copacetic values, and
>> - it has no side effects and is deterministic whenever it is
>> only called with copacetic argument values, and
>> - it uses no side-effecting or nondeterministic primitives.
>
> If I understand correctly, this seems overly strict. It should
> be safe for a functionally pure function to capture and invoke a
> reference to a non-copacetic function, shouldn't it?

Yes, provided the return value is deep-frozen. In that case, we
have almost exactly E's Functional auditor (modulo the lack of
explicit guards).

> I may not be thinking about this clearly, so there may well be
> numerous errors in what I wrote above!

No, that was an impressively quick and accurate response (but you
didn't catch the return value error ;-)

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T01:06:44Z

David-Sarah Hopwood wrote:

>The definitions of the relevant E auditors on that page are:
>
> Functional:
> Check that the object passes the DeepFrozen auditor and that the
> return value for each method is guarded by DeepFrozen.
>
> DeepFrozen:
> Check that all names accessed in the object expression are method
> arguments or declared final with def, and guarded by the DeepFrozen
> guard or by the guard for a primitive immutable type (such as int or
> char).
>
>The first difference is the check that the return value for each
>method of a Functional object is guarded by DeepFrozen. This should
>correspond to a check that the return value of a copacetic function is
>copacetic, which I forgot. Otherwise, a function like this:
>
> function() { return {}; }
>
>would be audited as copacetic and run without throwing an exception,
>which is wrong because each instance of {} is mutable.

In Joe-E, we do consider this function a pure function, and we do allow
pure functions to return mutable objects. So maybe it's worth thinking
about whether there is a compelling reason to forbid mutable return
values from pure functions.

I imagine I've plugged our paper on functionality purity in Joe-E before,
but just in case I haven't mentioned it recently:

http://www.cs.berkeley.edu/~daw/papers/pure-ccs08.pdf

P.S. Does Cajita's taming make it a design goal to tame away all access
to non-determinism? If not, that's something else you'll have to tackle.

P.P.S. Does Cajita expose object identity? If it does, what notion of
determinism do you want to use? Suppose o1,o2 are two objects such that
o1 !== o2, but o1 and o2 are "equivalent" in some sense (say, o1[s] ===
o2[s] for all s). If I know that f is purely functional and I call f(o1)
and f(o2), do you want to be able to infer that f(o1) is "equivalent"
to f(o2), or are you OK with the possibility that f(o1) might be totally
unrelated to f(o2)?
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-08T00:40:31Z

David-Sarah Hopwood wrote:

> David-Sarah Hopwood wrote:
>> [Cross-posted to e-lang from google-caja-discuss. I suggest continuing on
>> e-lang, since there's not much that is Caja-specific here.]
> [...]
>> It's unclear whether we should call a function "purely functional"
>> if it is possible for it to be called with impure arguments (or use
>> captured values that are impure). If it can, and we require it to
>> have no side effects and be deterministic in that case, then almost
>> no useful functions are pure.
>>
>> To dodge this issue, let's provisionally call a function *instance*
>> "copacetic" [*] if:
>> - that instance has only captured copacetic values, and
>> - it has no side effects and is deterministic whenever it is
>> only called with copacetic argument values, and
>> - it uses no side-effecting or nondeterministic primitives.

I meant external side effects here.

> I should also have specified that
> - primitive immutable values are copacetic,
> - frozen objects that directly refer only to copacetic values
> are copacetic,
> - no values are copacetic unless they can be inferred to be so by
> the rules above.
>
> I think this corresponds to the Functional auditor in E (see
> <http://www.erights.org/elang/kernel/auditors/>). Is that correct?

To answer my own question, no -- mainly because there is an error in
my proposal.

The definitions of the relevant E auditors on that page are:

Functional:
Check that the object passes the DeepFrozen auditor and that the
return value for each method is guarded by DeepFrozen.

DeepFrozen:
Check that all names accessed in the object expression are method
arguments or declared final with def, and guarded by the DeepFrozen
guard or by the guard for a primitive immutable type (such as int or
char).

The first difference is the check that the return value for each
method of a Functional object is guarded by DeepFrozen. This should
correspond to a check that the return value of a copacetic function is
copacetic, which I forgot. Otherwise, a function like this:

function() { return {}; }

would be audited as copacetic and run without throwing an exception,
which is wrong because each instance of {} is mutable.

So the (unoptimized) rewrite for

const z = ...;
var f = /*@functional*/ function(x, y) { return x+y+z; };

should have been:

const z = ...;
var f = (___.checkCopacetic(z), ___.copaceticFunc(function(x, y) {
___.checkCopacetic(x); ___.checkCopacetic(y);
return ___.checkCopacetic(x+y+z);
}));

Notice the additional check on x+y+z.

(As it happens, the ECMAScript addition operator always returns either a
string or a number, which are primitive immutable values, therefore the
check can be optimized out in this case.)

The other difference between Functional and my proposal is that
copacetic is defined just in terms of itself; there are not two auditors
with one defined in terms of the other. In this respect, copacetic is
more conservative, and possibly less surprising -- but Functional is
not incorrect in being more lenient.

Consider the following example (in ECMAScript syntax since I'll just get
confused switching between ECMAScript and E):

var f = /*@functional*/ function() {
return cajita.deepFreeze(function() { return {}; });
};
var obj = f(); obj.p = "surprise";

If this followed equivalent auditor definitions to E, it would pass the
audit checks, even though 'var obj = f(); obj.p = "surprise";' causes
an *internal* side-effect (that is, a side effect to an object that does
not escape).

Note that if we wrap this with another function g:

var g = /*@functional*/ function() {
var f = /*@functional*/ function() {
return cajita.deepFreeze(function() { return {}; });
};
var obj = f(); obj.p = "surprise";
return cajita.deepFreeze(obj);
}

the object returned by g (with p:"surprise") is deep-frozen by the
time it is returned, so f and g arguably *are* functional, even
though the object f returns is *not* functional.

The auditors page also defines

Deterministic:
Check that each name accessed in the object expression is either:
declared final with def and guarded by the DeepFrozen auditor; or
visible only to this object expression and declared guarded by
Deterministic.

but this is not included in Figure 3 showing the implications between
audited properties. I'm not yet clear on how Functional relates to
Deterministic.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-08T00:26:44Z

David-Sarah Hopwood wrote:
>First let's decide what property we want from a purely functional auditing
>system. I think the property we want is:
>
> If all values directly referred to by an expression in language L
> (that is, all captured values and literals, including function literals)
> pass some auditor, then I know that evaluating the expression in L will
> have no side effects, and the resulting value or exception will be a
> deterministic function of those values.

Hmm. I wonder if this is overly strict. Maybe what we care about
is observational side-effect-freedom: i.e., it's OK if the function
internally performs some side-effect on an object it allocated itself
as long as that side effect is not externally visible. Does that
sound right?

e.g., this is all fine, even though it internally performs
some side-effecting operations:

/* functional */ function f() {
var n = 0;
n++; // OK
var o = {};
o.x = 5; // OK
return o; // OK
}

Does Cajita have the equivalent of Java's OutOfMemoryError,
which can be thrown under effectively non-deterministic conditions?
(The condition under which it is thrown is a deeply non-local
condition.)

Are we guaranteed in Cajita that whether an exception is thrown at
any point is a deterministic function of the local state of named
values involved in a specified computation (not the global state
of the interpreter; not as a function of values not named explicitly
in the code)?

>To dodge this issue, let's provisionally call a function *instance*
>"copacetic" [*] if:
> - that instance has only captured copacetic values, and
> - it has no side effects and is deterministic whenever it is
> only called with copacetic argument values, and
> - it uses no side-effecting or nondeterministic primitives.

If I understand correctly, this seems overly strict. It should
be safe for a functionally pure function to capture and invoke a
reference to a non-copacetic function, shouldn't it?

e.g., this should be fine, even though it captures a reference to
the non-functional function sort():

/* functional */ function g() {
var l = [1, 5, 3];
sort(l); // OK, even though sort() side-effects its argument
return l;
}

Is it enough to know that all captured values and all arguments
are transitively immutable, and that the function body uses no
side-effecting or deterministic language primitives (whatever they
may be)?

I may not be thinking about this clearly, so there may well be
numerous errors in what I wrote above!
_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

Re: [Caja] Functional auditor for Cajita

2009-12-07T23:33:18Z

David-Sarah Hopwood wrote:
> [Cross-posted to e-lang from google-caja-discuss. I suggest continuing on
> e-lang, since there's not much that is Caja-specific here.]
[...]

> It's unclear whether we should call a function "purely functional"
> if it is possible for it to be called with impure arguments (or use
> captured values that are impure). If it can, and we require it to
> have no side effects and be deterministic in that case, then almost
> no useful functions are pure.
>
> To dodge this issue, let's provisionally call a function *instance*
> "copacetic" [*] if:
> - that instance has only captured copacetic values, and
> - it has no side effects and is deterministic whenever it is
> only called with copacetic argument values, and
> - it uses no side-effecting or nondeterministic primitives.

I should also have specified that
- primitive immutable values are copacetic,
- frozen objects that directly refer only to copacetic values
are copacetic,
- no values are copacetic unless they can be inferred to be so by
the rules above.

I think this corresponds to the Functional auditor in E (see
<http://www.erights.org/elang/kernel/auditors/>). Is that correct?

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment

Re: [Caja] Functional auditor for Cajita

2009-12-07T23:17:53Z

[Cross-posted to e-lang from google-caja-discuss. I suggest continuing on
e-lang, since there's not much that is Caja-specific here.]

ihab.awad@... wrote:
> Hi folks,
>
> Here is a proposal for a Cajita auditor for purely functional values:

First let's decide what property we want from a purely functional auditing
system. I think the property we want is:

If all values directly referred to by an expression in language L
(that is, all captured values and literals, including function literals)
pass some auditor, then I know that evaluating the expression in L will
have no side effects, and the resulting value or exception will be a
deterministic function of those values.

(In the Caja context, L here would be Cajita, i.e. the expression is
rewritten.)

> 1. All transitively frozen JSON values are functional.
>
> 2. All function values are born frozen, so they are functional *as values*.
>
> 3. The compiler can conservatively annotate certain functions as
> purely functional. This will miss some cases, but again, this is
> conservative.

It's unclear whether we should call a function "purely functional"
if it is possible for it to be called with impure arguments (or use
captured values that are impure). If it can, and we require it to
have no side effects and be deterministic in that case, then almost
no useful functions are pure.

To dodge this issue, let's provisionally call a function *instance*
"copacetic" [*] if:
- that instance has only captured copacetic values, and
- it has no side effects and is deterministic whenever it is
only called with copacetic argument values, and
- it uses no side-effecting or nondeterministic primitives.

(This is a soft effect system, with non-copaceticity being the only effect.
Come to think of it, E-style auditing in general is a soft effect system.)

So, a rewritten copacetic function instance must check that all its
arguments are copacetic before using them. If such a function instance
refers to a captured variable, then we must also check whether the
variable's value is copacetic, and that the variable is 'const' (so
that the value cannot change), before using it. The latter checks can
(and for efficiency, should) be done when instantiating the function,
not on every call.

A failure of these checks would deterministically throw an exception,
which is not counted as a side effect.

> Perhaps we can add some annotation for programmers to
> ask the compiler to treat failure to verify as an error, like:
>
> var f = /*@functional*/ function(x, y) { ... };

In my proposal this would mark the function instance as copacetic,
add the above argument checks to the function body, and check any
captured values. So for example:

const z = ...;
var f = /*@functional*/ function(x, y) { return x+y+z; };

would be rewritten (ignoring other aspects of the Cajita rewriting) to:

const z = ...;
var f = (___.checkCopacetic(z), ___.copaceticFunc(function(x, y) {
___.checkCopacetic(x); ___.checkCopacetic(y);
return x+y+z;
}));

> 4. At runtime, any frozen data structure containing only functional
> values as described above can pass a functional auditor, and the
> decision can be memoized on the value.

It's not clear to me that "a frozen data structure containing only
functional values can pass a functional auditor" is correct or sufficient.
Such a structure can contain arbitrary functions, which can refer
to mutable, side-effecting, or nondeterministic values according to
your suggested rules. So, it would not be the case that an arbitrary
expression referring to such a structure would have no side effects
and be deterministic.

An auditor could however check that a frozen data structure contains
only copacetic values, and if so mark it as copacetic. That would give
a consistent soft effect / auditing system with the desired property.

> Does this work?

I think the modification I proposed works. As explained above, it
would require changes to the Cajita rewriting. It also requires 'const'
variables, and a programming style that uses them whereever possible.

[*] Please, don't let this provisional term catch on :-) In practice
we can probably get away with using "pure", or think of something
better.

--
David-Sarah Hopwood ⚥ http://davidsarah.livejournal.com

_______________________________________________
e-lang mailing list
e-lang@...
http://www.eros-os.org/mailman/listinfo/e-lang

signature.asc (300 bytes) Download Attachment