comma vs andalso

[Robert Virding]

2009/7/8 Thomas Lindgren <thomasl_erlang@...>

> ----- Original Message ----
> > From: Richard O'Keefe <ok@...>
> >
> > Let's take two apparently similar functions:
> >
> > f(X) when is_atom(element(3, X)) ; true -> 42.
> >
> > g(X) when is_atom(element(3, X)) orelse true -> 42.
> >
> > By actual test, f(99) -> 42.
> > By actual test, g(99) -> a function_clause exception.
> >
> > That is, in a guard, an expression using 'andalso' or
> > 'orelse' is still an expression, and if an exception
> > occurs in the left-hand operand, the whole expression
> > is still skipped.
> ....
> > I conclude that
> > (1) you had best be EXTREMELY cautious about using 'andalso'
> >     and 'orelse' in guards; they are NOT drop-in replacements
> >     for ',' and ';', and
> > (2) it is long past time that Erlang allowed nested use of
> >     ',' and ';' in guards.
>
> Guards are an awful mess in Erlang. I'd be quite surprised if the above
> difference in behaviour was intentional. If so, what possible purpose does
> it serve?
>

The above difference is intentional! There is one fundamental and important
difference between using orelse/or or ';' and that is in the case of
exceptions. If an exception occurs in the case:

G1 ; G2 ; G3 ; ...

say in G1 then G1 will fail and G2 will be attempted, as an exception is
equivalent to failure. This is logical as they are separate guards. However,
if an exception occurs in:

G1 orelse G2 orelse G3 ...

in G1 again then the WHOLE guard expression will fail, G2 will never be
tried as it is part of the same expression. This IS intentional as using ';'
is exactly equivalent to separate clauses with the same RHS. So

f(...) when G1 ; G2 ; G3 -> <RHS>.

is the same as:

f(...) when G1 -> <RHS>;
f(...) when G2 -> <RHS>;
f(...) when G3 -> <RHS>.

This functionality was added for just this case. There is logic in the
madness. It was simple before with only simple guard tests, it only became
confusing when logical expressions were allowed in guards. They are
practical but were only added later, THEY are the odd man out. This is also
why originally there were no problems with type tests without is_ in guards.

Guards were always meant to be part of pattern matching and no more.

Robert

[Richard O'Keefe]

Let's see if we can make this clear:
   `and` `or` `andalso` and `orelse` are allowed ANYWHERE
   in a guard expression are are treated IDENTICALLY to
   the way they are treated elsewhere.
Of those things that are accepted at all in guards,
ONLY ',' and ';' are treated differently.

   X , Y acts like true=X, Y
   X ; Y acts a bit like case (catch X) of true -> true
                           ; _ -> Y end.

> That was one details, yes, but the main reason was to make it possible
> to refactor a piece of code without being forced to change the names
> of the function tests if you moved an expression from within a guard
> to an ordinary expression and vice versa.

It has never been clear to me why this was considered
important.  It's not something I often do in Haskell or
Clean, where guards _are_ perfectly ordinary expressions,
yet I refactor Haskell code all the time.

> Recall that before the is_...
> versions, there was no way of saying e.g., "Bool = (is_integer(X) and
> is_atom(Y))" without writing a case/if or a separate predicate  
> function.

It's _still_ the case that almost no non-trivial expressions
can be moved into a guard.

 > and with any existing
> code that defined a function such as list(X) or integer(X)),

And the new names created clashes with any previously existing
code that defined a function such as is_list(X).  In fact,
given the obvious rule

        if a module defines or explicitly imports a function
        F/N that matches the name of a built-in function,
        emit a warning, and for that module pretend that the
        built-in function does not exist

no code would have been affected at all, other than to get
new *warning* messages.  A language may need to add new
built-ins at any time.  Erlang has often done so.  Such a
rule is valuable.

So the argument for the "is_" prefix is a non-sequitur:  it
was neither necessary nor sufficient to avoid code breakage.

>  As a simple example, this macro nowadays works in both
> guard expressions and in general expressions - it didn't back then:
>
>  -define(is_a_foo(X), (is_integer(X) or is_binary(X))).
>
> (we can even use orelse here, and it will still work in a guard).

But we _can't_ write

    -define(is_a_bar(X), (Y = X*X, (Y < -2 or Y > 2)).

and expect _that_ to work in both guards and expressions.
[Abstract patterns would solve this problem another way.]
>
>
>> And now, for our convenience, the shorter form of these tests is  
>> being
>> deprecated.
>
> Hold your horses - nobody is deprecating the use of ',' and ';'.

He didn't say that.  He meant that the convenient short tests
like atom(X) are being deprecated in favour of the long names.

>  This fail-to-false
> behaviour was in my opinion a mistake, because it occasionally hides
> a bug in the guard and turns what should have been an observable  
> runtime
> crash into a silent "well, we take the next clause then".

There are arguments on both sides here.
>

> I'm not sure I have any real arguments against nesting of ','/';',
> but I fear the grammar could get rather messy, and that such nested
> guards could be quite difficult to read in practice.

I've been asked to help with a C compiler for a transport-
triggered architecture, so my micro-Erlang compiler is on
indefinite hold.  But I did actually tackle this problem
in the grammar.  The grammar rules for guards need to be
separate from the grammar rules for expressions, but neither
gets particularly "messy".  If I recall correctly, it took
me about half an hour to make the changes.

Such guards _could_ be hard to read in practice, but there's
no reason they _have_ to be.  It is certain that they would
make _some_ guards clearer.  Consider this example:

-define(is_vowel(X),
     ( X == $a ; X == $e ; X == $i ; X == $o ; X == $u )).

count_adjacent_vowels(L) ->
     count_adjacent_vowels(L, 0).

count_adjacent_vowels([V|S=[W|_]], N)
   when ?is_vowel(V), ?is_vowel(W) ->
     count_adjacent_vowels(S, N+1);
count_adjacent_vowels([_|S], N) ->
     count_adjacent_vowels(S, N);
count_adjacent_vowels([], N) ->
     N.

This could also be done using `orelse`, of course.
But what if you want to define guard tests that *can't*
be (ab)used as expressions?

Right now, we can't even do

-define(is_probability(X),
   ( is_float(X), X >= 0.0, 1.0 >= X )).

f(X, Y)
   when ?is_probability(X), ?is_probability(Y) ->
     X*Y.


\


________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

[Richard Carlsson-2]

Richard O'Keefe wrote:
> Let's see if we can make this clear:
>   `and` `or` `andalso` and `orelse` are allowed ANYWHERE
>   in a guard expression are are treated IDENTICALLY to
>   the way they are treated elsewhere.

Quite.

> Of those things that are accepted at all in guards,
> ONLY ',' and ';' are treated differently.
>
>   X , Y    acts like true=X, Y
>   X ; Y acts a bit like case (catch X) of true -> true
>                ; _ -> Y end.

A bit, yes. Just recall that even in the ',' case, no exception
will ever escape the guard as a whole (not sure if this was implied
above). A more exact equivalence could be written:

   -define(wrap(A), try (A)=:=true catch _:_ -> false end).
   -define(comma(A,B), (wrap(A) andalso wrap(B))).
   -define(semi(A,B), (wrap(A) orelse wrap(B))).

>> That was one details, yes, but the main reason was to make it possible
>> to refactor a piece of code without being forced to change the names
>> of the function tests if you moved an expression from within a guard
>> to an ordinary expression and vice versa.
>
> It has never been clear to me why this was considered
> important.  It's not something I often do in Haskell or
> Clean, where guards _are_ perfectly ordinary expressions,
> yet I refactor Haskell code all the time.

Important... maybe not, but definitely nicer when you do want to do it.
It makes it easier to teach the language (even though those who grew up
with Erlang may find the changes uncomfortable). And there were as I
said also considerations such as why you could talk about erlang:'+'/2
but not erlang:'list(X)'. In all, it was worth doing.

>> Recall that before the is_...
>> versions, there was no way of saying e.g., "Bool = (is_integer(X) and
>> is_atom(Y))" without writing a case/if or a separate predicate function.
>
> It's _still_ the case that almost no non-trivial expressions
> can be moved into a guard.

True. But every bit helps. Ive never found "this does not solve all
problems" to be an argument for not making a partial improvement
(as long as it does not create an obstacle for future development).

> And the new names created clashes with any previously existing
> code that defined a function such as is_list(X).  In fact,
> given the obvious rule
>
>     if a module defines or explicitly imports a function
>     F/N that matches the name of a built-in function,
>     emit a warning, and for that module pretend that the
>     built-in function does not exist
>
> no code would have been affected at all, other than to get
> new *warning* messages.  A language may need to add new
> built-ins at any time.  Erlang has often done so.  Such a
> rule is valuable.

Yes, however, there was already a similar rule in place since
ancient times, and it stated that in the case you describe, the
built-in function takes precedence. Bummer. (We are now slowly
trying to phase out this old rule, though, taking baby steps.)

> So the argument for the "is_" prefix is a non-sequitur:  it
> was neither necessary nor sufficient to avoid code breakage.

Neither necessary nor sufficient, but likely. It's a game of
probabilities. I _had_ seen several existing modules that used the
name list(Xs), and float(X) was already in use as a BIF for casting.
In comparison, the is_ convention was much less likely to cause
clashes (indeed, I recall no reports of any such when we introduced
the new names). And the convention has kept working for those type
tests that were added later, e.g., is_boolean(X), is_bitstring(X).

>>> And now, for our convenience, the shorter form of these tests is being
>>> deprecated.
>>
>> Hold your horses - nobody is deprecating the use of ',' and ';'.
>
> He didn't say that.  He meant that the convenient short tests
> like atom(X) are being deprecated in favour of the long names.

I'm sorry, I misread. But the "long" names are only 3 more characters,
and at least in my opinion, they improve readability.

>>  This fail-to-false
>> behaviour was in my opinion a mistake, because it occasionally hides
>> a bug in the guard and turns what should have been an observable runtime
>> crash into a silent "well, we take the next clause then".
>
> There are arguments on both sides here.

Absolutely. But I can tell you that it's one of the least fun kind of
facepalm-inducing bugs that can happen. I've seen programs that have
been running for years and years that now and then took the wrong case
but nobody detected it. Or worse, always took the wrong case.

  > I've been asked to help with a C compiler for a transport-
> triggered architecture, so my micro-Erlang compiler is on
> indefinite hold.  But I did actually tackle this problem
> in the grammar.  The grammar rules for guards need to be
> separate from the grammar rules for expressions, but neither
> gets particularly "messy".  If I recall correctly, it took
> me about half an hour to make the changes.

I'll take your word for that. Good to know.

> Such guards _could_ be hard to read in practice, but there's
> no reason they _have_ to be.  It is certain that they would
> make _some_ guards clearer.  Consider this example:
>
> -define(is_vowel(X),
>     ( X == $a ; X == $e ; X == $i ; X == $o ; X == $u )).

Could be nice, but I'm not so much worried about what a distinguished
computer scientist will do with it, as what will happen in a large
body of code written by well-meaning but average programmers. I'm
open to persuasion, though.

> But what if you want to define guard tests that *can't*
> be (ab)used as expressions?

I don't quite see the point in that, though. Do you feel a similar
urge when programming Haskell, that you'd like to be able to write
things that have a meaning in guards only?

     /Richard


________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

[Michael McDaniel-4]

On Thu, Jul 09, 2009 at 01:28:23AM +0200, Robert Virding wrote:

> 2009/7/8 Thomas Lindgren <thomasl_erlang@...>
>
> > ----- Original Message ----
> > > From: Richard O'Keefe <ok@...>
> > >
> > > Let's take two apparently similar functions:
> > >
> > > f(X) when is_atom(element(3, X)) ; true -> 42.
> > >
> > > g(X) when is_atom(element(3, X)) orelse true -> 42.
> > >
> > > By actual test, f(99) -> 42.
> > > By actual test, g(99) -> a function_clause exception.
> > >
> > > That is, in a guard, an expression using 'andalso' or
> > > 'orelse' is still an expression, and if an exception
> > > occurs in the left-hand operand, the whole expression
> > > is still skipped.
> > ....
> > > I conclude that
> > > (1) you had best be EXTREMELY cautious about using 'andalso'
> > >     and 'orelse' in guards; they are NOT drop-in replacements
> > >     for ',' and ';', and
> > > (2) it is long past time that Erlang allowed nested use of
> > >     ',' and ';' in guards.
> >
> > Guards are an awful mess in Erlang. I'd be quite surprised if the above
> > difference in behaviour was intentional. If so, what possible purpose does
> > it serve?
> >
>
> The above difference is intentional! There is one fundamental and important
> difference between using orelse/or or ';' and that is in the case of
> exceptions. If an exception occurs in the case:
>
> G1 ; G2 ; G3 ; ...
>
> say in G1 then G1 will fail and G2 will be attempted, as an exception is
> equivalent to failure. This is logical as they are separate guards. However,
> if an exception occurs in:
>
> G1 orelse G2 orelse G3 ...
>
> in G1 again then the WHOLE guard expression will fail, G2 will never be
> tried as it is part of the same expression. This IS intentional as using ';'
> is exactly equivalent to separate clauses with the same RHS. So
>
> f(...) when G1 ; G2 ; G3 -> <RHS>.
>
> is the same as:
>
> f(...) when G1 -> <RHS>;
> f(...) when G2 -> <RHS>;
> f(...) when G3 -> <RHS>.
>
> This functionality was added for just this case. There is logic in the
> madness. It was simple before with only simple guard tests, it only became
> confusing when logical expressions were allowed in guards. They are
> practical but were only added later, THEY are the odd man out. This is also
> why originally there were no problems with type tests without is_ in guards.
>
> Guards were always meant to be part of pattern matching and no more.
>
> Robert

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 Thank you for this history and explanation, Robert.  It makes it
 much easier for me to remember and understand how the ';'  works.
 (and, by extension, the other guard separators)


~Michael

________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

[Thomas Lindgren]

Sorry for dropping my end of this; here we go.



----- Original Message ----

> From: Richard Carlsson <richardc@...>
>
> Thomas Lindgren wrote:
> > Guards are an awful mess in Erlang. I'd be quite surprised if the
> > above difference in behaviour was intentional. If so, what possible
> > purpose does it serve?
>
> It is intentional in the respect that 'andalso'/'orelse' behave just
> like any other operator in a guard test. As for purpose, all Erlang's
> operators are allowed in guard expressions, so why make an exception?

First, note that guards as such already are a limited form of boolean formulae, where comma is interpreted as conjunction/and, and semicolon is interpreted as disjunction/"or". Evaluation of guards was, I would suppose, inspired by logic programming: if the formula could be satisfied, the guard succeeded; if not, it simply failed. Thus, a guard like 1/X > Z/Y would not throw an exception when X or Y was zero, but simply fail (because the test is not true).

For reasons unknown (except to some of the very senior members of this mailing list), the initial form of guards only permitted conjunctions of expressions G = E1,E2,E3 ... Some time later, a limited form of disjunction was added, permitting us to write (G1 ; G2 ; ... ; Gn) where the Gi were conjunctions, but still not permitting arbitrary boolean formulae as guards: no disjunctions inside conjunctions, in particular. As I recall, this was because of parser problems. When and/or was being added to guards, I do recall suggesting that general boolean guards should be permitted, to be written using nested and/or. (This was discussed sometime in the 1997-1999 timeframe, right?) But they weren't.

> > At one point, you could also use "and"/"or", the boolean operators
> > manque, in guards. (I haven't checked recently whether this is still
> > the case.) So we then have three very similar sets of guard
> > operators.
>
> 'and'/'or' have always (as far back as I can remember, anyway) been
> allowed in guards, again, probably simply by virtue of being general
> operators in the language. And they don't behave like ','/';' either.

Well, to be precise they have only been allowed since some time after and/or were introduced into the language. (At the time when lots of stuff was added, such as funs, records and so on.) I mention this because I think the mess arises from stuff having been added incrementally with different intents by various people over the years.

Moving on, no, indeed they don't behave the same, and I do consider that a problem. As a consequence, we have three subtly different ways to write boolean formulae in guards. (Sorry about the jargon, dear readers, I'm trying to stay away from "boolean expression" here since we already are using "expression" in another sense) And unfortunately, at this point, _none_ of the three ways actually permit writing full boolean formulae. I consider this a failure in how guards are designed, which is why I'm always complaining about it.

> > Not to mention the twin set of type tests, introduced, as far as I
> > know, to get rid of the lone double entendre of float/1 (as a test or
> > conversion not the most frequent of operations).
>
> That was one details, yes, but the main reason was to make it possible
> to refactor a piece of code without being forced to change the names
> of the function tests if you moved an expression from within a guard
> to an ordinary expression and vice versa. Recall that before the is_...
> versions, there was no way of saying e.g., "Bool = (is_integer(X) and
> is_atom(Y))" without writing a case/if or a separate predicate function.

I seem to recall boolean BIFs for old-style type tests (atom/1, integer/1, ...) in older Erlangs, which would permit one to write (atom(X) and atom(Y)) in clause bodies. Perhaps I misremember; still, there is no technical reason to avoid them in favour of the longer new names, _except_ possibly the clash with float/1. In that regard, I would argue that it would have been far easier to rename the lone float conversion operation rather than all type tests.

> The old type tests didn't correspond to any built-in functions, so you
> had to treat them as "primops" inside the compiler, you couldn't refer
> to them and pass them around, etc. But the old type test names "atom(X)"
> and so forth could not simply be made to work outside guards because
> there would be name clashes (with the float(X) bif and with any existing
> code that defined a function such as list(X) or integer(X)), hence the
> is_... prefix for the generally usable versions that are proper built-in
> functions (defined in the 'erlang' module along with all the others).

But this doesn't solve the problem -- it merely shifts name clashes to another part of the name space. Nor is there anything inherently impossible about defining and providing the BIF erlang:atom/1 instead of erlang:is_atom/1.

So, to conclude: it seems to me as if keeping the short names would have been just about the same as the current approach, except saving three characters per type test.

> > And now, for our convenience, the shorter form of these tests is being
> > deprecated.
>
> Hold your horses - nobody is deprecating the use of ',' and ';'.

(I was talking about the short type tests here.)

> This fail-to-false
> behaviour was in my opinion a mistake, because it occasionally hides
> a bug in the guard and turns what should have been an observable runtime
> crash into a silent "well, we take the next clause then". Some people
> like to use this as a trick to write more compact guards, but that
> makes it hard for someone reading the code to tell whether the
> crash-jumps-to-the-next-case is intentional or not.

See above for what I would think is the reasoning behind the classic semantics. The main drawback of explicit crashes in guards is that as a guard-writer you don't have a lot of opportunities to catch them. To catch and hide explicit crashes, you may then have to turn clause guards into case-expressions. (In this context, I'm tempted to instead make an argument for fullblown expressions in guards, but let's leave that little hairball for another day.)

> ... I'm not sure I have any real arguments against nesting of ','/';',
> but I fear the grammar could get rather messy, and that such nested
> guards could be quite difficult to read in practice.

Well, let me then register my strong vote for actually, finally implementing full boolean formulae in guards. Rather than making the code harder to read, it will become easier: there is no need to code around the issue when you actually need to compose disjunctions, and the tests themselves will be hidden in well-formed macros. Macros which as a bonus can be composed fairly nicely without obscure parsing errors. And, in contrast with using and/or/andalso/orelse, the composed guards will still behave like classic guards.

Finally, a question regarding the grammar issue: this seems superficially like adding two more operators to the expression operator precedence grammar. Is there more and worse than that?

Best,
Thomas


     


________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

[Thomas Lindgren]

----- Original Message ----
> From: Robert Virding <rvirding@...>
> To: Thomas Lindgren <thomasl_erlang@...>
....
> This functionality was added for just this case. There is logic in the
> madness. It was simple before with only simple guard tests, it only became
> confusing when logical expressions were allowed in guards. They are
> practical but were only added later, THEY are the odd man out. This is also
> why originally there were no problems with type tests without is_ in guards.
>
> Guards were always meant to be part of pattern matching and no more.

(This fits my view of the history of guards, fwiw.) I've left a longer version of the argument in a reply to Richard, but basically, I think you would have saved some effort and gained some clarity and expressiveness by first of all implementing nested ","/";" as the connectives.

Best,
Thomas


     


________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org

[Thomas Lindgren]

----- Original Message ----
> From: Richard Carlsson <richardc@...>
>...
> > It's _still_ the case that almost no non-trivial expressions
> > can be moved into a guard.
>
> True. But every bit helps. Ive never found "this does not solve all
> problems" to be an argument for not making a partial improvement
> (as long as it does not create an obstacle for future development).

In my opinion a very dangerous attitude in a language like Erlang, where mistakes in design get set in stone. Off the top of my head:

1. records eternally forced to be tuples because of the chosen API;
2. and/or being strict, meaning we had to introduce andalso/orelse;
3. packages vs module names as atoms leading to unsoundness;
4. (the whole guard mess with all its needless duplication, as discussed elsewhere)

or the following:

....
> Yes, however, there was already a similar rule in place since
> ancient times, and it stated that in the case you describe, the
> built-in function takes precedence. Bummer. (We are now slowly
> trying to phase out this old rule, though, taking baby steps.)

I do think that rule has been criticized for being wrong basically since day one. (At least I have considered it an awful bug since I first encountered it; it makes a hollow mockery out of scoping) So it's good to hear it's getting fixed.

> Neither necessary nor sufficient, but likely. It's a game of
> probabilities. I _had_ seen several existing modules that used the
> name list(Xs), and float(X) was already in use as a BIF for casting.
> In comparison, the is_ convention was much less likely to cause
> clashes (indeed, I recall no reports of any such when we introduced
> the new names). And the convention has kept working for those type
> tests that were added later, e.g., is_boolean(X), is_bitstring(X).

Um, you do realize that the current deprecation of short tests is breaking miles and miles of code? We're hardly taking the path of least resistance here.

Best,
Thomas


     


________________________________________________________________
erlang-questions mailing list. See http://www.erlang.org/faq.html
erlang-questions (at) erlang.org