tag:old.nabble.com,2006:forum-12279Nabble - SableCC2009-12-07T14:49:55ZSableCC home is <a href="http://sablecc.org/" target="_top" rel="nofollow">here</a>.tag:old.nabble.com,2006:post-26686853Re: Help with Grammar Ambiguity2009-12-07T14:49:55Z2009-12-07T14:49:55ZJon Shapcott<br>On Sun, Dec 06, 2009 at 03:49:52PM -0500, Etienne M. Gagnon wrote:
<br>> I'm glad this helped you. Just buy my (future) book, next year. :)
<br><br>Is this going to be a general parsing/compilig book, or one about
<br>SableCC 4?
<br><br>--
<br>Jon Shapcott <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26686853&i=0" target="_top" rel="nofollow">eden@...</a>>
<br>"This is the Space Age, and we are Here To Go" - William S. Burroughs
<br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26686853&i=1" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26668993Re: Help with Grammar Ambiguity2009-12-06T12:49:52Z2009-12-06T12:49:52ZEtienne M. GagnonHi Eli,
<br><br>I'm glad this helped you. Just buy my (future) book, next year. :)
<br><br>Have fun!
<br><br>Etienne
<br><br>Eli Gottlieb wrote:
<br>> Ah, there's the mistake. I forgot to add a semicolon to the
<br>> expression case of the statement production.
<br>>
<br>> If you'd like e-cookies or a Christmas card or something, please just
<br>> say. I've spent the past two or three days pondering this!
<br><br>--
<br>Etienne M. Gagnon, Ph.D.
<br>SableCC: <a href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26668993&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26665075Re: Help with Grammar Ambiguity2009-12-05T21:44:01Z2009-12-05T21:44:01ZEli Gottlieb<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Ah, there's the mistake. I forgot to add a semicolon to the expression case of the statement production.<div><br></div><div>If you'd like e-cookies or a Christmas card or something, please just say. I've spent the past two or three days pondering this!<br><div><div>On Dec 6, 2009, at 12:14 AM, Etienne M. Gagnon wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite">
<div bgcolor="#ffffff" text="#000000">
Hi Eli,<br>
<br>
Interesting case. I've looked at your grammar. I think I found the
source of the conflict:<br>
<br>
statement_block = <br>
lbracket statement+ rbracket;<br>
<br>
statement = <br>
...<br>
{expressionst} expression;<br>
<br>
So, your language allows for a sequence of expressions without
separators:<br>
<br>
i 3 j k i (3) j k ...<br>
<br>
Of course, there is an ambiguity here. "i (3)" has two possible
syntactic interpretations: <br>
<ol>
<li>The expression "i" followed by the expression "(3)".</li>
<li>The expression "i(3)" representing a function call.</li>
</ol>
The solution: add separators or terminators to your statements.
Traditionally, language designers have used ";" as statement separator
or terminator.<br>
<br>
Have fun!<br>
<br>
Etienne<br>
<br>
Eli Gottlieb a écrit :
<blockquote cite="mid:1260067245.2298.9.camel@eli-netbook" type="cite">
<pre wrap="">I'm using SableCC 3.2 to obtain a parser for a custom compiler and it
reports a grammar ambiguity on my expression productions.
If anyone has the spare time to help, I could use some insight on the
ambiguity creating the shift/reduce conflict. A left_expression (or
expression of any kind) followed by a left parenthesis *should* always
be illegal except in the case of a function_call_expression, so
obviously I want it to go with the shift option to resolve this
conflict. I just can't find what makes the reduce possible or how to
specify to SableCC that I want a shift in that situation to parse out a
function call.
</pre>
<blockquote type="cite">
<pre wrap="">shift/reduce conflict in state [stack: TModule PQualifiedIdentifier TDefine PTypedVariableDeclaration TAssignmentOperator PQualifiedIdentifier *] on TLparen in {
[ PFunctionCallExpression = PQualifiedIdentifier * TLparen PExpressionList TRparen ] (shift),
[ PFunctionCallExpression = PQualifiedIdentifier * TLparen TRparen ] (shift),
[ PLeftExpression = PQualifiedIdentifier * ] followed by TLparen (reduce)
}
</pre>
</blockquote>
<pre wrap=""><!---->
Thanks for any light you might be able to shed,
Eli Gottlieb
</pre>
<pre wrap=""><hr size="4" width="90%">
_______________________________________________
SableCC-Discussion mailing list
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26665075&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<a class="moz-txt-link-freetext" href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org/" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</div>
_______________________________________________<br>SableCC-Discussion mailing list<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26665075&i=1" target="_top" rel="nofollow">SableCC-Discussion@...</a><br>http://lists.sablecc.org/listinfo/sablecc-discussion<br></blockquote></div><br></div></body></html><br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26665075&i=2" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26662388Re: Help with Grammar Ambiguity2009-12-05T21:14:24Z2009-12-05T21:14:24ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Eli,<br>
<br>
Interesting case. I've looked at your grammar. I think I found the
source of the conflict:<br>
<br>
statement_block = <br>
lbracket statement+ rbracket;<br>
<br>
statement = <br>
...<br>
{expressionst} expression;<br>
<br>
So, your language allows for a sequence of expressions without
separators:<br>
<br>
i 3 j k i (3) j k ...<br>
<br>
Of course, there is an ambiguity here. "i (3)" has two possible
syntactic interpretations: <br>
<ol>
<li>The expression "i" followed by the expression "(3)".</li>
<li>The expression "i(3)" representing a function call.</li>
</ol>
The solution: add separators or terminators to your statements.
Traditionally, language designers have used ";" as statement separator
or terminator.<br>
<br>
Have fun!<br>
<br>
Etienne<br>
<br>
Eli Gottlieb a écrit :
<blockquote cite="mid:1260067245.2298.9.camel@eli-netbook" type="cite">
<pre wrap="">I'm using SableCC 3.2 to obtain a parser for a custom compiler and it
reports a grammar ambiguity on my expression productions.
If anyone has the spare time to help, I could use some insight on the
ambiguity creating the shift/reduce conflict. A left_expression (or
expression of any kind) followed by a left parenthesis *should* always
be illegal except in the case of a function_call_expression, so
obviously I want it to go with the shift option to resolve this
conflict. I just can't find what makes the reduce possible or how to
specify to SableCC that I want a shift in that situation to parse out a
function call.
</pre>
<blockquote type="cite">
<pre wrap="">shift/reduce conflict in state [stack: TModule PQualifiedIdentifier TDefine PTypedVariableDeclaration TAssignmentOperator PQualifiedIdentifier *] on TLparen in {
[ PFunctionCallExpression = PQualifiedIdentifier * TLparen PExpressionList TRparen ] (shift),
[ PFunctionCallExpression = PQualifiedIdentifier * TLparen TRparen ] (shift),
[ PLeftExpression = PQualifiedIdentifier * ] followed by TLparen (reduce)
}
</pre>
</blockquote>
<pre wrap=""><!---->
Thanks for any light you might be able to shed,
Eli Gottlieb
</pre>
<pre wrap="">
<hr size="4" width="90%">
_______________________________________________
SableCC-Discussion mailing list
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26662388&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<a class="moz-txt-link-freetext" href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26662388&i=1" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26662285Help with Grammar Ambiguity2009-12-05T18:40:45Z2009-12-05T18:40:45ZEli GottliebI'm using SableCC 3.2 to obtain a parser for a custom compiler and it
<br>reports a grammar ambiguity on my expression productions.
<br><br>If anyone has the spare time to help, I could use some insight on the
<br>ambiguity creating the shift/reduce conflict. A left_expression (or
<br>expression of any kind) followed by a left parenthesis *should* always
<br>be illegal except in the case of a function_call_expression, so
<br>obviously I want it to go with the shift option to resolve this
<br>conflict. I just can't find what makes the reduce possible or how to
<br>specify to SableCC that I want a shift in that situation to parse out a
<br>function call.
<br>> shift/reduce conflict in state [stack: TModule PQualifiedIdentifier TDefine PTypedVariableDeclaration TAssignmentOperator PQualifiedIdentifier *] on TLparen in {
<br>> [ PFunctionCallExpression = PQualifiedIdentifier * TLparen PExpressionList TRparen ] (shift),
<br>> [ PFunctionCallExpression = PQualifiedIdentifier * TLparen TRparen ] (shift),
<br>> [ PLeftExpression = PQualifiedIdentifier * ] followed by TLparen (reduce)
<br>> }
<br><br>Thanks for any light you might be able to shed,
<br>Eli Gottlieb
<br><br />Package decasyntax;
<br><br>Helpers
<br> unicode_input_character = [0..0xffff];
<br> letter = ['A'..'Z'] | ['a'..'z'] | [0x7F .. 0xFF];
<br> digit = ['0'..'9'];
<br> nondigit = '_' | '?' | '!' | letter;
<br> all = [0 .. 0xFFFF];
<br> cr = 13;
<br> lf = 10;
<br> tab = 9;
<br> eol = cr | lf | cr lf;
<br> input_character = [unicode_input_character - [cr + lf]];
<br> not_cr_lf = [all - [cr + lf]];
<br> not_star = [all - '*'];
<br> not_star_slash = [not_star - '/'];
<br>
<br> non_zero_digit = ['1'..'9'];
<br> hex_digit = ['0'..'9'] | ['a'..'f'] | ['A'..'F'];
<br> octal_digit = ['0'..'7'];
<br> zero_to_three = ['0'..'3'];
<br> decimal_numeral = '0' | non_zero_digit digit*;
<br> hex_numeral = '0x' hex_digit+;
<br> octal_numeral = '0' octal_digit+;
<br>
<br> single_character = [input_character - ['"' + '\']];
<br> octal_escape = '\' (octal_digit octal_digit? | zero_to_three octal_digit octal_digit);
<br> escape_sequence = '\b' | '\t' | '\n' | '\f' | '\r' | '\"' | '\' ''' | '\\' | octal_escape;
<br> string_character = single_character | escape_sequence;
<br><br> bitwise_binary_operator = 'and' | 'or';
<br> comparison_operator = '=' | '>' | '<' | '>=' | '<=' | '<>';
<br> muldiv_operator = '*' | '/';
<br> addsub_operator = '+' | '-';
<br> at_operator = '@';
<br> exponentiation_operator = '^';
<br> apply_operator = '->';
<br>
<br>Tokens
<br><br> unqualified_identifier = nondigit (digit | nondigit)*;
<br> integer_constant = decimal_numeral | hex_numeral | octal_numeral;
<br> string_constant = '"' string_character* '"';
<br>
<br> global_variable_modifier = 'anonymous' | 'constant';
<br> class_section_name = 'public' | 'private' | 'protected';
<br> object_function_keyword = 'generic' | 'method';
<br> boolean_constant = 'true' | 'false';
<br> module_modifier = 'unsafe' | 'pure';
<br> binary_operator = addsub_operator | muldiv_operator | comparison_operator | bitwise_binary_operator | exponentiation_operator | apply_operator;
<br> comparison_operator = comparison_operator;
<br> muldiv_operator = muldiv_operator;
<br> addsub_operator = addsub_operator;
<br> bitwise_binary_operator = bitwise_binary_operator;
<br> minus_operator = '-';
<br> exponentiation_operator = '^';
<br> apply_operator = '->';
<br> at_operator = '@';
<br> not_operator = 'not';
<br> inout = 'inout';
<br> generic = 'generic';
<br> method = 'method';
<br> comma = ',';
<br> assignment_operator = ':=';
<br> lparen = '(';
<br> rparen = ')';
<br> lbracket = '{';
<br> rbracket = '}';
<br> lsbracket = '[';
<br> rsbracket = ']';
<br> dot = '.';
<br> semicolon = ';';
<br> end_module = 'end.';
<br> module = 'module';
<br> labracket = '<';
<br> rabracket = '>';
<br> import = 'import';
<br> define = 'define';
<br> constructor = 'constructor';
<br> destructor = 'destructor';
<br> extends = 'extends';
<br> class_keyword = 'class';
<br> subrange = 'subrange';
<br> enum = 'enum';
<br> exception = 'exception';
<br> from = 'from';
<br> to = 'to';
<br> equals = '=';
<br> function = 'function';
<br> lambda = 'lambda';
<br> return = 'return';
<br> while = 'while';
<br> do = 'do';
<br> until = 'until';
<br> for = 'for';
<br> case = 'case';
<br> cond = 'cond';
<br> let = 'let';
<br> except = 'except';
<br> finally = 'finally';
<br> try = 'try';
<br> throw = 'throw';
<br> else = 'else';
<br> cast = 'cast';
<br> alias = '&';
<br> as = 'as';
<br> type = 'type';
<br> closure = 'closure';
<br>
<br> blank = eol | tab | ' ';
<br> tabspace = [tab + ' '];
<br> space = [cr + [lf + [tab + ' ']]];
<br> comment = '/*' not_star* '*'+ (not_star_slash not_star* '*'+)* '/';
<br><br>Ignored Tokens
<br><br> blank,
<br> comment;
<br>
<br>Productions
<br> module_definition =
<br> module [name]:identifier module_parameters? [imports]:import_declaration* [definitions]:definition* end_module;
<br><br> qualified_identifier =
<br> {simple} unqualified_identifier |
<br> {imported} qualified_identifier dot unqualified_identifier;
<br><br> identifier = qualified_identifier;
<br><br> /* Syntax forms for module-level definitions, including types. */
<br> identifier_list = {one} unqualified_identifier | {many} identifier_list comma unqualified_identifier;
<br>
<br> module_parameter_list =
<br> {one} [name]:unqualified_identifier |
<br> {many} [rest]:module_parameter_list comma [last]:unqualified_identifier;
<br>
<br> module_parameters =
<br> labracket [parameters]:module_parameter_list rabracket;
<br>
<br> import_declaration =
<br> import [name]:qualified_identifier semicolon;
<br>
<br> type_definition =
<br> type [name]:unqualified_identifier equals type_form semicolon;
<br>
<br> type_form =
<br> {class} class_form |
<br> {subrange} subrange_form |
<br> {enum} enum_form |
<br> {exception} exception_form |
<br> {functype} functype_form |
<br> {closure} closure_form |
<br> {alias} alias_form |
<br> {rename} qualified_identifier;
<br>
<br> class_form =
<br> class_keyword [extension]:extension_clause? lbracket [sections]:class_section_definition* rbracket;
<br>
<br> subrange_form =
<br> subrange [parent]:identifier from [floor]:expression to [ceiling]:expression;
<br>
<br> enum_form =
<br> enum [extension]:extension_clause? lparen identifier_list rparen;
<br>
<br> exception_form =
<br> exception [extension]:extension_clause? lparen identifier_list rparen;
<br>
<br> functype_form =
<br> function [type]:qualified_identifier? lparen [arguments]:argument_list rparen;
<br>
<br> closure_form =
<br> closure lparen [functype]:qualified_identifier rparen lbracket [variables]:member_declaration* rbracket;
<br>
<br> alias_form =
<br> [type]:qualified_identifier alias;
<br>
<br> definition =
<br> {globaldef} global_variable_definition |
<br> {typedef} type_definition |
<br> {moduledef} module_definition |
<br> {fundef} function_definition;
<br><br> typed_variable_declaration =
<br> [type]:identifier [name]:unqualified_identifier;
<br>
<br> typed_variable_definition =
<br> [declaration]:typed_variable_declaration assignment_operator [initial_value]:expression;
<br>
<br> global_variable_definition =
<br> define [modifier]:global_variable_modifier? [definition]:typed_variable_definition semicolon;
<br>
<br> member_declaration =
<br> {named} [type]:identifier [names]:identifier_list semicolon |
<br> {indexed} [type]:identifier lsbracket [quantity]:integer_constant rsbracket semicolon;
<br>
<br> constructor_definition =
<br> constructor [name]:identifier lparen [arguments]:argument_list rparen [body]:statement_block;
<br>
<br> destructor_definition =
<br> destructor [name]:identifier lparen [arguments]:argument_list rparen [body]:statement_block;
<br>
<br> class_routine_definition =
<br> {constructor} constructor_definition |
<br> {destructor} destructor_definition;
<br>
<br> class_section_definition =
<br> [section_type]:class_section_name [members]:member_declaration* [routines]:class_routine_definition*;
<br>
<br> extension_clause =
<br> extends qualified_identifier;
<br>
<br> argument_declaration =
<br> [modifier]:inout? [type]:qualified_identifier [names]:identifier_list;
<br>
<br> argument_list =
<br> {one} argument_declaration |
<br> {many} argument_list semicolon argument_declaration;
<br>
<br> generic_body =
<br> {none} semicolon |
<br> {body} statement_block;
<br>
<br> function_definition =
<br> {generic} generic [type]:qualified_identifier? [name]:identifier lparen [arguments]:argument_list rparen generic_body |
<br> {method} method [name]:identifier lparen [arguments]:argument_list rparen [body]:statement_block |
<br> {function} function [name]:unqualified_identifier lparen [arguments]:argument_list rparen [body]:statement_block;
<br>
<br> expression = exp6;
<br>
<br> exp6 =
<br> {comparison} [exp1]:exp5 [operator]:comparison_operator [exp2]:exp5 |
<br> {others} exp5;
<br><br> exp5 =
<br> {apply} [function]:exp5 apply_operator [arguments]:exp4 |
<br> {others} exp4;
<br>
<br> exp4 =
<br> {addsub} [exp1]:exp4 [operator]:addsub_operator [exp2]:exp3 |
<br> {others} exp3;
<br>
<br> exp3 =
<br> {muldiv} [exp1]:exp3 [operator]:muldiv_operator [exp2]:exp2 |
<br> {others} exp2;
<br>
<br> exp2 =
<br> {binary_bitwise} [exp1]:exp2 [operator]:bitwise_binary_operator [exp2]:exp1 |
<br> {others} exp1;
<br>
<br> exp1 =
<br> {exponentiation} [exp1]:exp1 exponentiation_operator [exp2]:exp0 |
<br> {others} exp0;
<br>
<br> exp0 =
<br> {not} not_operator exp0 |
<br> {minus} minus_operator exp0 |
<br> {at} at_operator [location]:left_expression |
<br> {call} function_call_expression |
<br> {primitive} primitive_expression |
<br> {tupleexp} tuple_expression |
<br> {lambdaexp} lambda_expression |
<br> {parenthetical} parenthetical_expression |
<br> {left} left_expression;
<br>
<br> function_call_expression =
<br> {varargs} [function]:identifier lparen [arguments]:expression_list? rparen |
<br> {parentheticalargs} [function]:parenthetical_expression lparen [arguments]:expression_list? rparen;
<br>
<br> parenthetical_expression =
<br> lparen expression rparen;
<br>
<br> variable_subscript =
<br> lsbracket expression rsbracket;
<br>
<br> left_expression =
<br> {variable} [var]:qualified_identifier |
<br> {subscript} [var]:left_expression [subscript]:variable_subscript |
<br> {parensub} [var]:parenthetical_expression [subscript]:variable_subscript;
<br>
<br> primitive_expression =
<br> {integer} integer_constant |
<br> {boolean} boolean_constant |
<br> {string} string_constant;
<br>
<br> expression_list =
<br> {one} expression |
<br> {many} expression_list comma expression;
<br>
<br> tuple_expression =
<br> lsbracket expression_list rsbracket;
<br>
<br> lambda_expression =
<br> lambda lparen argument_list rparen statement_block;
<br><br> apply_expression =
<br> [function]:expression apply_operator [arguments]:expression;
<br>
<br> return_statement =
<br> return expression semicolon;
<br>
<br> assignment_statement =
<br> left_expression assignment_operator expression semicolon;
<br>
<br> let_assignment =
<br> identifier assignment_operator expression;
<br>
<br> let_assignment_list =
<br> {one} let_assignment |
<br> {many} let_assignment_list semicolon let_assignment;
<br>
<br> let_statement =
<br> let lparen [let_assigments]:let_assignment_list rparen statement;
<br>
<br> except_clause =
<br> except lparen typed_variable_declaration rparen statement;
<br>
<br> finally_clause =
<br> finally statement;
<br>
<br> try_statement =
<br> try statement [exceptions]:except_clause* [finally]:finally_clause?;
<br>
<br> for_statement =
<br> for lparen [initializer]:let_assignment [first_separator]:semicolon [test]:expression [second_separator]:semicolon [step]:statement rparen [loop]:statement;
<br>
<br> while_statement =
<br> while expression do statement;
<br>
<br> do_statement =
<br> do statement until expression semicolon;
<br>
<br> throw_statement =
<br> throw expression semicolon;
<br>
<br> else_clause =
<br> else statement;
<br>
<br> case_clause =
<br> case lparen expression rparen statement;
<br>
<br> cond_statement =
<br> {one} cond [case]:case_clause [else]:else_clause? |
<br> {many} cond lbracket [cases]:case_clause+ rbracket [else]:else_clause?;
<br>
<br> cast_clause =
<br> case lparen typed_variable_definition rparen [success]:statement;
<br>
<br> cast_statement =
<br> {one} cast [case]:case_clause else [failure]:statement |
<br> {many} cast lbracket [cases]:case_clause+ rbracket else [failure]: statement;
<br>
<br> statement_block =
<br> lbracket statement+ rbracket;
<br>
<br> statement =
<br> {returnst} return_statement |
<br> {assignmentst} assignment_statement |
<br> {letst} let_statement |
<br> {tryst} try_statement |
<br> {forst} for_statement |
<br> {whilest} while_statement |
<br> {dost} do_statement |
<br> {throwst} throw_statement |
<br> {condst} cond_statement |
<br> {blockst} statement_block |
<br> {expressionst} expression;
<br><br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26662285&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26620924Re: Newlines and EOF in SableCC 3.22009-12-02T20:42:33Z2009-12-02T20:42:33ZEtienne M. GagnonHi Roger,
<br><br>You could cheat. :) You could hack the input stream to generate a
<br>spurious newline in front of the stream's EOF. That would solve your
<br>problem, while waiting for SableCC 4.
<br><br>Have fun!
<br><br>Etienne
<br><br>Pomeroy, Roger C a écrit :
<div class='shrinkable-quote'><br>> I am currently using SableCC 3.2 (I can't wait for 4.0, its good to see the progress)... but in the meantime, I have a question.
<br>>
<br>> In my grammar, blanks and newlines are significant. User input lines are terminated with newlines. A snippet of the a couple productions look like:
<br>>
<br>> test_statements {->statement} = // everything except for testpreferences (that cannot be inside testgroup, case)
<br>> {push_stmt} push [trail]:blank* new_line {->New statement.push(push)}
<br>> | {pushc_stmt} pushc [trail]:blank* new_line {->New statement.pushc(pushc)}
<br>> | {pushg_stmt} pushg [trail]:blank* new_line {->New statement.pushg(pushg)}
<br>> | {pop} pop [trail]:blank* new_line {->New statement.pop(pop)}
<br>> .....
<br>>
<br>>
<br>>
<br>> My problem is that if a user inputs a file that has something like:
<br>> ...
<br>> PUSH <newline>
<br>> <eof>
<br>>
<br>> it works just fine, but if the user doesn't add a <newline> at the end of the input and has something like:
<br>> ....
<br>> PUSH <eof>
<br>>
<br>> I get a parser error on the last line of the file because there is no newline.
<br>>
<br>> So my question is ... how can I handle this? I can't make the new_line token optional in the production, because that gives rise to lots of shift/reduce errors. What I really want is to be able to say that a statement is terminated by EITHER a new_line or an EOF, but as far as I can tell, there is no way for me to write such a production... Is that correct?
<br>>
<br>> Thanks!
<br>>
<br>> Roger
<br>>
<br>>
<br>>
<br>>
<br>> Roger Pomeroy
<br>> Associate Technical Fellow
<br>> Aerodynamics/Geometry
<br>> Phone 425-237-2568
<br>>
<br>>
<br>>
<br>> _______________________________________________
<br>> SableCC-Discussion mailing list
<br>> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26620924&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br>> <a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br>>
<br>>
</div><br>--
<br>Etienne M. Gagnon, Ph.D.
<br>SableCC: <a href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26620924&i=1" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26620868Newlines and EOF in SableCC 3.22009-12-02T08:22:41Z2009-12-02T08:22:41Zroger_pomeroyI am currently using SableCC 3.2 (I can't wait for 4.0, its good to see the progress)... but in the meantime, I have a question.
<br><br>In my grammar, blanks and newlines are significant. User input lines are terminated with newlines. A snippet of the a couple productions look like:
<br><br> test_statements {->statement} = // everything except for testpreferences (that cannot be inside testgroup, case)
<br> {push_stmt} push [trail]:blank* new_line {->New statement.push(push)}
<br> | {pushc_stmt} pushc [trail]:blank* new_line {->New statement.pushc(pushc)}
<br> | {pushg_stmt} pushg [trail]:blank* new_line {->New statement.pushg(pushg)}
<br> | {pop} pop [trail]:blank* new_line {->New statement.pop(pop)}
<br> .....
<br><br><br><br> My problem is that if a user inputs a file that has something like:
<br> ...
<br> PUSH <newline>
<br><eof>
<br><br>it works just fine, but if the user doesn't add a <newline> at the end of the input and has something like:
<br>....
<br>PUSH <eof>
<br><br>I get a parser error on the last line of the file because there is no newline.
<br><br>So my question is ... how can I handle this? I can't make the new_line token optional in the production, because that gives rise to lots of shift/reduce errors. What I really want is to be able to say that a statement is terminated by EITHER a new_line or an EOF, but as far as I can tell, there is no way for me to write such a production... Is that correct?
<br><br>Thanks!
<br><br>Roger
<br><br><br><br><br>Roger Pomeroy
<br>Associate Technical Fellow
<br>Aerodynamics/Geometry
<br>Phone 425-237-2568
<br><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26620868&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26384030Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T19:46:49Z2009-11-16T19:46:49ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Niklas,<br>
<br>
<blockquote cite="mid:20091117030443.GC47883@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">Reporting an error on an "invalid" specification is easy.
In the lexer, with the partial order, should you report an error when an
ordering ends-up not being used? This might annoy users to no end.
</pre>
</blockquote>
<pre wrap=""><!---->
Do you mean redundant (e.g. due to transitivity) order declarations?
</pre>
</blockquote>
<br>
Let say that you have:<br>
<br>
Precedence<br>
a > b;<br>
a > c;<br>
b > c;<br>
<br>
Assume that there is a lexical conflict involving all of a, b, and c
(e.g. the intersection of the 3 languages is not empty, and it contains
a longest match).<br>
<br>
Then, a wins. This means that, unless there's another conflict, the "b
> c" priority is useless. It would not be annoying if we reported "b
> c" as an erroneous precedence (because unused). Future grammar
modifications might make this precedence useful.<br>
<br>
I think that we should only report an error on a lexical precedence
that does not involve tokens with "intersecting languages" (I know,
that's not mathematically precise; you get what I mean).<br>
<br>
<blockquote cite="mid:20091117030443.GC47883@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">I think that the general idea should be:
- Putting a parser binary priority on an alternative that is not left
and right recursive (after inlining) => error.
- Putting a parser unary priority on an alternative that is not left or
right recursive (after inlining) => error.
- Putting a parser priority clause containing a single unary level =>
error.
</pre>
</blockquote>
<pre wrap=""><!---->
...or multiple successive unary levels that have the same fixity
(prefix or postfix).
</pre>
</blockquote>
<br>
Right. Hrm... This is a lot of details to check. Sematic verifications
in SableCC 4 => a curse for me!<br>
<br>
<blockquote cite="mid:20091117030443.GC47883@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">If a specific priority (not in the above list) is specified but remains
unused => ignore instead of annoying the user.
</pre>
</blockquote>
<pre wrap=""><!---->
It might be interesting to know that some assumed ambiguity doesn't
arise after all. (On the other hand, it could just be output in form
of a diagnostic message.)
</pre>
</blockquote>
In the lexer -> see my comments earlier.<br>
<br>
In the parser -> unlikely:<br>
<pre>a = a ...whatever... a | ...</pre>
is necessarily ambiguous. As soon as you get both left and right
recursion in a "useful" production, there's ambiguity.<br>
<br>
If we detect one-sided only recursion in the Precedence declaration
(that was your detail above), then we know there's no need for
precedence and we report an error.<br>
<br>
<blockquote cite="mid:20091117030443.GC47883@nmhq.net" type="cite">
<pre wrap=""> statement =
...
{if:} 'if' expr 'then' statement
{if_else:} 'if' expr 'then' statement 'else' statement
Precedence
if_else;
if;
would still be a shift/reduce conflict? </pre>
</blockquote>
<br>
It should. You should definitely try it with beta 2. :)<br>
<br>
<blockquote cite="mid:20091117030443.GC47883@nmhq.net" type="cite">
<pre wrap="">And "Dangling" always forces shift?
</pre>
</blockquote>
<br>
Of course; reducing would change the language.<br>
<br>
Etienne<br>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26384030&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26383853Re: Linear-approximate LALR(K) and ambiguityresolution for expressions2009-11-16T19:22:33Z2009-11-16T19:22:33ZNiklas MatthiesOn Tue 2009-11-17 at 10:26h, Christopher Van Kirk wrote on sablecc-discussion:
<br>:
<br>> Also, consider perhaps using 'DelimitedBy' rather than 'Separator'
<br>> if you decide to go with a word.
<br><br>"Delimiter" is more general than "separator". Bracketing, quoting and
<br>terminator symbols are all delimiters, but are not separators.
<br><br>-- Niklas Matthies
<br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383853&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26383765Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T19:04:44Z2009-11-16T19:04:44ZNiklas MatthiesOn Mon 2009-11-16 at 19:05h, Etienne M. Gagnon wrote on sablecc-discussion:
<br>> Niklas Matthies wrote:
<br>:
<br>> Precedence
<br>> fac;
<br>> Left div;
<br>>
<br>> It's elegant => I buy it. :)
<br><br>Cool!
<br><br>> >(More generally maybe, making it an error to specify an ambiguity
<br>> >resolution constraint where there is no ambiguity.)
<br>> >
<br>> Reporting an error on an "invalid" specification is easy.
<br>>
<br>> In the lexer, with the partial order, should you report an error when an
<br>> ordering ends-up not being used? This might annoy users to no end.
<br><br>Do you mean redundant (e.g. due to transitivity) order declarations?
<br><br>> I think that the general idea should be:
<br>> - Putting a parser binary priority on an alternative that is not left
<br>> and right recursive (after inlining) => error.
<br>> - Putting a parser unary priority on an alternative that is not left or
<br>> right recursive (after inlining) => error.
<br>> - Putting a parser priority clause containing a single unary level =>
<br>> error.
<br><br>...or multiple successive unary levels that have the same fixity
<br>(prefix or postfix).
<br><br>> - Putting a lexer priority between two tokens that don't overlap => error
<br><br>Sounds sensible overall.
<br><br>Some of these will probably need to be relaxed though if or when a
<br>library/inclusion mechanism is introduced.
<br><br>> If a specific priority (not in the above list) is specified but remains
<br>> unused => ignore instead of annoying the user.
<br><br>It might be interesting to know that some assumed ambiguity doesn't
<br>arise after all. (On the other hand, it could just be output in form
<br>of a diagnostic message.)
<br><br>In general, there's always the possibility to remove an error when it
<br>turns out to be too annoying. That would be a compatible change, as
<br>opposed to adding an error.
<br><br>:
<div class='shrinkable-quote'><br>> There's a Dangling construct for the dangling ambiguity. It's one of
<br>> the not-yet implemented features.
<br>>
<br>> Will look like:
<br>>
<br>> stm =
<br>> {if:} 'if' '(' exp ')' stm Dangling else? |
<br>> ...
<br>>
<br>> Danging else =
<br>> 'else' stm;
</div><br>Okay. But
<br><br> statement =
<br> ...
<br> {if:} 'if' expr 'then' statement
<br> {if_else:} 'if' expr 'then' statement 'else' statement
<br><br> Precedence
<br> if_else;
<br> if;
<br><br>would still be a shift/reduce conflict?
<br><br>And "Dangling" always forces shift?
<br><br>-- Niklas Matthies
<br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383765&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26383509Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T18:30:13Z2009-11-16T18:30:13ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Niklas,<br>
<br>
<blockquote cite="mid:20091117020730.GB47883@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">x = ( a b c / d e f )*;
// vs
x = ( a b c Separator d e f)*;
</pre>
</blockquote>
<pre wrap=""><!---->
Hmm... For some reason it looks slightly odd to me that Separator has
lower precedence than concatenation here. Not that it doesn't make
sense, though.
</pre>
</blockquote>
It is difficult to read:<br>
<pre wrap=""> x = ( a b c / d e f )*;
</pre>
as<br>
<pre> x = ( (a b c) / (d e f) )*
</pre>
My brain keeps seeing:<br>
<pre> x = ( a b ( c / d ) e f )*
</pre>
But, my brain is able to easily give lower precedence to keywords.<br>
<br>
<blockquote cite="mid:20091117020730.GB47883@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">So, 'Separator' is gaining some points. Is there some shorter
synonym (not acronym nor abbreviation) for it?
</pre>
</blockquote>
<pre wrap=""><!---->
"Link". But "separator" is the established term for, well, separators.
Unless you want a more general term for when the "separator" is really
an (associative?) operator. In which case "Separated By" would work
better than "Separator".
</pre>
</blockquote>
<br>
OK, so 'Separator' it remains.<br>
<br>
<blockquote cite="mid:20091117020730.GB47883@nmhq.net" type="cite">
<pre wrap="">Now, that raises another question. Can I write
expr =
{mul:} (expr Separator '*')^(2,*)
{add:} (expr Separator '+')^(2,*)
{var:} var
Precedence
mul;
add;
;)
</pre>
</blockquote>
<br>
Nitpick: <br>
<pre wrap=""> {mul:} (expr Separator '*')^(2...)</pre>
Ouch! My brain hurts... Le me think about it...<br>
<br>
Inlining would give:<br>
<pre wrap="">{mul_1:} expr '*' expr |
{mul_2:} expr '*' expr tail |
tail = tail '*' expr | '*' expr;
</pre>
<br>
You would get a conflict, as mul and add are not unary operators. But,
had you written:<br>
<pre wrap=""> expr =
{mul:} (expr Separator '*')^(2...)
{add:} (expr Separator '+')^(2...)
{var:} var
Precedence
Left mul;
Left add;
</pre>
It would have failed, as after inlining, mul_2 is not directly left and
right recursive. Too bad.<br>
<br>
Have fun!<br>
<br>
Etienne<br>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383509&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26383559Re: Linear-approximate LALR(K) and ambiguityresolution for expressions2009-11-16T18:17:28Z2009-11-16T18:17:28ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Christopher,<br>
<br>
Christopher Van Kirk wrote:
<blockquote cite="mid:AA38450F833E4C5CB1A5C889A157C5BC@FDC490" type="cite">
<pre wrap="">With this separator syntax do we always have to boil our delimited element
down to a single production?
</pre>
</blockquote>
Yep, each of the delimited element and the separator element must be a
single production/token. It's enforced by the syntax.<br>
<br>
This is not the case in the lexer, which is much more flexible, as
regular expressions don't serve to create syntax trees. A regular
expression can be "ambiguous" without causing any problem; all the
lexer needs to report back is the matched string, not the structure of
the match.<br>
<br>
<blockquote cite="mid:AA38450F833E4C5CB1A5C889A157C5BC@FDC490" type="cite">
<pre wrap="">Could you say have a production like this:
grid = ( ( '{' ( ( '{' exp '}' ) / ',' )* '}' ) / ',' )*;
-> [[exp]] (list of lists of exp)
</pre>
</blockquote>
Are you sure of the location of '{' and '}'? Did you mean:<br>
<pre> grid = '{' (row Separator ',')* '}';
row = '{' (exp Separator ',')* '}';
</pre>
?<br>
<br>
If yes, you can see that the SableCC 4 syntax, even without freely used
parentheses, is much shorter than the SableCC3 syntax to express the
same thing. The big difference is that you get a workable AST without
needing to do CST->AST transformations with SableCC 4.<br>
<br>
<blockquote cite="mid:AA38450F833E4C5CB1A5C889A157C5BC@FDC490" type="cite">
<pre wrap="">Obviously these could be gotten around by breaking up the productions into
smaller pieces, but I'm curious whether they would be disallowed because
they attempt to describe delimited parenthesized expressions.</pre>
</blockquote>
Yes. Parentheses are not allowed in productions/alternatives, except
for separated elements.<br>
<blockquote cite="mid:AA38450F833E4C5CB1A5C889A157C5BC@FDC490" type="cite">
<pre wrap=""> It also goes
without saying that they are difficult to make sense of.
</pre>
</blockquote>
<br>
It's even worse when you get down to the AST.<br>
<br>
<blockquote cite="mid:AA38450F833E4C5CB1A5C889A157C5BC@FDC490" type="cite">
<pre wrap="">Next, can the delimiter itself be an expression or does it have to be a
token?</pre>
</blockquote>
<br>
It can be a production (referred by its name). The production can be as
complex as you wish.<br>
<br>
<blockquote cite="mid:AA38450F833E4C5CB1A5C889A157C5BC@FDC490" type="cite">
<pre wrap=""> I can't think of a use case for an expression delimiter off the top
of my head but it would be neat to be able to do something like that.
</pre>
</blockquote>
<br>
If you start adding comments in the grammar, which could potentially be
done with LR(K) without (hopefully) introducing too many conflicts, I
can see production separators.<br>
<br>
<blockquote cite="mid:AA38450F833E4C5CB1A5C889A157C5BC@FDC490" type="cite">
<pre wrap="">Ps: I'll try very hard not to ask about the AST, but really that's the part
I really care about, so it's tough! Also, consider perhaps using
'DelimitedBy' rather than 'Separator' if you decide to go with a word.
</pre>
</blockquote>
<rant> :) :)<br>
<br>
Is there really a problem understanding that <tt>(id Separator ',')*</tt>
is a list of ids separated by commas? The idea is not to write the
grammar in English, but to write a <i>readable</i> grammar.<br>
<br>
It's nice for the grammar to be close to English, but it is not a goal.
The goal is readability. If we wanted perfect English, we would need to
write:<br>
<pre>Token // singular
one_token;
</pre>
and<br>
<pre>Tokens // plural
first_token, second_token;
</pre>
I've chosen to try simplifying things in SableCC4. We simply write
"Ignored", instead of "Ignored Tokens". I don't think that grammars are
less readable for it. 'Separator' is shorter than 'Separated' 'By' and
'Delimited' 'By'. Ideally, I would like an 8 letters or less word.
'Separator', at 9 letters, seems a little long but I can live with it.<br>
<br>
</rant><br>
<br>
<br>
I know, I know... Your fingers are burning to write the C# back-end.
We'll get there. It's not too far, now!<br>
<br>
<br>
Have fun!<br>
<br>
Etienne<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383559&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26383347Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T18:07:30Z2009-11-16T18:07:30ZNiklas MatthiesOn Mon 2009-11-16 at 20:00h, Etienne M. Gagnon wrote on sablecc-discussion:
<br>:
<br>> The bigger danger, for the readability of the '/' operator would be
<br>> within lexers. Within the parser, the separeted element and the
<br>> separator element must both be simple elements (identifier or inline
<br>> token). Within the lexer, things get uglier:
<br>>
<br>> x = ( a b c / d e f )*;
<br>> // vs
<br>> x = ( a b c Separator d e f)*;
<br><br>Hmm... For some reason it looks slightly odd to me that Separator has
<br>lower precedence than concatenation here. Not that it doesn't make
<br>sense, though.
<br><br>> So, 'Separator' is gaining some points. Is there some shorter
<br>> synonym (not acronym nor abbreviation) for it?
<br><br>"Link". But "separator" is the established term for, well, separators.
<br>Unless you want a more general term for when the "separator" is really
<br>an (associative?) operator. In which case "Separated By" would work
<br>better than "Separator".
<br><br>Now, that raises another question. Can I write
<br><br> expr =
<br> {mul:} (expr Separator '*')^(2,*)
<br> {add:} (expr Separator '+')^(2,*)
<br> {var:} var
<br><br> Precedence
<br> mul;
<br> add;
<br><br>;)
<br><br>-- Niklas Matthies
<br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383347&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26383036RE: Linear-approximate LALR(K) and ambiguityresolution for expressions2009-11-16T17:29:24Z2009-11-16T17:29:24ZChristopher Van KirkNo problem. I'll get my revenge by asking many more silly questions over the
<br>course of your beta that you will be obliged to answer.
<br><br><br>-----Original Message-----
<br>From:
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383036&i=0" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>
<br>[mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383036&i=1" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>.
<br>org] On Behalf Of Etienne M. Gagnon
<br>Sent: Tuesday, November 17, 2009 10:07 AM
<br>To: Discussion mailing list for the SableCC project
<br>Subject: Re: Linear-approximate LALR(K) and ambiguityresolution for
<br>expressions
<br><br>Hi Christopher,
<br><br>Etienne M. Gagnon wrote:
<br>> Ah! No, I don't want to explain this!...
<br>>
<br>> Because you don't get 0 expressions? Why don't we simply write 'exp
<br>> exp*' instead of 'exp+' ? What does the AST looks like? What names do
<br>> you give to "exp", "," and "exp"? ...
<br><br>Reading myself back, I'm afraid that this didn't come out the way I
<br>intended. I should have put a few smiles around to indicate my intended
<br>light tone. It looks so serious on reading!
<br><br>You were just asking the question I had tried hard not to get! I really
<br>don't want to get into the AST discussion (which is, as you
<br>unconsciously realized, the real motivation for adding this operator).
<br><br>So, thanks for your question, and please (re-)read my answer with a few
<br>smiles.
<br><br>Etienne
<br><br>--
<br>Etienne M. Gagnon, Ph.D.
<br>SableCC: <a href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383036&i=2" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26383036&i=3" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26382975RE: Linear-approximate LALR(K) and ambiguityresolution for expressions2009-11-16T17:26:18Z2009-11-16T17:26:18ZChristopher Van KirkSorry about that. My example didn't quite match yours, but I do see why it
<br>wouldn't work.
<br><br>With this separator syntax do we always have to boil our delimited element
<br>down to a single production?
<br><br>Could you say have a production like this:
<br><br>grid = ( ( '{' ( ( '{' exp '}' ) / ',' )* '}' ) / ',' )*;
<br> -> [[exp]] (list of lists of exp)
<br><br>or this:
<br><br>grid = ( ( '{' row '}' ) / ',' )* )
<br> -> [row] (list of rows)
<br>row = ( ( '{' exp '}' ) / ',' )* )
<br> -> [exp] (list of exp's)
<br><br>Obviously these could be gotten around by breaking up the productions into
<br>smaller pieces, but I'm curious whether they would be disallowed because
<br>they attempt to describe delimited parenthesized expressions. It also goes
<br>without saying that they are difficult to make sense of.
<br><br>Next, can the delimiter itself be an expression or does it have to be a
<br>token? I can't think of a use case for an expression delimiter off the top
<br>of my head but it would be neat to be able to do something like that.
<br><br>Chris...
<br><br>Ps: I'll try very hard not to ask about the AST, but really that's the part
<br>I really care about, so it's tough! Also, consider perhaps using
<br>'DelimitedBy' rather than 'Separator' if you decide to go with a word.
<br><br>-----Original Message-----
<br>From:
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382975&i=0" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>
<br>[mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382975&i=1" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>.
<br>org] On Behalf Of Etienne M. Gagnon
<br>Sent: Tuesday, November 17, 2009 9:24 AM
<br>To: Discussion mailing list for the SableCC project
<br>Subject: Re: Linear-approximate LALR(K) and ambiguityresolution for
<br>expressions
<br><br>Christopher Van Kirk wrote:
<br>> Throwing my nonsensical two cents in:
<br>>
<br>> What would be wrong with:
<br>>
<br>> fun_call = id '(' exp [',' exp ]* ')';
<br><br>Ah! No, I don't want to explain this!...
<br><br>Because you don't get 0 expressions? Why don't we simply write 'exp
<br>exp*' instead of 'exp+' ? What does the AST looks like? What names do
<br>you give to "exp", "," and "exp"? ...
<br><br>The AST problem is real; you need names so that visitors work. You need
<br>to also be able to specify CST->AST transformations with a simple
<br>syntax. You lose simplicity as soon as you add parentheses to
<br>alternatives. The separator syntax is my only deviation off this, but it
<br>is a specific case that can be handled relatively simply (as I've put
<br>strict syntactic rules on its use).
<br><br>----
<br>(Opening a can of worm and regretting it soon...)
<br><br>With SableCC 4, you would write:
<br><br> fun_call = id '(' [args:](exp / ',')* ')';
<br><br>You could then write (in Java):
<br><br> case_FunCall(NFunCall node) {
<br> String function_name = node.get_Id().getText();
<br> node.getArgs().apply(this); // visit args
<br> ...
<br> }
<br><br>(Closing the can of worms, and hoping very much that nobody discusses
<br>the AST at this point...)
<br>---
<br><br>If you really want something similar, simply write:
<br><br>fun_call = id '(' exp_list? ')';
<br><br>exp_list = exp additional_exp*;
<br><br>additional_exp = ',' exp;
<br><br>At least, you'll get an AST with good names, and it'll be easy to do
<br>tree transformations.
<br><br>Etienne
<br><br>--
<br>Etienne M. Gagnon, Ph.D.
<br>SableCC: <a href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382975&i=2" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382975&i=3" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26382868Re: Linear-approximate LALR(K) and ambiguityresolution for expressions2009-11-16T17:06:31Z2009-11-16T17:06:31ZEtienne M. GagnonHi Christopher,
<br><br>Etienne M. Gagnon wrote:
<br>> Ah! No, I don't want to explain this!...
<br>>
<br>> Because you don't get 0 expressions? Why don't we simply write 'exp
<br>> exp*' instead of 'exp+' ? What does the AST looks like? What names do
<br>> you give to "exp", "," and "exp"? ...
<br><br>Reading myself back, I'm afraid that this didn't come out the way I
<br>intended. I should have put a few smiles around to indicate my intended
<br>light tone. It looks so serious on reading!
<br><br>You were just asking the question I had tried hard not to get! I really
<br>don't want to get into the AST discussion (which is, as you
<br>unconsciously realized, the real motivation for adding this operator).
<br><br>So, thanks for your question, and please (re-)read my answer with a few
<br>smiles.
<br><br>Etienne
<br><br>--
<br>Etienne M. Gagnon, Ph.D.
<br>SableCC: <a href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382868&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26382768Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T17:00:02Z2009-11-16T17:00:02ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
<br>
<blockquote cite="mid:20091117002959.GA47883@nmhq.net" type="cite">
<pre wrap="">No argument here, except that "matched by a token" sounds very wrong
to me, as opposed to "matched by a token definition/lexer rule" (and
the result of the match being a token). IMO it unduly blurs the
difference between 'id' (= a token class) and 'id("if")' (= a
particular token). The mathematical entities on which the precedence
is declared upon are the former, not the latter. These are distinct
concepts. It's like confusing objects with classes. If you say that
the string "if" is matched by the token 'id("id")', then you must also
say that it isn't matched by the token 'id("foo")'. That doesn't make
sense. What's relevant is that "if" is matched by 'id', and whether it
is a longest match, and then whether it is unique in that capacity or
whether there are other longest matches.
</pre>
</blockquote>
<br>
OK, my vocabulary wasn't mathematically perfect. :)<br>
<br>
<blockquote cite="mid:20091117002959.GA47883@nmhq.net" type="cite">
<pre wrap="">Yeah I'm ranting :). But this is exactly the kind of mixing up of
concepts that used to confuse the hell out of me when I was still
studying.
</pre>
</blockquote>
<br>
Luckily, the website is a Wiki. You'll be able to help clarify my
inexact explanations! :)<br>
<br>
<blockquote cite="mid:20091117002959.GA47883@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">The other thing, we shouldn't forget, is the context of use. A typical
use will look like:
var_decl = 'var' (id / ',')+ ';'
Question: If this was the first time you saw it, would you /guess/ the
semantics of "/" correctly?
</pre>
</blockquote>
<pre wrap=""><!---->
I _suspect_ that I would wonder whether it means some kind of
alternative (like |) or something like that. Ah, here's the real
deal: As there are parentheses around it, I would naturally assume
that "/" is a self-sufficient operator. For example that one could
define a production
foo = id / ',';
Any guesses on what "/" means would tend to be based on that
assumption. The truth, though, is that "id / ','" is incomplete by
itself, it requires a repetition operator to be applied.
Incidentally, this is one more reason why I would prefer putting the
separation operator outside rather than inside.For
</pre>
</blockquote>
For me:<br>
<pre wrap=""> var_decl = 'var' (id Separator ',')+ ';';</pre>
is explicit enough so that my brain sees clearly that 'Separator' is
only making sense within its enclosing '(...)+' list.<br>
<br>
The bigger danger, for the readability of the '/' operator would be
within lexers. Within the parser, the separeted element and the
separator element must both be simple elements (identifier or inline
token). Within the lexer, things get uglier:<br>
<pre>x = ( a b c / d e f )*;
// vs
x = ( a b c Separator d e f)*;
</pre>
So, 'Separator' is gaining some points. Is there some shorter synonym
(not acronym nor abbreviation) for it?<br>
<br>
<blockquote cite="mid:20091117002959.GA47883@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">And we revert "Diff" to "Difference" to remove the temptation of
proposing abbreviated keywords. :)
</pre>
</blockquote>
<pre wrap=""><!---->
"Diff" was infix, right? How about "Except" or "Minus" then?
It's always weird when infix operators don't have infix names
(like "x add y" instead of "x plus y").
</pre>
</blockquote>
<br>
Minus is not a good idea, as we already use "-" (the minus symbol) for
lexical subtraction.<br>
<br>
On the other hand, I very much like 'Except':<br>
<pre>id = letter (letter | digit)* Except ('if' | 'else' | 'while');
</pre>
Thanks!<br>
<br>
Etienne<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382768&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26382532Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T16:29:59Z2009-11-16T16:29:59ZNiklas MatthiesOn Mon 2009-11-16 at 17:05h, Etienne M. Gagnon wrote on sablecc-discussion:
<br>:
<div class='shrinkable-quote'><br>> Don't forget the the lexer *first* looks for the longest match. So,
<br>> if the input is:
<br>>
<br>> iffy
<br>>
<br>> the lexer will match and identifier over matching the 'if' keyword.
<br>> So, in terms of making things clear for users, I really prefer to
<br>> keep explaining things in terms of *first* finding the longest
<br>> string that is matched by tokens, then *second* selecting the token
<br>> with the highest precedence.
</div><br>No argument here, except that "matched by a token" sounds very wrong
<br>to me, as opposed to "matched by a token definition/lexer rule" (and
<br>the result of the match being a token). IMO it unduly blurs the
<br>difference between 'id' (= a token class) and 'id("if")' (= a
<br>particular token). The mathematical entities on which the precedence
<br>is declared upon are the former, not the latter. These are distinct
<br>concepts. It's like confusing objects with classes. If you say that
<br>the string "if" is matched by the token 'id("id")', then you must also
<br>say that it isn't matched by the token 'id("foo")'. That doesn't make
<br>sense. What's relevant is that "if" is matched by 'id', and whether it
<br>is a longest match, and then whether it is unique in that capacity or
<br>whether there are other longest matches.
<br><br>Yeah I'm ranting :). But this is exactly the kind of mixing up of
<br>concepts that used to confuse the hell out of me when I was still
<br>studying.
<br><br>:
<br>> >So how about:
<br>> >
<br>> > fun_call = id '(' exp* SeparatedBy ',' ')';
<br>> >
<br>>
<br>> I actually had "Separated By" as two keywords, initially. The problem is
<br>> that this results in a tedious syntax for a common pattern; not
<br>> necessarily a good idea.
<br><br>I don't know. In the long run, readability is always more important
<br>than ease of writing. And grammars aren't something you write all the
<br>time (well, *you* probably do ;)), it's something you engage in for a
<br>week or two or three, and then a year later or so you (or, as likely,
<br>a colleague) need to revisit the grammar to add a new feature or the
<br>like, and at that point you certainly don't remember all the details
<br>of the grammar syntax, and the subject matter is already complicated
<br>enough as it is. So, I would opt for the less cryptic.
<br><br>:
<br>> The problem, in your syntax, is that we see a single ",", yet, in the
<br>> parse tree, there are as many "," as "exp" minus one. This is why I
<br>> decided that the * would be outside the parentheses.
<br><br>Well, my perception seems to work differently here (I have not problem
<br>seeing "SeperatedBy something" as an operator on repetitions), but
<br>it's your choice of course.
<br><br>:
<div class='shrinkable-quote'><br>> >Maybe making "Sep" an alias would be acceptable:
<br>> > fun_call = id '(' exp* Sep ',' ')';
<br>> >
<br>> >A decent editor will have word completion of course. ;)
<br>>
<br>> I don't like aliases; we have to chose something and stick with it.
<br>>
<br>> Users, when they see two different syntactic structures, expect a
<br>> distinct semantics. Aliases break this assumption, and users start
<br>> dreaming up arbitrary semantic differences...
</div><br>True. As a matter of fact, this is exactly the "different/same names"
<br>argument I made elsewehere. :)
<br><br>:
<br>> The other thing, we shouldn't forget, is the context of use. A typical
<br>> use will look like:
<br>>
<br>> var_decl = 'var' (id / ',')+ ';'
<br>>
<br>> Question: If this was the first time you saw it, would you /guess/ the
<br>> semantics of "/" correctly?
<br><br>I _suspect_ that I would wonder whether it means some kind of
<br>alternative (like |) or something like that. Ah, here's the real
<br>deal: As there are parentheses around it, I would naturally assume
<br>that "/" is a self-sufficient operator. For example that one could
<br>define a production
<br><br> foo = id / ',';
<br><br>Any guesses on what "/" means would tend to be based on that
<br>assumption. The truth, though, is that "id / ','" is incomplete by
<br>itself, it requires a repetition operator to be applied.
<br><br>Incidentally, this is one more reason why I would prefer putting the
<br>separation operator outside rather than inside.
<br><br>> And we revert "Diff" to "Difference" to remove the temptation of
<br>> proposing abbreviated keywords. :)
<br><br>"Diff" was infix, right? How about "Except" or "Minus" then?
<br>It's always weird when infix operators don't have infix names
<br>(like "x add y" instead of "x plus y").
<br><br>-- Niklas Matthies
<br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382532&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26382499Re: Linear-approximate LALR(K) and ambiguityresolution for expressions2009-11-16T16:24:18Z2009-11-16T16:24:18ZEtienne M. GagnonChristopher Van Kirk wrote:
<br>> Throwing my nonsensical two cents in:
<br>>
<br>> What would be wrong with:
<br>>
<br>> fun_call = id '(' exp [',' exp ]* ')';
<br><br>Ah! No, I don't want to explain this!...
<br><br>Because you don't get 0 expressions? Why don't we simply write 'exp
<br>exp*' instead of 'exp+' ? What does the AST looks like? What names do
<br>you give to "exp", "," and "exp"? ...
<br><br>The AST problem is real; you need names so that visitors work. You need
<br>to also be able to specify CST->AST transformations with a simple
<br>syntax. You lose simplicity as soon as you add parentheses to
<br>alternatives. The separator syntax is my only deviation off this, but it
<br>is a specific case that can be handled relatively simply (as I've put
<br>strict syntactic rules on its use).
<br><br>----
<br>(Opening a can of worm and regretting it soon...)
<br><br>With SableCC 4, you would write:
<br><br> fun_call = id '(' [args:](exp / ',')* ')';
<br><br>You could then write (in Java):
<br><br> case_FunCall(NFunCall node) {
<br> String function_name = node.get_Id().getText();
<br> node.getArgs().apply(this); // visit args
<br> ...
<br> }
<br><br>(Closing the can of worms, and hoping very much that nobody discusses
<br>the AST at this point...)
<br>---
<br><br>If you really want something similar, simply write:
<br><br>fun_call = id '(' exp_list? ')';
<br><br>exp_list = exp additional_exp*;
<br><br>additional_exp = ',' exp;
<br><br>At least, you'll get an AST with good names, and it'll be easy to do
<br>tree transformations.
<br><br>Etienne
<br><br>--
<br>Etienne M. Gagnon, Ph.D.
<br>SableCC: <a href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382499&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26382492Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T16:05:26Z2009-11-16T16:05:26ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Niklas Matthies wrote:
<blockquote cite="mid:20091116225948.GB91785@nmhq.net" type="cite">[...]
<pre wrap="">So we need to write
Precedence
Right fac;
Left div;
or
Precedence
Left fac;
Left div;
where Left vs. Right doesn't make any difference for fac.
</pre>
</blockquote>
<br>
Right, but it still works. Not elegant, I concede, and I seek elegance
in SableCC.<br>
<br>
<blockquote cite="mid:20091116225948.GB91785@nmhq.net" type="cite">
<pre wrap="">[...]
Saying that these operators have the same precedence is meaningless,
and having to specify Left or Right for prefix/postfix operators is
pointless.
</pre>
</blockquote>
<br>
Makes sense.<br>
<br>
<blockquote cite="mid:20091116225948.GB91785@nmhq.net" type="cite">
<pre wrap="">
Therefore it would be useful to be able to write
Precedence
fac;
Left div;
[...]
(Or possibly something like "Unary fac;" if a keyword is absolutely
needed.)
</pre>
</blockquote>
<br>
'Unary' and 'Associative' are two keywords I have eliminated.
Originally, the syntax was:<br>
<pre> Precedence
Right Unary fac; // operator appears on the right
Left Associative div;
</pre>
I was seeking a lighter syntax, thus what SableCC currently implements.<br>
<br>
I like your idea to specify unary opertor precedence levels without any
keyword. This way, the use of a keyword is reserved for "associativity".<br>
<pre> Precedence
fac;
Left div;
</pre>
It's elegant => I buy it. :)<br>
<br>
<blockquote cite="mid:20091116225948.GB91785@nmhq.net" type="cite">
<pre wrap="">(More generally maybe, making it an error to specify an ambiguity
resolution constraint where there is no ambiguity.)
</pre>
</blockquote>
Reporting an error on an "invalid" specification is easy.<br>
<br>
In the lexer, with the partial order, should you report an error when
an ordering ends-up not being used? This might annoy users to no end.<br>
<br>
I think that the general idea should be:<br>
- Putting a parser binary priority on an alternative that is not left
and right recursive (after inlining) => error.<br>
- Putting a parser unary priority on an alternative that is not left
or right recursive (after inlining) => error.<br>
- Putting a parser priority clause containing a single unary level
=> error.<br>
- Putting a lexer priority between two tokens that don't overlap =>
error<br>
<br>
If a specific priority (not in the above list) is specified but remains
unused => ignore instead of annoying the user.<br>
<br>
<blockquote cite="mid:20091116225948.GB91785@nmhq.net" type="cite">
<pre wrap="">More questions:
Can the ambiguity resolution also be used to disambiguate dangling
else?
</pre>
</blockquote>
<br>
There's a Dangling construct for the dangling ambiguity. It's one of
the not-yet implemented features.<br>
<br>
Will look like:<br>
<pre> stm =
{if:} 'if' '(' exp ')' stm Dangling else? |
...
Danging else =
'else' stm;
</pre>
<blockquote cite="mid:20091116225948.GB91785@nmhq.net" type="cite">
<pre wrap="">And would the following work:
exp =
{add:} add_exp |
{mul:} mul_exp |
{num:} num;
Priority
Left mul;
Left add;
add_exp = exp '+' exp;
mul_exp = exp '*' exp;
</pre>
</blockquote>
<br>
Yes through implicit inlining, when tree transformations and inlining
are implemented. The following will also work:<br>
<pre> exp =
{add:} exp add_op exp |
{mul:} exp mul_op exp |
{fac:} exp fac_op |
{num:} num;
Priority
fac;
Left mul;
Left add;
add_op = '+' | '-';
mul_op = '*' | '/' | '%';
fac_op = '!';
</pre>
SableCC will inline the element right of "exp" until it finds a token
and report an error if it can't get a token.<br>
<br>
The rule, for inlining a production, is that it must not be directly
recursive. SableCC will apply explicit inlining requests, first, to
avoid giving users surprising error messages.<br>
<br>
Etienne<br>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382492&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26382075RE: Linear-approximate LALR(K) and ambiguityresolution for expressions2009-11-16T15:52:33Z2009-11-16T15:52:33ZChristopher Van KirkThrowing my nonsensical two cents in:
<br><br>What would be wrong with:
<br><br>fun_call = id '(' exp [',' exp ]* ')';
<br><br>Where the square braces indicate an optional parameter. This is usually how
<br>it looks in the manuals.
<br><br>Chris...
<br><br><br>-----Original Message-----
<br>From:
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382075&i=0" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>
<br>[mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382075&i=1" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>.
<br>org] On Behalf Of Niklas Matthies
<br>Sent: Tuesday, November 17, 2009 6:11 AM
<br>To: Discussion mailing list for the SableCC project
<br>Subject: Re: Linear-approximate LALR(K) and ambiguityresolution for
<br>expressions
<br><br>Hi,
<br><br>On Mon 2009-11-16 at 14:15h, Etienne M. Gagnon wrote on sablecc-discussion:
<br>> Niklas Matthies wrote:
<br>> >>Just to make sure: I should use 'Precedence' to describe both "token
<br>> >>precedence" (in the lexer) and "operator precedence" (in the parser), as
<br><div class='shrinkable-quote'><br>> >>in the following example. Right?
<br>> >>
<br>> >
<br>> >It's less common to apply the term "precedence" to tokens (actually
<br>> >it's not about the precedence of tokens here, but of token definitions!),
<br>> >but it isn't wrong either. So, why not.
<br>>
<br>> Given a string matched by N distinct tokens, we want to select the token
<br>> that has precedence over all other tokens. Of course, the implementation
<br>> can be done by "changing" token definitions so that only one token
<br>> matches a given string, yet, theoretically, all of the N tokens match.
</div><br>I'd say that it is input words that match, or are matched by, token
<br>definitions (which define sets of words), and the result of each match
<br>is a token. Of course you can say that
<br><br> plus > id
<br><br>means that each token that is the result of a successful match of
<br>"plus" has precedence over all tokens that are the result of a
<br>successful match of "id", but IMO it's more straightforward to say
<br>that the token definition "plus" has precedence over the token
<br>definition "id" for input words that match both definitions.
<br><br>Just like precedence declarations on the parser level define
<br>precedence of one production over another, not of syntax tree nodes
<br>over other syntax tree nodes.
<br><div class='shrinkable-quote'><br>> >One other thing I'm noticing here is that in the parser, the
<br>> >precedence levels are separated by ";", while here the precedence is
<br>> >marked by ">". This could be a reason to use different names for these
<br>> >sections or to align their syntax.
<br>>
<br>> In the parser, precedence is a total order, and each precedence level
<br>> requires an associativity declaration. The lexer supports partial
<br>> ordering (actually, currently it requires a total order, but this will
<br>> change). The lexer has no concept of associativity, so ">" just gets the
<br>> job done.
<br>>
<br>> SableCC 4 already has many new keywords. I've worked hard at reducing
<br>> their number to a minimum, while not making the language cryptic.
</div><br>Right, but consistency is also a design force. Different things should
<br>have different syntax/names and same things the same syntax/names.
<br>I'm not saying the current syntax is bad, but it's a point to be
<br>considered.
<br><br>> There's still one keyword I'm not sure about: "Separator" (usable in
<br>> lexers and parsers):
<br>>
<br>> fun_call = id '(' (exp Separator ',')* ')';
<br>>
<br>> I am toying with the idea of replacing it with '/' as is "divided by"
<br>> (which is an common definition for this operator):
<br>>
<br>> fun_call = id '(' (exp / ',')* ')';
<br><br>The drawback of "/" is that one has to know what it means, as it isn't
<br>self-evident at all, and not a standard syntax like the Kleene star.
<br>Using the division operator "/" in the sense of "separated by" is
<br>quite a far stretch, even as a mnemonic.
<br><br>In a fluent[1] Java API for regular expressions I wrote, one can write
<br><br> Regex expr = ...;
<br> Regex args = zeroOrMore(expr).separatedBy(',');
<br><br>So how about:
<br><br> fun_call = id '(' exp* SeparatedBy ',' ')';
<br><br>That would be more self-explanatory. Putting the separator "outside"
<br>of the repition has some benefits: It doesn't require an extra set of
<br>parentheses, it's arguably easier to read out loud in a comprehensible
<br>way, it keeps the AST facts ("here we have a list of expressions")
<br>together ("exp*"), and lastly the separator comes more expected when
<br>the repetition has already been established (e.g.
<br>"expr.separatedBy(',').zeroOrMore()" wouldn't be quite as readable).
<br><br>[1] <a href="http://martinfowler.com/bliki/FluentInterface.html" target="_top" rel="nofollow">http://martinfowler.com/bliki/FluentInterface.html</a><br><br>:
<br>> The separator is likely to be a commonly used operator in parsing
<br>> grammars, so making common things easy to type is not a bad idea.
<br><br>Maybe making "Sep" an alias would be acceptable:
<br><br> fun_call = id '(' exp* Sep ',' ')';
<br><br>A decent editor will have word completion of course. ;)
<br><br>-- Niklas Matthies
<br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382075&i=2" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26382075&i=3" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26381566Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T14:59:48Z2009-11-16T14:59:48ZNiklas MatthiesOn Sun 2009-11-15 at 15:24h, Etienne M. Gagnon wrote on sablecc-discussion:
<br>:
<br>> Actually, with unary operators you still have to decide whether you
<br>> want to give left or right priority, e.g.:
<br>>
<br>> ++ a - b (assuming ++ and - have the same precedence level) =>
<br>> left : (++ a) - b
<br>> right : ++ (a - b)
<br><br>Er, no. This is not right vs. left, it's just different precedence.
<br>Take the example of a language with two operators, a left-associative
<br>binary operator div := '/' and a postfix operator fac != '!', where
<br><br> a!/b!/c! (*)
<br><br>should be parsed as
<br><br> ((a!)/(b!))/(c!)
<br><br>We can't write
<br><br> Precedence
<br> Left div, fac;
<br><br>because this would result in (*) being parsed as
<br><br> (((((a!)/b)!)/c)!
<br><br>Of course we also can't write
<br><br> Precedence
<br> Right div, fac;
<br><br>which would cause (*) to be parsed as
<br><br> (a!)/((b!)/(c!))
<br><br>So we need to write
<br><br> Precedence
<br> Right fac;
<br> Left div;
<br><br>or
<br><br> Precedence
<br> Left fac;
<br> Left div;
<br><br>where Left vs. Right doesn't make any difference for fac.
<br><br>More generally,
<br><br> Precedence
<br> Left prefix_op, infix_op, suffix_op;
<br><br>is always equivalent to
<br><br> Precedence
<br> <Left or Right> prefix_op;
<br> Left infix_op;
<br> <Left or Right> suffix_op;
<br><br>and
<br><br> Precedence
<br> Right prefix_op, infix_op, suffix_op;
<br><br>is always equivalent to
<br><br> Precedence
<br> <Left or Right> suffix_op;
<br> Right infix_op;
<br> <Left or Right> prefix_op;
<br><br>Saying that these operators have the same precedence is meaningless,
<br>and having to specify Left or Right for prefix/postfix operators is
<br>pointless.
<br><br>Therefore it would be useful to be able to write
<br><br> Precedence
<br> fac;
<br> Left div;
<br><br>and (to take my previous example)
<br><br> Precedence
<br> cond, filter;
<br> Left map, map_filter;
<br><br>(Or possibly something like "Unary fac;" if a keyword is absolutely
<br>needed.)
<br><br>I would also advocate making
<br><br> Left fac;
<br><br>an error, and by consequence also
<br><br> Left div, fac;
<br><br>(More generally maybe, making it an error to specify an ambiguity
<br>resolution constraint where there is no ambiguity.)
<br><br>:
<br>> Yet, I wouldn't be surprised if your intentions were actually to
<br>> disallow : if if a else if b else c else if d else e.
<br><br>There are some 'thens' missing here. I suppose you mean something like:
<br><br> if if a then b else c then if d then e else f else if g then h else i
<br><br>I wouldn't want to disallow that, as it's unambiguous. It also can be
<br>made quite readable by appropriate formatting:
<br><br> x = if
<br> if a then b else c
<br> then
<br> if d then e else f
<br> else
<br> if g then h else i
<br><br>I wouldn't want to be forced to add parentheses here.
<br><br>More questions:
<br><br>Can the ambiguity resolution also be used to disambiguate dangling
<br>else?
<br><br>And would the following work:
<br><br> exp =
<br> {add:} add_exp |
<br> {mul:} mul_exp |
<br> {num:} num;
<br>
<br> Priority
<br> Left mul;
<br> Left add;
<br><br> add_exp = exp '+' exp;
<br> mul_exp = exp '*' exp;
<br><br>-- Niklas Matthies
<br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26381566&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26381202Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T14:42:03Z2009-11-16T14:42:03ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Stephen P Spackman wrote:
<blockquote cite="mid:1ae90d080911161424x6e13c2c0xb3aea3889455cb2@mail.gmail.com" type="cite"><br>
<div class="gmail_quote">
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">In
a fluent[1] Java API for regular expressions I wrote, one can write<br>
<br>
Regex expr = ...;<br>
Regex args = zeroOrMore(expr).separatedBy(',');<br>
</blockquote>
<div><br>
Which I have to agree is quite nice. <br>
</div>
</div>
</blockquote>
SableCC has the following:<br>
<br>
base.zeroOrMoreWithSeparator(separator);<br>
<br>
You can see it in the RegularExpressionEvaluator visitor:<br>
<br>
@Override<br>
public void outASeparatedZeroOrMoreExpression(<br>
ASeparatedZeroOrMoreExpression node) {<br>
<br>
Automaton base = retrieve(node.getBase());<br>
Automaton separator = retrieve(node.getSeparator());<br>
save(node, base.zeroOrMoreWithSeparator(separator));<br>
}<br>
<br>
<br>
<blockquote cite="mid:1ae90d080911161424x6e13c2c0xb3aea3889455cb2@mail.gmail.com" type="cite">
<div class="gmail_quote">
<div>...Which makes me want to suggest that we write "foo{}" instead
of "foo*" and "foo{','}" for the separated case. Not, you know, that
that would fit into the rest of the notation.<br>
<br>
fun_call = id '(' exp */ ',' ')'; , anyone?<br>
</div>
</div>
</blockquote>
<br>
Not for me. There should be a "*" somewhere on ','. The multiplicity
operator affects both the expression and the comma (with a count
difference of one).<br>
<br>
Etienne<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26381202&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26380992Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T14:24:27Z2009-11-16T14:24:27ZStephen P SpackmanHi, all.<br><br><div class="gmail_quote">2009/11/16 Niklas Matthies <span dir="ltr"><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26380992&i=0" target="_top" rel="nofollow">ml_sablecc-user@...</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
The drawback of "/" is that one has to know what it means, as it isn't<br>
self-evident at all, and not a standard syntax like the Kleene star.<br>
Using the division operator "/" in the sense of "separated by" is<br>
quite a far stretch, even as a mnemonic.<br></blockquote><div><br>I'm not sure I agree. Concatenation is clearly the natural multiplicative operator in string grammars; divison should thus mean truncation. It's true that (foo / ',')* does not literally mean that foo minus some trailing comma repeats, but I can't agree that it's unmnemonic: it *is* something to do with division, just as the Kleene star *is* something to do with multiplication - which is where * gets its mnemonic power. <br>
</div><div> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
In a fluent[1] Java API for regular expressions I wrote, one can write<br>
<br>
Regex expr = ...;<br>
Regex args = zeroOrMore(expr).separatedBy(',');<br></blockquote><div><br>Which I have to agree is quite nice. <br></div><div> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
So how about:<br>
<br>
fun_call = id '(' exp* SeparatedBy ',' ')';<br>
<br>
That would be more self-explanatory. Putting the separator "outside"<br>
of the repition has some benefits: It doesn't require an extra set of<br>
parentheses, it's arguably easier to read out loud in a comprehensible<br>
way, it keeps the AST facts ("here we have a list of expressions")<br>
together ("exp*"), and lastly the separator comes more expected when<br>
the repetition has already been established (e.g.<br>
"expr.separatedBy(',').zeroOrMore()" wouldn't be quite as readable).<br></blockquote><div><br>...Which makes me want to suggest that we write "foo{}" instead of "foo*" and "foo{','}" for the separated case. Not, you know, that that would fit into the rest of the notation.<br>
<br>fun_call = id '(' exp */ ',' ')'; , anyone?<br><br>Regards<br>Stephen<br></div></div>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26380992&i=1" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26380631Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T14:05:10Z2009-11-16T14:05:10ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Niklas,<br>
<br>
<br>
<blockquote cite="mid:20091116211031.GA91785@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">Given a string matched by N distinct tokens, we want to select the token
that has precedence over all other tokens. Of course, the implementation
can be done by "changing" token definitions so that only one token
matches a given string, yet, theoretically, all of the N tokens match.
</pre>
</blockquote>
<pre wrap=""><!---->
I'd say that it is input words that match, or are matched by, token
definitions (which define sets of words), and the result of each match
is a token. Of course you can say that
plus > id
means that each token that is the result of a successful match of
"plus" has precedence over all tokens that are the result of a
successful match of "id", but IMO it's more straightforward to say
that the token definition "plus" has precedence over the token
definition "id" for input words that match both definitions.
Just like precedence declarations on the parser level define
precedence of one production over another, not of syntax tree nodes
over other syntax tree nodes.
</pre>
</blockquote>
<br>
Don't forget the the lexer <b>first</b> looks for the longest match.
So, if the input is:<br>
<pre>iffy
</pre>
the lexer will match and identifier over matching the 'if' keyword. So,
in terms of making things clear for users, I really prefer to keep
explaining things in terms of <b>first</b> finding the longest string
that is matched by tokens, then <b>second</b> selecting the token with
the highest precedence.<br>
<br>
For this reason, only tokens involved in a "lexical conflict" (e.g.
longest match conflict), need to have precedence. SableCC is able to
figure out natural precedence (e.g. language of token A is subset of
language of token B => A has natural precedence over B). Currently,
SableCC 4 requires a total order (explicit and/or implicit) among
conflicting tokens, yet a partial order where a single token wins over
all others is sufficient. This more flexible (partial order)
requirement will be implemented.<br>
<br>
Yet, "plus > id" already means that plus has precedence over id.
It's just that token precedence comes second, after the longest match.
So, don't think that we disagree about anything here.<br>
<br>
<blockquote cite="mid:20091116211031.GA91785@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">In the parser, precedence is a total order, and each precedence level
requires an associativity declaration. The lexer supports partial
ordering (actually, currently it requires a total order, but this will
change). The lexer has no concept of associativity, so ">" just gets the
job done.
SableCC 4 already has many new keywords. I've worked hard at reducing
their number to a minimum, while not making the language cryptic.
</pre>
</blockquote>
<pre wrap=""><!---->
Right, but consistency is also a design force. Different things should
have different syntax/names and same things the same syntax/names.
I'm not saying the current syntax is bad, but it's a point to be
considered.
</pre>
</blockquote>
<br>
The difference between Priority and Precedence in the my Oxford
Dictionary is not big. I am in favor of selecting a single keyword and
sticking to it. The difference in details, between lexer and parser
precedence is apparent in their different syntactic structure ( ">"
in the lexer, "Left" and "Right" in the parser).<br>
<br>
<blockquote cite="mid:20091116211031.GA91785@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">There's still one keyword I'm not sure about: "Separator" (usable in
lexers and parsers):
fun_call = id '(' (exp Separator ',')* ')';
I am toying with the idea of replacing it with '/' as is "divided by"
(which is an common definition for this operator):
fun_call = id '(' (exp / ',')* ')';
</pre>
</blockquote>
<pre wrap=""><!---->
The drawback of "/" is that one has to know what it means, as it isn't
self-evident at all, and not a standard syntax like the Kleene star.
Using the division operator "/" in the sense of "separated by" is
quite a far stretch, even as a mnemonic.
In a fluent[1] Java API for regular expressions I wrote, one can write
Regex expr = ...;
Regex args = zeroOrMore(expr).separatedBy(',');
So how about:
fun_call = id '(' exp* SeparatedBy ',' ')';
</pre>
</blockquote>
<br>
I actually had "Separated By" as two keywords, initially. The problem
is that this results in a tedious syntax for a common pattern; not
necessarily a good idea.<br>
<br>
<blockquote cite="mid:20091116211031.GA91785@nmhq.net" type="cite">
<pre wrap="">That would be more self-explanatory. Putting the separator "outside"
of the repition has some benefits: It doesn't require an extra set of
parentheses, it's arguably easier to read out loud in a comprehensible
way, it keeps the AST facts ("here we have a list of expressions")
together ("exp*"), and lastly the separator comes more expected when
the repetition has already been established (e.g.
"expr.separatedBy(',').zeroOrMore()" wouldn't be quite as readable).
</pre>
</blockquote>
<br>
The problem, in your syntax, is that we see a single ",", yet, in the
parse tree, there are as many "," as "exp" minus one. This is why I
decided that the * would be outside the parentheses.<br>
<br>
I see you coming:<br>
<pre> '(' (exp* SeparatedBy ','^(*-1) ) ')'
</pre>
But, then, is every expression followed by a bunch of commas? Nah...
I've thought about these cases and rejected them.<br>
<br>
<blockquote cite="mid:20091116211031.GA91785@nmhq.net" type="cite">Maybe
making "Sep" an alias would be acceptable:<br>
<pre wrap="">
fun_call = id '(' exp* Sep ',' ')';
A decent editor will have word completion of course. ;)
</pre>
</blockquote>
<br>
I don't like aliases; we have to chose something and stick with it.<br>
<br>
Users, when they see two different syntactic structures, expect a
distinct semantics. Aliases break this assumption, and users start
dreaming up arbitrary semantic differences...<br>
<br>
Currently, SableCC 4 has a single abbreviated keyword (and I wouldn't
mind making it a whole word): Diff => Difference. Sep is not an
intuitive abbreviation. We might as well use "/" rather than "Sep", if
we want to force users to read the documentation.<br>
<br>
The other thing, we shouldn't forget, is the context of use. A typical
use will look like:<br>
<pre> var_decl = 'var' (id / ',')+ ';'
</pre>
Question: If this was the first time you saw it, would you <i>guess</i>
the semantics of "/" correctly? If yes, I say we select "/". If no, we
select "Separator".<br>
<br>
And we revert "Diff" to "Difference" to remove the temptation of
proposing abbreviated keywords. :) <br>
<br>
Etienne<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26380631&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26380585Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T13:26:19Z2009-11-16T13:26:19ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Stephen,<br>
<br>
<blockquote cite="mid:1ae90d080911161149h46456ad6q565868318f35eb6f@mail.gmail.com" type="cite">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div bgcolor="#ffffff" text="#000000">Any opinion? Should I keep
the tedious but clear "Separator" keyword,
or should I replace it by the short '/' operator, potentially confusing
Lex/Flex converts?<br>
</div>
</blockquote>
</div>
<br>
For myself, I have a strong preference for '/' (or '$' or '%' or '#' if
'/' is found to be too confusing). This is an important abbreviating
notation, and making it too verbose will really compromise its utility.<br>
</blockquote>
Wouldn't '$' or '%' be even more cryptic? They don't have "natural
semantic". "%" is often used as "modulo" or "comment". '$' has so many
uses. In ObjectMacro, it is the command character. I'd like, as much as
possible, that grammars be readable by new users that haven't yet read
the (...upcoming...) documentation. At least, the natural semantics of
"/" is to divide: a list of identifier is divided by comma separators.<br>
<br>
I went through the list of ASCII symbols; to me "/" was the only
candidate as replacement for "Separator".<br>
<br>
<blockquote cite="mid:1ae90d080911161149h46456ad6q565868318f35eb6f@mail.gmail.com" type="cite">Honestly, I'm a little bothered by things like 'Diff',
too, since to me they compromise too far on 'expressionness'; I'd be
more inclined either to err on the side of line noise within
expressions, or to use something that stands out lexically (.diff.,
:diff:, $diff, or even DIFF).<br>
</blockquote>
<br>
There are very few Keyword operators: And, Diff, Look, Look Not,
Shortest, Longest, and (currently) Separator. These operators shouldn't
be of common use. Separator is the only one that will potentially be of
common use.<br>
<br>
Commonly used operators are expressed with symbols: - (lexical
subtraction), + * ? (multiplicity), ^N ^(N..M) ^(N...) exponent, etc.<br>
<br>
In particular, Diff should not be of common use, as lexical subtraction
almost always yields the expected result:<br>
<pre> not_f = ('a'..'z') - 'f'; // equivalent to ('a'..'z') Diff 'f'
comment_start = '/*';
comment_body = Any* - '*/';
comment_end = '/*';</pre>
<br>
<blockquote cite="mid:1ae90d080911161149h46456ad6q565868318f35eb6f@mail.gmail.com" type="cite">I guess I feel that making coarse structure apparent is a
greater kindness to the beginner and a smaller impediment to the expert
than verbosity. After all, if you don't know what an operator does,
you're going to look it up anyway. (And I think this is the idea behind
precedence declarations, too: to separate the coarse grammar from the
technical details.)<br>
<br>
Regards, and great to see it coming together,<br>
</blockquote>
<br>
Thanks for the comments.<br>
<br>
Etienne<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26380585&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26379574Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T13:10:31Z2009-11-16T13:10:31ZNiklas MatthiesHi,
<br><br>On Mon 2009-11-16 at 14:15h, Etienne M. Gagnon wrote on sablecc-discussion:
<div class='shrinkable-quote'><br>> Niklas Matthies wrote:
<br>> >>Just to make sure: I should use 'Precedence' to describe both "token
<br>> >>precedence" (in the lexer) and "operator precedence" (in the parser), as
<br>> >>in the following example. Right?
<br>> >>
<br>> >
<br>> >It's less common to apply the term "precedence" to tokens (actually
<br>> >it's not about the precedence of tokens here, but of token definitions!),
<br>> >but it isn't wrong either. So, why not.
<br>>
<br>> Given a string matched by N distinct tokens, we want to select the token
<br>> that has precedence over all other tokens. Of course, the implementation
<br>> can be done by "changing" token definitions so that only one token
<br>> matches a given string, yet, theoretically, all of the N tokens match.
</div><br>I'd say that it is input words that match, or are matched by, token
<br>definitions (which define sets of words), and the result of each match
<br>is a token. Of course you can say that
<br><br> plus > id
<br><br>means that each token that is the result of a successful match of
<br>"plus" has precedence over all tokens that are the result of a
<br>successful match of "id", but IMO it's more straightforward to say
<br>that the token definition "plus" has precedence over the token
<br>definition "id" for input words that match both definitions.
<br><br>Just like precedence declarations on the parser level define
<br>precedence of one production over another, not of syntax tree nodes
<br>over other syntax tree nodes.
<br><div class='shrinkable-quote'><br>> >One other thing I'm noticing here is that in the parser, the
<br>> >precedence levels are separated by ";", while here the precedence is
<br>> >marked by ">". This could be a reason to use different names for these
<br>> >sections or to align their syntax.
<br>>
<br>> In the parser, precedence is a total order, and each precedence level
<br>> requires an associativity declaration. The lexer supports partial
<br>> ordering (actually, currently it requires a total order, but this will
<br>> change). The lexer has no concept of associativity, so ">" just gets the
<br>> job done.
<br>>
<br>> SableCC 4 already has many new keywords. I've worked hard at reducing
<br>> their number to a minimum, while not making the language cryptic.
</div><br>Right, but consistency is also a design force. Different things should
<br>have different syntax/names and same things the same syntax/names.
<br>I'm not saying the current syntax is bad, but it's a point to be
<br>considered.
<br><br>> There's still one keyword I'm not sure about: "Separator" (usable in
<br>> lexers and parsers):
<br>>
<br>> fun_call = id '(' (exp Separator ',')* ')';
<br>>
<br>> I am toying with the idea of replacing it with '/' as is "divided by"
<br>> (which is an common definition for this operator):
<br>>
<br>> fun_call = id '(' (exp / ',')* ')';
<br><br>The drawback of "/" is that one has to know what it means, as it isn't
<br>self-evident at all, and not a standard syntax like the Kleene star.
<br>Using the division operator "/" in the sense of "separated by" is
<br>quite a far stretch, even as a mnemonic.
<br><br>In a fluent[1] Java API for regular expressions I wrote, one can write
<br><br> Regex expr = ...;
<br> Regex args = zeroOrMore(expr).separatedBy(',');
<br><br>So how about:
<br><br> fun_call = id '(' exp* SeparatedBy ',' ')';
<br><br>That would be more self-explanatory. Putting the separator "outside"
<br>of the repition has some benefits: It doesn't require an extra set of
<br>parentheses, it's arguably easier to read out loud in a comprehensible
<br>way, it keeps the AST facts ("here we have a list of expressions")
<br>together ("exp*"), and lastly the separator comes more expected when
<br>the repetition has already been established (e.g.
<br>"expr.separatedBy(',').zeroOrMore()" wouldn't be quite as readable).
<br><br>[1] <a href="http://martinfowler.com/bliki/FluentInterface.html" target="_top" rel="nofollow">http://martinfowler.com/bliki/FluentInterface.html</a><br><br>:
<br>> The separator is likely to be a commonly used operator in parsing
<br>> grammars, so making common things easy to type is not a bad idea.
<br><br>Maybe making "Sep" an alias would be acceptable:
<br><br> fun_call = id '(' exp* Sep ',' ')';
<br><br>A decent editor will have word completion of course. ;)
<br><br>-- Niklas Matthies
<br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26379574&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26379151Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T11:49:45Z2009-11-16T11:49:45ZStephen P SpackmanHi, Etienne.<br><br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div bgcolor="#ffffff" text="#000000">SableCC 4 already has many new keywords. I've worked hard at reducing
their number to a minimum, while not making the language cryptic.
There's still one keyword I'm not sure about: "Separator" (usable in
lexers and parsers):<br>
<pre> fun_call = id '(' (exp Separator ',')* ')';
</pre>
I am toying with the idea of replacing it with '/' as is "divided by"
(which is an common definition for this operat<div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div bgcolor="#ffffff" text="#000000">
<br>
<blockquote type="cite">
<pre>Incidentally, I've seen it argued that operator precedence shouldn't<br>necessarily be transitive (e.g. in [1]), which would support the ">"<br>notation for operator precedence. But I guess that in the SableCC<br>
context, non-transitive operator precedence wouldn't be able to<br>achieve non-ambiguity without changing (reducing) the accepted<br>language.<br> </pre>
</blockquote>
<br>
Yes, this would change the language. Interesting paper, though. Thanks
for the link!<br>
<br>
Etienne<br>
<br>
<pre cols="72">-- <br>Etienne M. Gagnon, Ph.D.<br>SableCC: <a href="http://sablecc.org/" target="_blank" rel="nofollow">http://sablecc.org</a><br></pre>
</div>
<br>_______________________________________________<br>
SableCC-Discussion mailing list<br>
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26379151&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a><br>
<a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_blank" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br>
<br></blockquote></div>or):<br>
<pre> fun_call = id '(' (exp / ',')* ')';
</pre>
The unfortunate thing is that "/" has been used by Lex and Flex as
synonym (!) for "Lookahead". I wouldn't want users to confuse "Look"
and "Separator"!<br>
<br>
The separator is likely to be a commonly used operator in parsing
grammars, so making common things easy to type is not a bad idea.<br>
<br>
Any opinion? Should I keep the tedious but clear "Separator" keyword,
or should I replace it by the short '/' operator, potentially confusing
Lex/Flex converts?<br></div></blockquote></div><br>For myself, I have a strong preference for '/' (or '$' or '%' or '#' if '/' is found to be too confusing). This is an important abbreviating notation, and making it too verbose will really compromise its utility.<br>
<br>Honestly, I'm a little bothered by things like 'Diff', too, since to me they compromise too far on 'expressionness'; I'd be more inclined either to err on the side of line noise within expressions, or to use something that stands out lexically (.diff., :diff:, $diff, or even DIFF).<br>
<br>I guess I feel that making coarse structure apparent is a greater kindness to the beginner and a smaller impediment to the expert than verbosity. After all, if you don't know what an operator does, you're going to look it up anyway. (And I think this is the idea behind precedence declarations, too: to separate the coarse grammar from the technical details.)<br>
<br>Regards, and great to see it coming together,<br>Stephen<br>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26379151&i=1" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26377911Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T11:15:13Z2009-11-16T11:15:13ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Niklas,<br>
<br>
Niklas Matthies wrote:<br>
<blockquote cite="mid:20091116182851.GA75434@nmhq.net" type="cite">
<blockquote type="cite">
<pre wrap="">Just to make sure: I should use 'Precedence' to describe both "token
precedence" (in the lexer) and "operator precedence" (in the parser), as
in the following example. Right?
</pre>
</blockquote>
<pre wrap=""><!---->
It's less common to apply the term "precedence" to tokens (actually
it's not about the precedence of tokens here, but of token definitions!),
but it isn't wrong either. So, why not.
</pre>
</blockquote>
<br>
Given a string matched by N distinct tokens, we want to select the
token that has precedence over all other tokens. Of course, the
implementation can be done by "changing" token definitions so that only
one token matches a given string, yet, theoretically, all of the N
tokens match.<br>
<br>
<blockquote cite="mid:20091116182851.GA75434@nmhq.net" type="cite">
<pre wrap="">Do I understand this correctly that the above has the same effect
as if 'id' had been defined as
id = letter (letter | digit)* - (plus | minus | star | slash);
?
</pre>
</blockquote>
<br>
Almost. In SableCC 4, the "-" operator is a lexical subtraction
operator, not a regular expression difference operator (or set
difference). So, to be precise, it would be equivalent to:<br>
<pre> id = letter (letter | digit)* Diff (plus | minus | star | slash);
</pre>
<br>
The "-" lexical subtraction operator has been explained in
<a class="moz-txt-link-freetext" href="http://lists.sablecc.org/pipermail/sablecc-discussion/msg00578.html" target="_top" rel="nofollow">http://lists.sablecc.org/pipermail/sablecc-discussion/msg00578.html</a><br>
<br>
<blockquote cite="mid:20091116182851.GA75434@nmhq.net" type="cite">
<pre wrap="">One other thing I'm noticing here is that in the parser, the
precedence levels are separated by ";", while here the precedence is
marked by ">". This could be a reason to use different names for these
sections or to align their syntax.
</pre>
</blockquote>
<br>
In the parser, precedence is a total order, and each precedence level
requires an associativity declaration. The lexer supports partial
ordering (actually, currently it requires a total order, but this will
change). The lexer has no concept of associativity, so ">" just gets
the job done.<br>
<br>
<br>
SableCC 4 already has many new keywords. I've worked hard at reducing
their number to a minimum, while not making the language cryptic.
There's still one keyword I'm not sure about: "Separator" (usable in
lexers and parsers):<br>
<pre> fun_call = id '(' (exp Separator ',')* ')';
</pre>
I am toying with the idea of replacing it with '/' as is "divided by"
(which is an common definition for this operator):<br>
<pre> fun_call = id '(' (exp / ',')* ')';
</pre>
The unfortunate thing is that "/" has been used by Lex and Flex as
synonym (!) for "Lookahead". I wouldn't want users to confuse "Look"
and "Separator"!<br>
<br>
The separator is likely to be a commonly used operator in parsing
grammars, so making common things easy to type is not a bad idea.<br>
<br>
Any opinion? Should I keep the tedious but clear "Separator" keyword,
or should I replace it by the short '/' operator, potentially confusing
Lex/Flex converts?<br>
<br>
<blockquote cite="mid:20091116182851.GA75434@nmhq.net" type="cite">
<pre wrap="">
Incidentally, I've seen it argued that operator precedence shouldn't
necessarily be transitive (e.g. in [1]), which would support the ">"
notation for operator precedence. But I guess that in the SableCC
context, non-transitive operator precedence wouldn't be able to
achieve non-ambiguity without changing (reducing) the accepted
language.
</pre>
</blockquote>
<br>
Yes, this would change the language. Interesting paper, though. Thanks
for the link!<br>
<br>
Etienne<br>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26377911&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26377200Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T10:28:51Z2009-11-16T10:28:51ZNiklas MatthiesOn Mon 2009-11-16 at 10:36h, Etienne M. Gagnon wrote on sablecc-discussion:
<br>> Hi Christopher,
<br>:
<br>> >Regarding the 'Priority' versus 'Precedence" suggestion posed by
<br>> >Niklas, I would have to agree that 'Precedence' is a more natural word
<br>> >to describe the behavior in English. That's the term we're taught to
<br>> >describe it in CS courses, so it's probably best to stick with it.
<br>>
<br>> Just to make sure: I should use 'Precedence' to describe both "token
<br>> precedence" (in the lexer) and "operator precedence" (in the parser), as
<br>> in the following example. Right?
<br><br>It's less common to apply the term "precedence" to tokens (actually
<br>it's not about the precedence of tokens here, but of token definitions!),
<br>but it isn't wrong either. So, why not.
<br><div class='shrinkable-quote'><br>> Language demo;
<br>>
<br>> Lexer
<br>>
<br>> id = letter (letter | digit)*;
<br>> num = digit+;
<br>>
<br>> letter = 'a'..'z' | 'A'..'Z';
<br>> digit = '0'..'9';
<br>>
<br>> plus = '+' | 'add';
<br>> minus = '-' | 'sub';
<br>> star = '*' | 'mul';
<br>> slash = '/' | 'div';
<br>>
<br>> Ignored
<br>> ' ', #9, #10, #13;
<br>>
<br>> Precedence
<br>> plus > id;
<br>> minus > id;
<br>> star > id;
<br>> slash > id;
</div><br>Do I understand this correctly that the above has the same effect
<br>as if 'id' had been defined as
<br><br> id = letter (letter | digit)* - (plus | minus | star | slash);
<br><br>?
<br><br>One other thing I'm noticing here is that in the parser, the
<br>precedence levels are separated by ";", while here the precedence is
<br>marked by ">". This could be a reason to use different names for these
<br>sections or to align their syntax.
<br><br>Incidentally, I've seen it argued that operator precedence shouldn't
<br>necessarily be transitive (e.g. in [1]), which would support the ">"
<br>notation for operator precedence. But I guess that in the SableCC
<br>context, non-transitive operator precedence wouldn't be able to
<br>achieve non-ambiguity without changing (reducing) the accepted
<br>language.
<br><br>-- Niklas Matthies
<br><br>[1] <a href="http://www.cs.nott.ac.uk/~nad/publications/danielsson-norell-mixfix.pdf" target="_top" rel="nofollow">http://www.cs.nott.ac.uk/~nad/publications/danielsson-norell-mixfix.pdf</a><br><br>_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26377200&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26374129Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T07:36:24Z2009-11-16T07:36:24ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Christopher,<br>
<br>
<blockquote cite="mid:5138344FE8C840EF9A4C1244B45545A8@FDC490" type="cite">
<div class="Section1">
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">I had a
brief look at this over the
weekend with a view to making a C# backend and got stuck with
ObjectMacro. It
appears that we need to the names and meaning of the various hooks that
SableCC
calls it with, but these didn’t really leap out at me looking at your
Java template file, the samples, or the post expansion version.<br>
Also, the new syntax seems to be
profoundly different from the old one, which is not a problem really
except
that it’s not clear what some of the new keywords mean or do. Is there
document
somewhere that discusses them? The ones in particular that had me
scratching my
head were:<br>
Transformation, Tree, Restartable,
Investigator, Dangling, Any and Epsilon.</span></font></p>
</div>
</blockquote>
<br>
The current code generation is not in ready to start targeting other
languages. I have taken many shortcuts in order to make the parsing
engine available for testing.<br>
<br>
Currently, the code directly generates Java. This will be completely
changed. The real SableCC 4 backend will first generate "intermediate"
code/tree. This intermediate tree will be used as source for generating
other targets (including Java). This is how the ObjectMacro backend is
already implemented.<br>
<br>
Too many SableCC 4 features are missing at this point (tree
transformations, investigators, selectors, contexts, etc.). It is too
early to start targeting other languages.<br>
<br>
<blockquote cite="mid:5138344FE8C840EF9A4C1244B45545A8@FDC490" type="cite">
<div class="Section1">
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Regarding
the ‘Priority’
versus ‘Precedence” suggestion posed by Niklas, I would have to
agree that ‘Precedence’ is a more natural word to describe the
behavior in English. That’s the term we’re taught to describe it in
CS courses, so it’s probably best to stick with it.</span></font></p>
</div>
</blockquote>
<br>
Just to make sure: I should use 'Precedence' to describe both "token
precedence" (in the lexer) and "operator precedence" (in the parser),
as in
the following example. Right?<br>
<br>
<pre>Language demo;
Lexer
id = letter (letter | digit)*;
num = digit+;
letter = 'a'..'z' | 'A'..'Z';
digit = '0'..'9';
plus = '+' | 'add';
minus = '-' | 'sub';
star = '*' | 'mul';
slash = '/' | 'div';
Ignored
' ', #9, #10, #13;
Precedence
plus > id;
minus > id;
star > id;
slash > id;
Parser
exp =
{add:} exp plus exp |
{sub:} exp minus exp |
{mul:} exp star exp |
{div:} exp slash exp |
{par:} '(' exp ')' |
{num:} num;
Precedence
Left mul, div;
Left add, sub;
</pre>
<br>
I just noticed that this example reveals a little semantic verification
discrepancy (SableCC wants me to explicitly declare tokens that should
be known implicitly)...<br>
<blockquote cite="mid:5138344FE8C840EF9A4C1244B45545A8@FDC490" type="cite">
<div class="Section1">
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Needless to
say it’s exciting to see
it nearing completion.</span></font></p>
</div>
</blockquote>
<br>
I know. But, there's still a fair amount of work to do to put all the
pieces together.<br>
<br>
Etienne<br>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26374129&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26369692RE: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-16T02:21:13Z2009-11-16T02:21:13ZChristopher Van Kirk<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 10 (filtered)">
</head>
<body bgcolor=white lang=EN-US link=blue vlink=blue>
<div class=Section1>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Hi Etienne,</span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>I had a brief look at this over the
weekend with a view to making a C# backend and got stuck with ObjectMacro. It
appears that we need to the names and meaning of the various hooks that SableCC
calls it with, but these didn’t really leap out at me looking at your
Java template file, the samples, or the post expansion version. </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Also, the new syntax seems to be
profoundly different from the old one, which is not a problem really except
that it’s not clear what some of the new keywords mean or do. Is there document
somewhere that discusses them? The ones in particular that had me scratching my
head were:</span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Transformation, Tree, Restartable,
Investigator, Dangling, Any and Epsilon.</span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Regarding the ‘Priority’
versus ‘Precedence” suggestion posed by Niklas, I would have to
agree that ‘Precedence’ is a more natural word to describe the
behavior in English. That’s the term we’re taught to describe it in
CS courses, so it’s probably best to stick with it.</span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Needless to say it’s exciting to see
it nearing completion. </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Cheers,</span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Chris…</span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'> </span></font></p>
<p class=MsoNormal><font size=2 color=black face=Tahoma><span style='font-size:
10.0pt;font-family:Tahoma;color:windowtext'>-----Original Message-----<br>
<b><span style='font-weight:bold'>From:</span></b>
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26369692&i=0" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>
[mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26369692&i=1" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>]
<b><span style='font-weight:bold'>On Behalf Of </span></b>Etienne M. Gagnon<br>
<b><span style='font-weight:bold'>Sent:</span></b> Friday, November 13, 2009
11:29 AM<br>
<b><span style='font-weight:bold'>To:</span></b> Discussion mailing list for
the SableCC project.<br>
<b><span style='font-weight:bold'>Subject:</span></b> Linear-approximate
LALR(K) and ambiguity resolution for expressions</span></font></p>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span style='font-size:12.0pt'> </span></font></p>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span style='font-size:12.0pt'>Hi!<br>
<br>
I've just released SableCC 4-beta.2. This version is a major milestone as it
generates a complete framework: lexer, parser, syntax tree, and tree walker.
Among the most interesting features is the new linear-approximate LALR(K)
engine which also support safe ambiguity resolution for expressions (i.e. left
and right directly-recursive productions). Yes, you can now write the simple
expression grammar as:</span></font></p>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Language exp;</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Lexer</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> num = digit+;</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> digit = '0'..'9';</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> Ignored</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> ' ', #9, #10, #13;</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Parser</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> exp =</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> {add:} exp '+' exp |</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> {mul:} exp '*' exp |</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> {num:} num;</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> Priority // highest priority first</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> Left mul; // "Left" associative </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> Left add;</span></font></pre>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span style='font-size:12.0pt'>SableCC is very precise in resolving an <i><span style='font-style:italic'>ambiguity</span></i> according to a priority
specification. It will not accidentally resolve unrelated <i><span style='font-style:italic'>conflicts</span></i>. The <i><span style='font-style:
italic'>language</span></i> (i.e. the set of syntactically valid programs)
accepted by the grammar is not changed by the priority specification. This
differs from traditional <i><span style='font-style:italic'>conflict-resolution</span></i>
mechanisms found in other tools that can change the language and
unintentionally resolve unrelated conflicts.<br>
<br>
It would be fun if you helped testing this version by rewriting some of your
SableCC 2 and 3 grammars into SableCC 4 syntax, in order to test the
generated parser and report back on it.<br>
<br>
Please be indulgent with me... The parser grammar does not, at this point,
support any lexical operators (*, +, ?, etc.). So, you'll have to manually
write your lists as recursive productions. This is temporary as I first need to
implement the tree transformation engine before supporting these operators.<br>
<br>
You'll notice that I have somewhat simplified the tree walker class and related
classes. It is now simply called "Walker" (instead of
DepthFirstAdapter).<br>
<br>
You can download it from <a href="http://sablecc.org/wiki/DownloadPage" target="_top" rel="nofollow">http://sablecc.org/wiki/DownloadPage</a>
.<br>
<br>
I hope you'll have as much fun as I am starting to have with the new syntax and
lexer/parser engines.<br>
<br>
Etienne<br>
<br>
<br>
</span></font></p>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>-- </span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Etienne M. Gagnon, Ph.D.</span></font></pre><pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>SableCC: <a href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a></span></font></pre></div>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26369692&i=2" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26363008Re: Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-15T12:24:06Z2009-11-15T12:24:06ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Niklas,<br>
<br>
See below.<br>
<br>
Niklas Matthies wrote:
<blockquote cite="mid:20091115000538.GA18266@nmhq.net" type="cite">
<pre wrap="">Hi,
On Thu 2009-11-12 at 21:29h, Etienne M. Gagnon wrote on sablecc-discussion:
:
</pre>
<blockquote type="cite">
<pre wrap="">I've just released SableCC 4-beta.2. This version is a major milestone
as it generates a complete framework: lexer, parser, syntax tree, and
tree walker. Among the most interesting features is the new
linear-approximate LALR(K) engine which also support safe ambiguity
resolution for expressions (i.e. left and right directly-recursive
productions).
</pre>
</blockquote>
<pre wrap=""><!---->:
</pre>
<blockquote type="cite">
<pre wrap=""> exp =
{add:} exp '+' exp |
{mul:} exp '*' exp |
{num:} num;
Priority // highest priority first
Left mul; // "Left" associative
Left add;
</pre>
</blockquote>
<pre wrap=""><!---->
That really sounds great! Unfortunately I'm not sure if and when I
will have the spare time to do some real testing. Anyway, some
questions:
- How would one define groups of operators (productions) having the
same priority (and associativity)?
Like for example '+' and '-' having the same precedence.
(By the way, isn't "precedence" the more customary term for this than
"priority"?)
</pre>
</blockquote>
You would write:<br>
<pre> Left add, sub;</pre>
The keyword "Priority" is also used in the lexer. We could change the
keyword to "Precedence", as long as we also use it to determine token
priority. I just don't want distinct keywords. Priority seems shorter
and as clear as Precedence (that's would be one point in favor of
keeping
it).<br>
<br>
<blockquote cite="mid:20091115000538.GA18266@nmhq.net" type="cite">
<pre wrap="">- For one-sided operators (like unary operators), associativity isn't
applicable, but one still wants to specify precedence. Is this
supported? By "one-sided" I mean operators like
'if' expr 'then' expr 'else' expr
as opposed to "two-sided" operators like
expr '?' expr ':' expr
Are there maybe better terms for this distinction?
</pre>
</blockquote>
<br>
Actually, with unary operators you still have to decide whether you
want to give left or right priority, e.g.:<br>
<br>
<pre>++ a - b (assuming ++ and - have the same precedence level) =>
left : (++ a) - b
right : ++ (a - b)
</pre>
So, of course, you simply specify them using the same syntax:<br>
<pre> Right unop1, otherop, unop2;
</pre>
This allows for binary, left-unary, and right-unary operators at the
same priority level. So, it is simpler to keep "Left" and "Right"
specifications for each priority level.<br>
<br>
<blockquote cite="mid:20091115000538.GA18266@nmhq.net" type="cite">
<pre wrap="">
- Sometimes there are two-sided operators that you don't want to be
associative, either because it would be ill-typed, or because it would
be prone to misunderstandings, so you want users to use parentheses to
be explicit. An example would be an expression like "a < b < c", where
'<' is a less-than operator that doesn't take booleans, or if it does
could then be misunderstood as meaning "a < b and b < c". I know, in
general this is better handled at the analysis level, but sometimes
it's convenient to just let the grammar take care of it.
</pre>
</blockquote>
<br>
This would change the accepted language. So, given that SableCC 4
guarantees that expression priorities does not change the language, you
have two
choices:<br>
<ol>
<li>Detect invalid constructs at the semantic level, as you proposed,
or</li>
<li>Add distinct productions for non-associative operators.</li>
</ol>
Of course, 2 requires a little grammar rewriting, but my experience is
that it is relatively easy to accomplish. As for 1, it has the
advantage of leading to clearer semantic error messages.<br>
<br>
<blockquote cite="mid:20091115000538.GA18266@nmhq.net" type="cite">
<pre wrap="">
- Are n-ary operators with n > 2 supported? For example I have a
grammar which includes these operators (the last three are list
comprehension expressions):
expr =
...
{cond:} 'if' expr 'then' expr 'else' expr
{filter:} id 'in' expr 'where' expr
{map:} expr 'for' id 'in' expr
{map_filter:} expr 'for' id 'in' expr 'where' expr
with the following precedence/associativity table:
associativity operator
------------------------------------------------
n/a cond, filter
left map, map_filter
I.e. cond and filter have the same precedence and aren't associative,
but have higher precedence than map and map_filter, which again have
the same precedence and are left associative. Is the ambiguity
resolution able to handle such more complex scenarios?
(It took me quite a while to write a correct SableCC 3 grammar for it
back then. ;))
</pre>
</blockquote>
<br>
I would write: <br>
<pre wrap="">expr =
{cond:} 'if' subexpr 'then' subexpr 'else' subexpr |
{filter:} id 'in' subexpr 'where' subexpr |
{simple:} subexpr;
subexpr
{map:} subexpr 'for' id 'in' subexpr |
{map_filter:} subexpr 'for' id 'in' subexpr 'where' subexpr |
{parenthesis:} '(' expr ')'
...
Precedence
Left map, map_filter;
</pre>
In the cond and filter alternatives, you can put the internal recursion
on "expr" instead of "subexpr"; this would give you the same result as
you would get with a "non-associative" specification. Yet, I wouldn't
be surprised
if your intentions were actually to disallow : <tt>if if a else if b
else c else if d else e</tt>. The above grammar disallows it; you
would have to use parentheses to express it. A "non-associative"
precedence
simply disallows left and right recursion, it does not affect internal
recursion.<br>
<br>
<blockquote cite="mid:20091115000538.GA18266@nmhq.net" type="cite">
<pre wrap="">
Thanks,
-- Niklas Matthies
</pre>
</blockquote>
Thanks for the feedback!<br>
<br>
Etienne<br>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26363008&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-26330405Linear-approximate LALR(K) and ambiguity resolution for expressions2009-11-12T18:29:03Z2009-11-12T18:29:03ZEtienne M. Gagnon<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
</head>
<body bgcolor="#ffffff" text="#000000">
Hi!<br>
<br>
I've just released SableCC 4-beta.2. This version is a major milestone
as it generates a complete framework: lexer, parser, syntax tree, and
tree walker. Among the most interesting features is the new
linear-approximate LALR(K) engine which also support safe ambiguity
resolution for expressions (i.e. left and right directly-recursive
productions). Yes, you can now write the simple expression grammar as:<br>
<pre>Language exp;
Lexer
num = digit+;
digit = '0'..'9';
Ignored
' ', #9, #10, #13;
Parser
exp =
{add:} exp '+' exp |
{mul:} exp '*' exp |
{num:} num;
Priority // highest priority first
Left mul; // "Left" associative
Left add;
</pre>
SableCC is very precise in resolving an <i>ambiguity</i> according to
a priority specification. It will not accidentally resolve unrelated <i>conflicts</i>.
The <i>language</i> (i.e. the set of syntactically valid programs)
accepted by the grammar is not changed by the priority specification.
This differs from traditional <i>conflict-resolution</i> mechanisms
found in other tools that can change the language and unintentionally
resolve unrelated conflicts.<br>
<br>
It would be fun if you helped testing this version by rewriting some of
your SableCC 2 and 3 grammars into SableCC 4 syntax, in order to test
the generated parser and report back on it.<br>
<br>
Please be indulgent with me... The parser grammar does not, at this
point, support any lexical operators (*, +, ?, etc.). So, you'll have
to manually write your lists as recursive productions. This is
temporary as I first need to implement the tree transformation engine
before supporting these operators.<br>
<br>
You'll notice that I have somewhat simplified the tree walker class and
related classes. It is now simply called "Walker" (instead of
DepthFirstAdapter).<br>
<br>
You can download it from <a class="moz-txt-link-freetext" href="http://sablecc.org/wiki/DownloadPage" target="_top" rel="nofollow">http://sablecc.org/wiki/DownloadPage</a> .<br>
<br>
I hope you'll have as much fun as I am starting to have with the new
syntax and lexer/parser engines.<br>
<br>
Etienne<br>
<br>
<pre class="moz-signature" cols="72">--
Etienne M. Gagnon, Ph.D.
SableCC: <a class="moz-txt-link-freetext" href="http://sablecc.org" target="_top" rel="nofollow">http://sablecc.org</a>
</pre>
</body>
</html>
<br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26330405&i=0" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>tag:old.nabble.com,2006:post-25836432Re: Fixing Parser Weirdness2009-10-10T09:18:13Z2009-10-10T09:18:13Ztwashing<html><head></head><body><div style="font-family:arial,helvetica,sans-serif;font-size:10pt"><div>Ok great. I got this problem sorted out. A long haul, but looking at the token stream definitely helped. <br><br>Thanks a bunch Chris<br>Tim<br><br></div><div style="font-family: arial,helvetica,sans-serif; font-size: 10pt;"><br><div style="font-family: times new roman,new york,times,serif; font-size: 12pt;"><font face="Tahoma" size="2"><hr size="1"><b><span style="font-weight: bold;">From:</span></b> Christopher Van Kirk <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=0" target="_top" rel="nofollow">chris.vankirk@...</a>><br><b><span style="font-weight: bold;">To:</span></b> Discussion mailing list for the SableCC project <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=1" target="_top" rel="nofollow">sablecc-discussion@...</a>><br><b><span style="font-weight: bold;">Sent:</span></b> Thu, October 8, 2009 10:39:39 PM<br><b><span style="font-weight: bold;">Subject:</span></b> RE: Fixing Parser Weirdness<br></font><br>
<div class="Section1">
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";">This is a useful gem from
Indrek Mandre that prints out the lexer stream (my version is for .net but you
can translate easily). Use this then analyse the lexer output and see if you
can match it to your syntax. I suspect you’re getting tokens you don’t expect.</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> </span></font></p>
<p class="MsoNormal" style=""><font color="blue" face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New"; color: blue;">class</span></font><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> <font color="#2b91af"><span style="color: rgb(43, 145, 175);">TestLexer</span></font></span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";">{</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> <font color="blue"><span style="color: blue;">public</span></font> <font color="blue"><span style="color: blue;">static</span></font> <font color="blue"><span style="color: blue;">void</span></font> </span></font><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";">Main</span></font><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";">(<font color="#2b91af"><span style="color: rgb(43, 145, 175);">String</span></font>[] args)</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> {</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> Lexer l = <font color="blue"><span style="color: blue;">new</span></font> Lexer(<font color="#2b91af"><span style="color: rgb(43, 145, 175);">Console</span></font>.In);</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> <font color="blue"><span style="color: blue;">while</span></font> (<font color="blue"><span style="color: blue;">true</span></font>)</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> {</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> Token token =
l.Next();</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> <font color="#2b91af"><span style="color: rgb(43, 145, 175);">Console</span></font>.WriteLine (<font color="#a31515"><span style="color: rgb(163, 21, 21);">"Read token '"</span></font> +
token.GetType().Name +</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> <font color="#a31515"><span style="color: rgb(163, 21, 21);">"', Text = ["</span></font>
+ token.Text + <font color="#a31515"><span style="color: rgb(163, 21, 21);">"] at
["</span></font> + token.Line +</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> <font color="#a31515"><span style="color: rgb(163, 21, 21);">","</span></font> +
token.Pos + <font color="#a31515"><span style="color: rgb(163, 21, 21);">"]"</span></font>);</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> <font color="blue"><span style="color: blue;">if</span></font> ( token <font color="blue"><span style="color: blue;">is</span></font> EOF ) <font color="blue"><span style="color: blue;">break</span></font>;</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> }</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";"> }</span></font></p>
<p class="MsoNormal" style=""><font face="Courier New" size="2"><span style="font-size: 10pt; font-family: "Courier New";">}</span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<p class="MsoNormal"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">-----Original Message-----<br>
<b><span style="font-weight: bold;">From:</span></b>
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=2" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>
[mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=3" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>]
<b><span style="font-weight: bold;">On Behalf Of </span></b>Timothy Washington<br>
<b><span style="font-weight: bold;">Sent:</span></b> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">Friday, October
09, 2009</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">5:37 AM</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"><br>
<b><span style="font-weight: bold;">To:</span></b> Discussion mailing list for
the SableCC project<br>
<b><span style="font-weight: bold;">Subject:</span></b> Re: Fixing Parser
Weirdness</span></font></p>
<p class="MsoNormal"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><font face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial;">Hey there, thanks for the feedback.
Agreed, I think that if I solve 'A', thenthe rest will fall into place. What I
am spitting out, the stream as you put it, is the "<b><span style="font-weight: bold;">action array / lexer.peek() / state</span></b>"
as seen from the Parser's point of view. The output was in bold red in fig. 2
(logs). Line 98 in fig. 1 was what generated that error (everything works if I
remove 98). And see line 166 from the Parser.parse method below. That's where
I'm putting the debug information. But there's probably a better way of
isolating the error. <br>
<br>
<br>
Parser.java (client won't let me attach java file) <br>
...<br>
<font color="#737373"><span style="color: rgb(115, 115, 115);"> 112
@SuppressWarnings("unchecked")<br>
113 public Start parse() throws ParserException,
LexerException, IOException<br>
114 {<br>
115 push(0, null, true);<br>
116 List<Node> ign
= null;<br>
117 while(true)<br>
118 {<br>
119
while(index(this.lexer.peek()) == -1)<br>
120
{<br>
121
if(ign == null)<br>
122
{<br>
123
ign = new LinkedList<Node>();<br>
124
}<br>
125 <br>
126
ign.add(this.lexer.next());<br>
127
}<br>
128 <br>
</span></font><span style=""> 129
if(ign != null)</span><font color="#737373"><span style="color: rgb(115, 115, 115);"><br>
130
{<br>
131
this.ignoredTokens.setIn(this.lexer.peek(), ign);<br>
132
ign = null;<br>
133
}<br>
</span></font><span style=""> 134 </span><font color="#737373"><span style="color: rgb(115, 115, 115);"><br>
135
this.last_pos = this.lexer.peek().getPos();<br>
136
this.last_line = this.lexer.peek().getLine();<br>
137
this.last_token = this.lexer.peek();<br>
138 <br>
139
int index = index(this.lexer.peek());<br>
140
this.action[0] = Parser.actionTable[state()][0][1];<br>
141
this.action[1] = Parser.actionTable[state()][0][2];<br>
142 <br>
143
int low = 1;<br>
144
int high = Parser.actionTable[state()].length - 1;</span></font><br style="">
<font color="#737373"><span style="color: rgb(115, 115, 115);"> 145 <br>
146
while(low <= high)<br>
147
{<br>
148
int middle = (low + high) / 2;<br>
149 <br>
150
if(index < Parser.actionTable[state()][middle][0])<br>
151
{<br>
152
high = middle - 1;<br>
153
}<br>
154
else if(index > Parser.actionTable[state()][middle][0])<br>
155
{<br>
156
low = middle + 1;<br>
157
}<br>
158
else<br>
159
{<br>
160
this.action[0] = Parser.actionTable[state()][middle][1];<br>
161
this.action[1] = Parser.actionTable[state()][middle][2];<br>
162
break;<br>
163
}<br>
164
}<br>
165 <br>
</span></font><b><font color="black"><span style="color: black; font-weight: bold;"> 166
System.out.println("debugging... action["+this.action[0]+"] /
lexer.peek["+this.lexer.peek()+"] /
state["+state()+"]");<br>
</span></font></b><font color="#737373"><span style="color: rgb(115, 115, 115);"> 167 <br>
168
switch(this.action[0])<br>
</span></font><span style=""> 169
{</span><font color="#737373"><span style="color: rgb(115, 115, 115);"><br>
170
case SHIFT:<br>
171
{</span></font><br>
<br>
<br>
Thanks<br>
Tim</span></font></p>
</div>
<div>
<p class="MsoNormal"><font face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial;"> </span></font></p>
<div>
<div class="MsoNormal" style="text-align: center;" align="center"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">
<hr align="center" size="1" width="100%">
</span></font></div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma; font-weight: bold;">From:</span></font></b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">
Christopher Van Kirk <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=4" target="_top" rel="nofollow">chris.vankirk@...</a>><br>
<b><span style="font-weight: bold;">To:</span></b> Discussion mailing list for
the SableCC project <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=5" target="_top" rel="nofollow">sablecc-discussion@...</a>><br>
<b><span style="font-weight: bold;">Sent:</span></b> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">Wed, October 7,
2009</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">8:18:22 PM</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"><br>
<b><span style="font-weight: bold;">Subject:</span></b> RE: Fixing Parser
Weirdness</span></font></p>
<div>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">Have you tried the approach to solving ‘A’
that I suggested in my last posting? If you have and you’re still confused then
post the token stream emitted by your lexer and we can go from there.</span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;">In solving A you may acquire the skills
you need to solve B, so I think you should just let B slide for the moment. </span></font></p>
<p class="MsoNormal"><font color="navy" face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial; color: navy;"> </span></font></p>
<p class="MsoNormal"><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">-----Original Message-----<br>
<b><span style="font-weight: bold;">From:</span></b>
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=6" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>
[mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=7" target="_top" rel="nofollow">sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...</a>]
<b><span style="font-weight: bold;">On Behalf Of </span></b>Timothy Washington<br>
<b><span style="font-weight: bold;">Sent:</span></b> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">Thursday,
October 08, 2009</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"> </span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;">7:50 AM</span></font><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"><br>
<b><span style="font-weight: bold;">To:</span></b> Discussion mailing list for
the SableCC project<br>
<b><span style="font-weight: bold;">Subject:</span></b> Re: Fixing Parser
Weirdness</span></font></p>
<p class="MsoNormal"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><font face="Times New Roman" size="2"><span style="font-size: 10pt;">Hey Chris, thanks for
responding to this problem thus far. I've been tied up on other projects, but
have come back to these problems. Right now I want to do 2 things: <br>
<br>
A) Be able to add a new token (that provides an extra option in the language,
-returninput). <br>
B) Allow special characters in an XML block (<mytag value='<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=8" target="_top" rel="nofollow">twashing@...</a>' />). <br>
<br>
So looking at problem A), I actually was printing out the characters the Parser
was picking from the Lexer. If I add the token on line 98 (see fig. 1), I get
the error in fig. 2. Normally, using the expression "<i><font color="#737373"><span style="color: rgb(115, 115, 115); font-style: italic;">login ( user
-username root -password password );</span></font></i>" works fine. But
just with line 98, I get the error that the Parser / Lexer is expecting an
opening bracket "(", which is obviously there. So now the question is
i) what does it think the "<b><i><font color="#737373"><span style="color: rgb(115, 115, 115); font-weight: bold; font-style: italic;">login</span></font></i></b></span></font><b><i><font color="black" size="4"><span style="font-size: 13.5pt; color: black; font-weight: bold; font-style: italic;">(</span></font></i></b><b><i><font color="#737373" size="2"><span style="font-size: 10pt; color: rgb(115, 115, 115); font-weight: bold; font-style: italic;">...</span></font></i></b><font size="2"><span style="font-size: 10pt;">" is and ii) how does line 98 change
a working token to this broken state? I've also re-attached my sablecc file. <br>
<br>
<br>
<i><font color="#737373"><span style="color: rgb(115, 115, 115); font-style: italic;">97
currency_opt = '-currency' ws+ ( lowercase | uppercase |
dash | colon_helper | ats | underscore | digit | dot )*;<br>
98 returninput_opt = '-returninput' ws+ ( lowercase
| uppercase | dash | colon_helper | ats | underscore | digit | dot )*;</span></font></i><br>
fig. 1 - bkeeping.cc sablecc file <br>
<br>
<br>
<i><font color="#737373"><span style="color: rgb(115, 115, 115); font-style: italic;">login (
user -username root -password password ); <br>
</span></font><b><font color="red"><span style="color: red; font-weight: bold;">debugging...
action[0] / lexer.peek[login ] / state[0]<br>
debugging... action[3] / lexer.peek[( ] / state[12]</span></font></b><font color="#737373"><span style="color: rgb(115, 115, 115);"><br>
ERROR [Thread-3] (Bkell.java:187) - [14,7] expecting: lbracket<br>
com.interrupt.bookkeeping.cc.parser.ParserException: [14,7] expecting: lbracket<br>
at com.interrupt.bookkeeping.cc.parser.Parser.parse(Parser.java:1024)<br>
at com.interrupt.bookkeeping.cc.bkell.Bkell.run(Bkell.java:146)<br>
at java.lang.Thread.run(Thread.java:637)<br>
ERROR [Thread-3] (Util.java:96) - generateBkellException CALLED<br>
ERROR [Thread-3] (Util.java:102) - TOP error message[[14,7] expecting:
lbracket]<br>
ERROR [Thread-3] (Util.java:103) - ROOT error message[[14,7] expecting:
lbracket]<br>
ERROR [Thread-3] (Bkell.java:189) - <logs xmlns='com/interrupt/logs' id=''
><log xmlns='com/interrupt/logs' level='ERROR' id='' ><logMessages
xmlns='com/interrupt/logs' id='' ><logMessage xmlns='com/interrupt/logs'
id='' >[14,7] expecting: lbracket</logMessage><br>
</logMessages><br>
</span></font></log></i><br>
</logs><br>
fig. 2 - logs <br>
</span></font><br>
<br>
<font size="2"><span style="font-size: 10pt;">Thanks<br>
Tim<br>
</span></font><br>
<br>
<br>
----- Original Message ----<br>
From: Christopher Van Kirk <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=9" target="_top" rel="nofollow">chris.vankirk@...</a>><br>
To: Discussion mailing list for the SableCC project <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=10" target="_top" rel="nofollow">sablecc-discussion@...</a>><br>
Sent: Thu, September 3, 2009 10:47:11 PM<br>
Subject: Re: Fixing Parser Weirdness<br>
<br>
When I have a problem like this, generally I'll print out the token stream for
the affected area and try to match it to the production I think the parser
should choose. When I say print out the token stream, I mean print out the
actual stream as generated by the lexer.<br>
<br>
If you do that, I think you will find that you either have grammatical or
lexical problems (either the token stream isn't what you think it is, or the
syntax isn't what you want it to be). It's good to get into practice doing this
sort of thing because this will come up from time to time and it's best to be
able to resolve it by yourself.<br>
<br>
Also note that SableCC is telling you what token it thinks should come next. From
that you can deduce what production is actually being selected, which again
should help you identify the problem.<br>
<br>
--- On Thu, 9/3/09, Timothy Washington <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=11" target="_top" rel="nofollow">timothyjwashington@...</a>>
wrote:<br>
<br>
> From: Timothy Washington <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=12" target="_top" rel="nofollow">timothyjwashington@...</a>><br>
> Subject: Re: Fixing Parser Weirdness<br>
> To: "Discussion mailing list for the SableCC project" <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=13" target="_top" rel="nofollow">sablecc-discussion@...</a>><br>
> Date: Thursday, September 3, 2009, 11:49 PM<br>
> <br>
> That's actually a really good point - I made that<br>
> change. But the parser is still breaking with the error in<br>
> fig. 1 when <br>
> I do a simple command ( see fig. 0 ). Ultimately I want to<br>
> make a command like in fig. 2. To remove the error, all I<br>
> have to <br>
> do is remove the "returninput_opt" parts ( lines<br>
> 98 & 288 ). This is what I meant by a sensitive parser,<br>
> because I can't see <br>
> how those 2 lines would break an unrelated token /<br>
> production. <br>
> <br>
> <br>
> login ( user -username root -password password ); <br>
> fig. 0 <br>
> <br>
> <br>
> login ( user -username root -password password ); <br>
> ERROR [Thread-3] (Bkell.java:186) - [4,7] expecting:<br>
> lbracket<br>
> com.interrupt.bookkeeping.cc.parser.ParserException:<br>
> [4,7] expecting:<br>
> lbracket<br>
> at<br>
> com.interrupt.bookkeeping.cc.parser.Parser.parse(Parser.java:1022)<br>
> at<br>
> com.interrupt.bookkeeping.cc.bkell.Bkell.run(Bkell.java:145)<br>
> at<br>
> java.lang.Thread.run(Thread.java:637)<br>
> ERROR [Thread-3] (Bkell.java:188) - <logs<br>
> xmlns='com/interrupt/logs' id='' ><log<br>
> xmlns='com/interrupt/logs' level='ERROR'<br>
> id='' ><logMessages<br>
> xmlns='com/interrupt/logs' id=''<br>
> ><logMessage xmlns='com/interrupt/logs'<br>
> id='' >[4,7] expecting:<br>
> lbracket</logMessage><br>
> </logMessages><br>
> </log><br>
> </logs><br>
> <br>
> ERROR [Thread-3] (Bkell.java:186) - [1,2] expecting:<br>
> 'var', 'create', 'add',<br>
> 'update', 'remove', 'reverse',<br>
> 'find', 'list', 'print',<br>
> 'commit', 'load', 'login',<br>
> 'logout', 'exit'<br>
> com.interrupt.bookkeeping.cc.parser.ParserException:<br>
> [1,2] expecting: 'var', 'create',<br>
> 'add', 'update', 'remove',<br>
> 'reverse', 'find', 'list',<br>
> 'print', 'commit', 'load',<br>
> 'login', 'logout',<br>
> 'exit'<br>
> at<br>
> com.interrupt.bookkeeping.cc.parser.Parser.parse(Parser.java:1022)<br>
> at<br>
> com.interrupt.bookkeeping.cc.bkell.Bkell.run(Bkell.java:145)<br>
> at<br>
> java.lang.Thread.run(Thread.java:637)<br>
> ERROR [Thread-3] (Bkell.java:188) - <logs<br>
> xmlns='com/interrupt/logs' id='' ><log<br>
> xmlns='com/interrupt/logs' level='ERROR'<br>
> id='' ><logMessages<br>
> xmlns='com/interrupt/logs' id=''<br>
> ><logMessage xmlns='com/interrupt/logs'<br>
> id='' >[1,2] expecting: 'var',<br>
> 'create', 'add', 'update',<br>
> 'remove', 'reverse', 'find',<br>
> 'list', 'print', 'commit',<br>
> 'load', 'login', 'logout',<br>
> 'exit'</logMessage><br>
> </logMessages><br>
> </log><br>
> </logs><br>
> ...<br>
> <br>
> fig. 1 <br>
> <br>
> <br>
> var aauthUsers = add ( ( load ( `/system[<br>
> @id='main.system' ]` ) )<br>
> <br>
> <user xmlns='com/interrupt/bookkeeping/users'<br>
> id='seven' username='seven'<br>
> password='seven' logintimeout='600000' ><br>
> <br>
> <profileDetails<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='user.details' ><br>
> <br>
> <profileDetail<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='firstname' name='first.name'<br>
> value='seven' /><br>
> <br>
> <profileDetail<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='lastname' name='last.name'<br>
> value='seven' /><br>
> <br>
> <profileDetail<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='email' name='email'<br>
> value='<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=14" target="_top" rel="nofollow">twashing@...</a>'<br>
> /><br>
> <br>
> <profileDetail<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='country' name='country'<br>
> value='U.S.A' /><br>
> <br>
> </profileDetails><br>
> <br>
> </user><br>
> ,<br>
> <br>
> -returninput<br>
> true <br>
> );<br>
> <br>
> fig. 2 <br>
> <br>
> <br>
> Tim<br>
> <br>
> <br>
> <br>
> From: Christopher Van Kirk<br>
> <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=15" target="_top" rel="nofollow">chris.vankirk@...</a>><br>
> To:<br>
> Discussion mailing list for the SableCC project<br>
> <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=16" target="_top" rel="nofollow">sablecc-discussion@...</a>><br>
> Sent:<br>
> Thursday, September 3, 2009 12:08:21 AM<br>
> Subject: RE:<br>
> Fixing Parser Weirdness<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> This is<br>
> probably your problem:<br>
> <br>
> <br>
> <br>
> dash = '-';<br>
> <br>
> .<br>
> <br>
> .<br>
> <br>
> .<br>
> <br>
> entry_opt = '-entry' ws+ (<br>
> lowercase | uppercase | dash | colon_helper | ats |<br>
> underscore | digit | dot<br>
> )*; <br>
> <br>
> <br>
> <br>
> You’ve<br>
> defined a dash token in the lexer<br>
> section, but you’re expecting to see it joined to a word<br>
> in the parser,<br>
> Remember that what comes out of the Tokens section is a<br>
> stream of tokens you<br>
> have defined there, so what your parser will see<br>
> is<br>
> <br>
> <br>
> <br>
> dash<br>
> entry …<br>
> not -entry<br>
> <br>
> <br>
> <br>
> Try<br>
> changing your production to something<br>
> like:<br>
> <br>
> <br>
> <br>
> entry_opt = dash<br>
> 'entry' ws+ (<br>
> lowercase | uppercase | dash | colon_helper | ats |<br>
> underscore | digit | dot<br>
> )*; <br>
> <br>
> <br>
> <br>
> Cheers,<br>
> <br>
> <br>
> <br>
> Chris…<br>
> <br>
> <br>
> <br>
> -----Original<br>
> Message-----<br>
> <br>
> From:<br>
> sablecc-discussion-bounces+chris.vankirk=<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=17" target="_top" rel="nofollow">fdcjapan.com@...</a><br>
> [mailto:sablecc-discussion-bounces+chris.vankirk=<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=18" target="_top" rel="nofollow">fdcjapan.com@...</a>]<br>
> On Behalf Of<br>
> Timothy Washington<br>
> <br>
> Sent:<br>
> Wednesday,<br>
> September 02, 2009<br>
> 11:43<br>
> PM<br>
> <br>
> To:<br>
> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=19" target="_top" rel="nofollow">sablecc-discussion@...</a><br>
> <br>
> Subject:<br>
> Fixing Parser Weirdness<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> Hey<br>
> all, I've implemented a language using sablecc that<br>
> looks like in fig. 1. <br>
> <br>
> <br>
> <br>
> command( <br>
> <br>
> ( <context/> ) <br>
> <br>
> <input/>, <br>
> <br>
> `/input[<br>
> @att='value' ]`, <br>
> <br>
> -inputoption value <br>
> <br>
> ); <br>
> <br>
> fig. 1<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> A) I'm trying to implement an input option, "<br>
> -inputoption value ",<br>
> but if I try to implement the command in fig. 2, I get the<br>
> error <br>
> <br>
> seen in fig. 4. <br>
> <br>
> <br>
> <br>
> var removedE = remove ( ( load( `/system[<br>
> @id="main.system" ]/groups[<br>
> @id="main.groups" ]/group[<br>
> @id="seven.group" ]/bookkeeping[<br>
> @id="main.bookkeeping" ]/journals[<br>
> @id="main.journals"<br>
> ]/journal[ @id="generalledger" ]/entries[<br>
> @id="main.entries" ]` ) ) <br>
> <br>
> <entry<br>
> xmlns='com/interrupt/bookkeeping/journal'<br>
> id='qwer' entrynum='' state=''<br>
> journalid='' date='02022006'<br>
> currency='CDN' > <br>
> <br>
> <br>
> <debit<br>
> xmlns='com/interrupt/bookkeeping/account'<br>
> id='asdf' amount='11.00' entryid=''<br>
> accountid='1' account='office equipment'<br>
> currency='CDN' /> <br>
> <br>
> <br>
> <debit<br>
> xmlns='com/interrupt/bookkeeping/account'<br>
> id='zxcv' amount='1.50' entryid=''<br>
> accountid='2' account='tax'<br>
> currency='CDN' /> <br>
> <br>
> <br>
> <credit<br>
> xmlns='com/interrupt/bookkeeping/account'<br>
> id='tyui' amount='12.50' entryid=''<br>
> accountid='3' account='bank'<br>
> currency='CDN' /> <br>
> <br>
> </entry> <br>
> <br>
> ,<br>
> <br>
> <br>
> -returninput<br>
> true <br>
> <br>
> );<br>
> <br>
> fig. 2<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> B) I'm also trying to get SableCC to ignore special<br>
> characters between single<br>
> and double quotes ( ' " ). So for example, I want<br>
> <br>
> <br>
> to be able to say <profileDetail<br>
> value='<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=20" target="_top" rel="nofollow">twashing@...</a>' />, but the<br>
> parser dies with the error in fig. 5. <br>
> <br>
> <br>
> <br>
> var aauthUsers = add ( ( load ( `/system[<br>
> @id='main.system' ]` ) )<br>
> <br>
> <br>
> <user<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='seven' username='seven'<br>
> password='seven' logintimeout='600000'<br>
> ><br>
> <br>
> <br>
> <profileDetails<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='user.details'<br>
> ><br>
> <br>
> <br>
> <profileDetail<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='firstname' name='first.name'<br>
> value='seven' /><br>
> <br>
> <br>
> <profileDetail<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='lastname' name='last.name'<br>
> value='seven' /><br>
> <br>
> <br>
> <profileDetail<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='email' name='email'<br>
> value='<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=21" target="_top" rel="nofollow">twashing@...</a>' /><br>
> <br>
> <br>
> <profileDetail<br>
> xmlns='com/interrupt/bookkeeping/users'<br>
> id='country' name='country'<br>
> value='U.S.A' /><br>
> <br>
> <br>
> </profileDetails><br>
> <br>
> <br>
> </user><br>
> <br>
> );<br>
> <br>
> fig. 3<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> It feels like Sablecc is very sensitive in the order of<br>
> tokens. Can anyone help<br>
> me isolate and fix this problem. I've attached <br>
> <br>
> my sablecc file.<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> Thanks in advance <br>
> <br>
> Tim <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> com.interrupt.bookkeeping.cc.parser.ParserException: [1,1]<br>
> expecting: 'var',<br>
> 'create', 'add', 'update',<br>
> 'remove', 'reverse', 'find',<br>
> 'list', 'print',<br>
> 'commit', 'load', 'login',<br>
> 'logout', 'exit'<br>
> <br>
> at<br>
> com.interrupt.bookkeeping.cc.parser.Parser.parse(Parser.java:1028)<br>
> <br>
> at<br>
> com.interrupt.bookkeeping.cc.bkell.Bkell.run(Bkell.java:145)<br>
> <br>
> at<br>
> java.lang.Thread.run(Thread.java:637)<br>
> <br>
> fig. 4<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> com.interrupt.bookkeeping.cc.parser.ParserException: [19,3]<br>
> expecting:<br>
> 'create', 'add', 'update',<br>
> 'remove', 'reverse', 'find',<br>
> 'list', 'print',<br>
> 'commit', 'load', 'login',<br>
> 'logout', 'exit', 'system',<br>
> 'debit', 'credit',<br>
> 'entry', 'entries', 'journal',<br>
> 'journals', 'transaction',<br>
> 'account',<br>
> 'accounts', 'user', 'users',<br>
> 'group', 'groups', 'allowedActions',<br>
> 'command',<br>
> 'profileDetails', 'profileDetail',<br>
> 'userSession', '<', rbracket, atsign,<br>
> '`'<br>
> <br>
> at<br>
> com.interrupt.bookkeeping.cc.parser.Parser.parse(Parser.java:1022)<br>
> <br>
> at<br>
> com.interrupt.bookkeeping.cc.bkell.Bkell.run(Bkell.java:145)<br>
> <br>
> at<br>
> java.lang.Thread.run(Thread.java:637)<br>
> <br>
> fig. 5<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> Looking for the<br>
> perfect gift? Give the gift of<br>
> Flickr!<br>
> <br>
> <br>
> <br>
> <br>
> <br>
> <br>
> Looking for the perfect gift? Give<br>
> the gift of Flickr!<br>
> -----Inline Attachment Follows-----<br>
> <br>
> _______________________________________________<br>
> SableCC-Discussion mailing list<br>
> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=22" target="_top" rel="nofollow">SableCC-Discussion@...</a><br>
> <a rel="nofollow" target="_blank" href="http://lists.sablecc.org/listinfo/sablecc-discussion">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br>
> <br>
<br>
_______________________________________________<br>
SableCC-Discussion mailing list<br>
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=23" target="_top" rel="nofollow">SableCC-Discussion@...</a><br>
<a rel="nofollow" target="_blank" href="http://lists.sablecc.org/listinfo/sablecc-discussion">http://lists.sablecc.org/listinfo/sablecc-discussion</a></p>
</div>
<p class="MsoNormal"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<div class="MsoNormal" style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">
<hr align="center" size="1" width="100%">
</span></font></div>
<p class="MsoNormal"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"><img src="http://us.i1.yimg.com/us.yimg.com/i/ca/iotg_search.jpg" align="absbottom" border="0" height="25" hspace="4" width="25"><a rel="nofollow" target="_blank" href="http://ca.toolbar.yahoo.com/"><b><span style="font-weight: bold;" lang="NO-BOK">Yahoo! Canada
Toolbar :</span></b><span lang="NO-BOK"> Search from anywhere on the web and
bookmark your favourite sites. Download it now! </span></a></span></font></p>
</div>
<p><font face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial;">No
virus found in this incoming message.<br>
Checked by AVG - <a rel="nofollow" target="_blank" href="http://www.avg.com">www.avg.com</a><br>
Version: 8.5.421 / Virus Database: 270.14.3/2414 - Release Date: </span></font><font face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial;">10/07/09</span></font><font face="Arial" size="2"><span style="font-size: 10pt; font-family: Arial;"> 20:49:00</span></font></p>
</div>
</div>
</div>
<p class="MsoNormal"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<div class="MsoNormal" style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">
<hr align="center" size="1" width="100%">
</span></font></div>
<p class="MsoNormal"><font face="Times New Roman" size="3"><span style="font-size: 12pt;"><img src="http://us.i1.yimg.com/us.yimg.com/i/ca/iotg_search.jpg" align="absbottom" border="0" height="25" hspace="4" width="25"><a rel="nofollow" target="_blank" href="http://ca.toolbar.yahoo.com/"><b><span style="font-weight: bold;" lang="NO-BOK">Yahoo! Canada Toolbar :</span></b><span lang="NO-BOK"> Search from anywhere on the web and bookmark your favourite sites.
Download it now! </span></a></span></font></p>
</div>
<p><font face="Arial" size="2">No virus found in this incoming message.<br><span>
Checked by AVG - <a target="_blank" href="http://www.avg.com" rel="nofollow">www.avg.com</a></span><br>
Version: 8.5.421 / Virus Database: 270.14.3/2414 - Release Date: 10/08/09 18:33:00<br>
</font></p>
<p><font face="Arial" size="2"> </font> </p></div></div></div><br>
<hr size=1> <a href="http://ca.promos.yahoo.com/jacko/" target="_top" rel="nofollow">Get the name you've always wanted </a>! <strong>@ymail.com </strong>or <strong>@rocketmail.com</strong>.</body></html><br />_______________________________________________
<br>SableCC-Discussion mailing list
<br><a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=25836432&i=24" target="_top" rel="nofollow">SableCC-Discussion@...</a>
<br><a href="http://lists.sablecc.org/listinfo/sablecc-discussion" target="_top" rel="nofollow">http://lists.sablecc.org/listinfo/sablecc-discussion</a><br><p>From forum: <a href="http://old.nabble.com/SableCC---User-f12281.html" embed="fixTarget[12281]" target="_top" >SableCC - User</a></p>