[PATCH] todo: short term

[Akim Demaille-3]

On master only.

 From 5ad90d528dc5feedb0c1d8afe82719440ec17217 Mon Sep 17 00:00:00 2001
From: Akim Demaille <demaille@...>
Date: Thu, 17 Sep 2009 09:41:21 +0200
Subject: [PATCH] todo: short term

        * TODO (syntax_error, variable names): New.
---
  TODO |   22 ++++++++++++++++++++++
  1 files changed, 22 insertions(+), 0 deletions(-)

diff --git a/TODO b/TODO
index 216c97f..c3aac31 100644
--- a/TODO
+++ b/TODO
@@ -1,6 +1,28 @@
  -*- outline -*-

  * Short term
+** Use syntax_error from the scanner?
+This would provide a means to raise syntax error from function called
+from the scanner.  Actually, there is no good solution to report a
+lexical error in general.  Usually they are kept at the scanner level
+only, ignoring the guilty token.  But that might not be the best bet,
+since we don't benefit from the syntactic error recovery.
+
+We still have the possibility to return an invalid token number, which
+does the trick.  But then the error message from the parser is poor
+(something like "unexpected $undefined").  Since the scanner probably
+already reported the error, we should directly enter error-recovery,
+without reporting the error message (i.e., YYERROR's semantics).
+
+Back to lalr1.cc (whose name is now quite unfortunate, since it also
+covers lr and ielr), if we support exceptions from yylex, should we
+propose a lexical_error in addition to syntax_error?  Should they have
+a common root, say parse_error?  Should syntax_error be renamed
+syntactic_error for consistency with lexical_error?
+
+** Variable names.
+What should we name `variant' and `lex_symbol'?
+
  ** Use b4_symbol in all the skeleton
  Then remove the older system, including the tables generated by
  output.c
--
1.6.4.3

[Joel E. Denny-4]

On Thu, 17 Sep 2009, Akim Demaille wrote:

> +** Use syntax_error from the scanner?
> +This would provide a means to raise syntax error from function called
> +from the scanner.  Actually, there is no good solution to report a
> +lexical error in general.  Usually they are kept at the scanner level
> +only, ignoring the guilty token.  But that might not be the best bet,
> +since we don't benefit from the syntactic error recovery.
> +
> +We still have the possibility to return an invalid token number, which
> +does the trick.  But then the error message from the parser is poor
> +(something like "unexpected $undefined").

To fix that, use %token.

> Since the scanner probably
> +already reported the error, we should directly enter error-recovery,
> +without reporting the error message (i.e., YYERROR's semantics).

Maybe Bison can define a special token that the scanner can return to
induce a YYERROR?  That seems simpler than a function call.

[Akim Demaille-3]

Le 26 sept. 2009 à 06:24, Joel E. Denny a écrit :

Hi Joel!

> On Thu, 17 Sep 2009, Akim Demaille wrote:
>
>> +** Use syntax_error from the scanner?
>> +This would provide a means to raise syntax error from function  
>> called
>> +from the scanner.  Actually, there is no good solution to report a
>> +lexical error in general.  Usually they are kept at the scanner  
>> level
>> +only, ignoring the guilty token.  But that might not be the best  
>> bet,
>> +since we don't benefit from the syntactic error recovery.
>> +
>> +We still have the possibility to return an invalid token number,  
>> which
>> +does the trick.  But then the error message from the parser is poor
>> +(something like "unexpected $undefined").
>
> To fix that, use %token.

Yes, but what would you return?  %token INVALID "invalid token"?  This  
is poor, as compared to wrapping an error message as for instance we  
currently do in Bison's scanner:

   {directive} {
     complain_at (*loc, _("invalid directive: %s"), quote (yytext));
   }

>> Since the scanner probably
>> +already reported the error, we should directly enter error-recovery,
>> +without reporting the error message (i.e., YYERROR's semantics).
>
> Maybe Bison can define a special token that the scanner can return to
> induce a YYERROR?  That seems simpler than a function call.

I think both are nice.  A special token is nice when you're in the  
scanner itself.  Throwing an exception is nice in C++, especially from  
functions called from the scanner.