2 line feeds "\n"

View: New views

2 Messages — Rating Filter: Alert me

2 line feeds "\n"

by Kurt Guenther :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Looks like I'm on 4.2:

$ javacc -version
Java Compiler Compiler Version 4.2 (Parser Generator)

I'm trying to figure why 2 line feeds "\n" are consumed for a token when
I'd expect two different tokens.

TOKEN :
{
<EOL: "\n" | "\r" | "\r\n" >
| <NOT_EOL: ~["\n","\r"] >
| <INTEGER: ["1"-"9"] (["0"-"9"])* >
| <TAG: "<400>" : TEXT
}

<TEXT> TOKEN :
{
<LINE: ( <NOT_EOL> )+ >
| <CONTINUATION: ( <EOL> " " ) >
// 1234567
| <TEXT_EOL: <EOL> > : DEFAULT
}

--- input ----
<400> 1

Lys Cys Asn
--- input ---

I'm text mode in the lexer. I'd expect to get:

TAG, LINE, TEXT_EOL, EOL

but, I'm missing the final EOL. Here's the output:

****** FOUND A <LINE> MATCH ( 1) ******

<TEXT>Current character : \n (10) at line 37 column 9
Starting NFA to match one of : { <LINE>, <CONTINUATION> }
<TEXT>Current character : \n (10) at line 37 column 9
Currently matched the first 1 characters as a <TEXT_EOL> token.
Possible kinds of longer matches : { <CONTINUATION>, <TEXT_EOL> }
<TEXT>Current character : \n (10) at line 38 column 1
Currently matched the first 2 characters as a <TEXT_EOL> token.
Possible kinds of longer matches : { <TEXT_EOL> }
<TEXT>Current character : L (76) at line 39 column 1
Currently matched the first 2 characters as a <TEXT_EOL> token.
Putting back 1 characters into the input stream.
****** FOUND A <TEXT_EOL> MATCH (\n\n) ******

I'm confused why TEXT_EOL would match "\n\n"???

--Kurt

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: 2 line feeds "\n"

by Kurt Guenther :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Nevermind. I was just forgetting to refresh my eclipse environment.
I thought I didn't have to do that for editors not open.

Kurt Guenther wrote:

> Looks like I'm on 4.2:
>
> $ javacc -version
> Java Compiler Compiler Version 4.2 (Parser Generator)
>
> I'm trying to figure why 2 line feeds "\n" are consumed for a token
> when I'd expect two different tokens.
>
> TOKEN :
> {
> <EOL: "\n" | "\r" | "\r\n" >
> | <NOT_EOL: ~["\n","\r"] >
> | <INTEGER: ["1"-"9"] (["0"-"9"])* >
> | <TAG: "<400>" : TEXT
> }
>
> <TEXT> TOKEN :
> { <LINE: ( <NOT_EOL> )+ > | <CONTINUATION: (
> <EOL> " " ) >
> // 1234567
> | <TEXT_EOL: <EOL> > : DEFAULT
> }
>
> --- input ----
> <400> 1
>
> Lys Cys Asn
> --- input ---
>
> I'm text mode in the lexer. I'd expect to get:
>
> TAG, LINE, TEXT_EOL, EOL
>
> but, I'm missing the final EOL. Here's the output:
>
> ****** FOUND A <LINE> MATCH ( 1) ******
>
> <TEXT>Current character : \n (10) at line 37 column 9
> Starting NFA to match one of : { <LINE>, <CONTINUATION> }
> <TEXT>Current character : \n (10) at line 37 column 9
> Currently matched the first 1 characters as a <TEXT_EOL> token.
> Possible kinds of longer matches : { <CONTINUATION>, <TEXT_EOL> }
> <TEXT>Current character : \n (10) at line 38 column 1
> Currently matched the first 2 characters as a <TEXT_EOL> token.
> Possible kinds of longer matches : { <TEXT_EOL> }
> <TEXT>Current character : L (76) at line 39 column 1
> Currently matched the first 2 characters as a <TEXT_EOL> token.
> Putting back 1 characters into the input stream.
> ****** FOUND A <TEXT_EOL> MATCH (\n\n) ******
>
> I'm confused why TEXT_EOL would match "\n\n"???
>
> --Kurt
>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...