If statement vs short if statement

[Alex James-2]

Hi,

I'm quite to new JavaCC and need a little help with the parsing of an  
if statement in the language I'm trying to replicate.

The language has the following definition for if statements:

<if> condition
   statement+
(<else>
   statement+)?
<endif>

It also has a short if statement:

<if> condition statement

Trouble is I can't get my parser to recognise when the short if  
statement has been used as there is nothing to distinguish it from the  
long if statement. The language I'm trying to replicate generally  
ignores end of line characters but seems to take them into account  
when recognising the short if statement. Is there any way I can ignore  
end of line characters except in this one circumstance?

Any help would be greatly appreciated.

Cheers,

Alex

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Can statements be broken up on multiple lines? If they can't, the easiest
solution you have is probably to treat end of lines just like any other token
in your grammar instead of ignoring them.

then you could just define your statement and if as

Statement = ( Whatever | IfStatement | WhileStatement | Declaration | ...)
<EOL>+
IfStatement = <IF> condition (LongIf | Statement)
LongIf = <EOL> Statement+ (<ELSE> statement+)? <ENDIF>

The advantage of this is that your parser won't need to do any deep,
complicated lookahead to decide which branch to take, so it should be fast
enough.

Are there any other cases that have a short form (say while and for loops, for
example)?

On Sunday 17 May 2009 11:03:22 pm Alex James wrote:

> Hi,
>
> I'm quite to new JavaCC and need a little help with the parsing of an
> if statement in the language I'm trying to replicate.
>
> The language has the following definition for if statements:
>
> <if> condition
>    statement+
> (<else>
>    statement+)?
> <endif>
>
> It also has a short if statement:
>
> <if> condition statement
>
> Trouble is I can't get my parser to recognise when the short if
> statement has been used as there is nothing to distinguish it from the
> long if statement. The language I'm trying to replicate generally
> ignores end of line characters but seems to take them into account
> when recognising the short if statement. Is there any way I can ignore
> end of line characters except in this one circumstance?
>
> Any help would be greatly appreciated.
>
> Cheers,
>
> Alex
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Alex James-2]

Unfortunately, statements can be broken up on multiple lines so I  
would have to define an <EOL> token after every possible input except  
for this situation.
I'm thinking I might have to go for the deep complicated look ahead  
option.
I've been playing around with it and have come up with this (it  
doesn't work but it's a starting point):

<IF> condition()
       
                        ((statement())+ (<ELSE> (statement())+)? <ENDIF>
                |
                        statement() )

How would I use lookahead in this situation? At the moment it chooses  
the long if incorrectly.


On 17 May 2009, at 20:29, Nicolas Piguet wrote:

> Can statements be broken up on multiple lines? If they can't, the  
> easiest
> solution you have is probably to treat end of lines just like any  
> other token
> in your grammar instead of ignoring them.
>
> then you could just define your statement and if as
>
> Statement = ( Whatever | IfStatement | WhileStatement | Declaration  
> | ...)
> <EOL>+
> IfStatement = <IF> condition (LongIf | Statement)
> LongIf = <EOL> Statement+ (<ELSE> statement+)? <ENDIF>
>
> The advantage of this is that your parser won't need to do any deep,
> complicated lookahead to decide which branch to take, so it should  
> be fast
> enough.
>
> Are there any other cases that have a short form (say while and for  
> loops, for
> example)?
>
> On Sunday 17 May 2009 11:03:22 pm Alex James wrote:
>> Hi,
>>
>> I'm quite to new JavaCC and need a little help with the parsing of an
>> if statement in the language I'm trying to replicate.
>>
>> The language has the following definition for if statements:
>>
>> <if> condition
>>   statement+
>> (<else>
>>   statement+)?
>> <endif>
>>
>> It also has a short if statement:
>>
>> <if> condition statement
>>
>> Trouble is I can't get my parser to recognise when the short if
>> statement has been used as there is nothing to distinguish it from  
>> the
>> long if statement. The language I'm trying to replicate generally
>> ignores end of line characters but seems to take them into account
>> when recognising the short if statement. Is there any way I can  
>> ignore
>> end of line characters except in this one circumstance?
>>
>> Any help would be greatly appreciated.
>>
>> Cheers,
>>
>> Alex
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@...
>> For additional commands, e-mail: users-help@...
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

You can probably do something like

if = <IF> condition() statement() LOOKAHEAD(LongIf()) LongIf()?
LongIf = Statement()* (<ELSE> Statement() +) <ENDIF>

But that would be a horrible, horrible solution, with generally very bad
performance (and also, it might not work at all). Your problem is that if you
ignore all whitespace there really is no way to tell if the second statement
after the if belongs to the if without first parsing all the source code until
you reach an ENDIF (and I'm not sure it would even work). If there is any kind
of termination character, then you have a practical solution, otherwise, it's
going to be really hard to solve.

You might be able to do something by exploring token manager states instead.
You could have a WHITESPACE_IGNORE state (which is what you already have), and
a new NEWLINE state, which would only return an <EOL> token.

you could then switch to that NEWLINE state after you parsed the condition. If
the tokenizer sees no <EOL> token, then you have a short if, otherwise, you
have a long if. In both cases, you need to return to the WHITESPACE_IGNORE
state before attempting to parse the statement.

I haven't played with JavaCC for a while so I'm not sure about the syntax any
more, but you could do something like

If = <IF> condition() IfSuffix
IfSuffix = {{tokenManager.switchTo(NEWLINE)}}
              EOL+ LongIf() |
              ShortIf()
ShortIf = {{tokenManager.switchTo(WHITESPACE_IGNORE)}} statement()
LongIf  = {{tokenManager.switchTo(WHITESPACE_IGNORE)}}
               Statement()* (<ELSE> Statement() +) <ENDIF>

This sort of thing might work if the start of the statement of a short if is
guaranteed to be on the same line as the end of the if condition. If that is
not guaranteed, then I'm afraid you problem might not be solvable with JavaCC.

On Tuesday 19 May 2009 02:26:47 pm Alex James wrote:

> Unfortunately, statements can be broken up on multiple lines so I
> would have to define an <EOL> token after every possible input except
> for this situation.
> I'm thinking I might have to go for the deep complicated look ahead
> option.
> I've been playing around with it and have come up with this (it
> doesn't work but it's a starting point):
>
> <IF> condition()
>
> ((statement())+ (<ELSE> (statement())+)? <ENDIF>
>
> statement() )
>
> How would I use lookahead in this situation? At the moment it chooses
> the long if incorrectly.
>
> On 17 May 2009, at 20:29, Nicolas Piguet wrote:
> > Can statements be broken up on multiple lines? If they can't, the
> > easiest
> > solution you have is probably to treat end of lines just like any
> > other token
> > in your grammar instead of ignoring them.
> >
> > then you could just define your statement and if as
> >
> > Statement = ( Whatever | IfStatement | WhileStatement | Declaration
> >
> > | ...)
> >
> > <EOL>+
> > IfStatement = <IF> condition (LongIf | Statement)
> > LongIf = <EOL> Statement+ (<ELSE> statement+)? <ENDIF>
> >
> > The advantage of this is that your parser won't need to do any deep,
> > complicated lookahead to decide which branch to take, so it should
> > be fast
> > enough.
> >
> > Are there any other cases that have a short form (say while and for
> > loops, for
> > example)?
> >
> > On Sunday 17 May 2009 11:03:22 pm Alex James wrote:
> >> Hi,
> >>
> >> I'm quite to new JavaCC and need a little help with the parsing of an
> >> if statement in the language I'm trying to replicate.
> >>
> >> The language has the following definition for if statements:
> >>
> >> <if> condition
> >>   statement+
> >> (<else>
> >>   statement+)?
> >> <endif>
> >>
> >> It also has a short if statement:
> >>
> >> <if> condition statement
> >>
> >> Trouble is I can't get my parser to recognise when the short if
> >> statement has been used as there is nothing to distinguish it from
> >> the
> >> long if statement. The language I'm trying to replicate generally
> >> ignores end of line characters but seems to take them into account
> >> when recognising the short if statement. Is there any way I can
> >> ignore
> >> end of line characters except in this one circumstance?
> >>
> >> Any help would be greatly appreciated.
> >>
> >> Cheers,
> >>
> >> Alex
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe@...
> >> For additional commands, e-mail: users-help@...
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe@...
> > For additional commands, e-mail: users-help@...
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...