Nabble - java.net - javacc users

Re: Use Longs for Line Numbers

2009-11-09T12:46:22Z

Yes, we should fix it. But for now, you can simply edit the generated
files - Token.java and SimpleCharStream.java to make the line numbers
long. There might be some issues with error reporting, but it should work.

Tom, I'm not sure I understand what you are saying. The line number stuff
is rather straightforward. It's used purely for reporting. It doesn't
affect the generated code at all.

>
> On Oct 6, 2009, at 6:57 PM, Kurt Guenther wrote:
>
>>
>> I'm currently parsing an 8 GB file with 241,000,000 lines which is
>> about 12.5% of the 2^31 -1 max value of int. I'm supposed to
>> support files up to 200 GB, so is there a way to generate the token
>> manager with longs instead of ints?
>
> I keep thinking about this one. We can't do just a search and replace
> on the generated code since other stuff uses ints as well - e.g., the
> lexical states are contained in an array of integers. I think it'd
> also affect the parser generating code, since we'd need to make jj_gen
> and jj_ntk and others into longs as well.
>
> At the end of the day, it might be easier to preprocess your input
> data to chunk it into smaller files.
>
> Yours,
>
> Tom
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Dynamic tokenizing

2009-11-09T07:56:47Z

Can't you just use a symbol table? Macro processors often do this kind
of thing. Another similar example is "typedef" in C.

I haven't used COMMON_TOKEN_ACTION since I use USER_TOKEN_MANAGER, but I
would think that you can update a symbol table as the "UserKeywords"
statement is parsed. Then in the common action code you can check
generic word tokens in the symbol table, and if found, change the "kind"
field in the token to a different token type (e.g. USER_KW). The actual
keyword text is in the "image" field. In the grammar you can specify
the <USER_KW> "{" ... "}" sequence anywhere it may occur and process it
as necessary as it is parsed.

Depending on what you are doing with the content within the braces, you
may have to do some additional processing in the common action code. If
you were trying to do text substitution or macro processing (i.e.
replacing the USER_KW {...} sequence with a different string of tokens
to be parsed), that would be a whole different ballgame.

Bill

Bachelier, Georges wrote:

> Hi!
>
>
>
> I have been searching a solution for three days to the following problem:
>
>
>
> I am building a JavaCC parser for a language having an original feature: the language allows you to define user key words which may be found anywhere in the input file after their declaration.
>
>
>
> User key words are declared with the following statement : UserKeywords <my_user_kw1> ... <my_user_kwn> ;
>
>
>
> Once keywords have been declared, they may be used like this :
>
>
>
> <my_user_kw1> {
>
> <any_sequence_of_characters>
>
> }
>
>
>
> <my_user_kwn> {
>
> <any_sequence_of_characters>
>
> }
>
>
>
> The sequences of characters do not matter; we do not use them, but they are surrounded by braces.
>
>
>
> I have written a parser production in order to get the user key words and I have tried to play with the tokenizer CommonTokenAction function, but without success.
>
>
>
> Could someone help me on this issue, please?
>
>
>
> Georges
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Use Longs for Line Numbers

2009-11-09T06:19:53Z

On Oct 6, 2009, at 6:57 PM, Kurt Guenther wrote:

>
> I'm currently parsing an 8 GB file with 241,000,000 lines which is
> about 12.5% of the 2^31 -1 max value of int. I'm supposed to
> support files up to 200 GB, so is there a way to generate the token
> manager with longs instead of ints?

I keep thinking about this one. We can't do just a search and replace
on the generated code since other stuff uses ints as well - e.g., the
lexical states are contained in an array of integers. I think it'd
also affect the parser generating code, since we'd need to make jj_gen
and jj_ntk and others into longs as well.

At the end of the day, it might be easier to preprocess your input
data to chunk it into smaller files.

Yours,

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: JavaCC Version 5.0 released.

2009-11-09T06:09:45Z

That's an interesting idea.... you might want to file an issue here:

https://javacc.dev.java.net/servlets/ProjectIssues

Just to keep it on the radar.

Yours,

tom

On Sep 1, 2009, at 5:02 AM, Artur Rataj wrote:

> Hello.
>
> What about a way to ensure that, at certain points of the grammar,
> backing up a single token is safe and, that a next token exists?
>
> I use the former for error recovery, but I do not know if it won't
> cause weird lexer/parser behavior in some cases.
>
> I can ensure the latter by adding artificial choices to the grammar,
> but I still do not know if it works in all cases.
>
> As of me, such features would add a bit of flexibility that one has
> with hand--made parsers.
>
> But, perhaps there is virtually no demand for the features among
> JavaCC users.
>
> Best regards,
> Artur
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

RE: Dynamic tokenizing

2009-11-09T00:25:59Z

I do not believe you can do dynamic tokenizing directly in JavaCC.

I would think about writing a first parser that recognizes your UserKeywords syntax, and in the corresponding production a) generates a second parser (through println() statements, or a through a template engine) which will contain all the keywords token definitions, b) compile it, and c) execute it on the remaining of the input file.

Marc

De : Bachelier, Georges [mailto:georges.bachelier@...]
Envoyé : vendredi 6 novembre 2009 20:14
À : users@...
Objet : [JavaCC] Dynamic tokenizing

Hi!

I have been searching a solution for three days to the following problem:

I am building a JavaCC parser for a language having an original feature: the language allows you to define user key words which may be found anywhere in the input file after their declaration.

User key words are declared with the following statement : UserKeywords <my_user_kw1> <my_user_kwn> ;

Once keywords have been declared, they may be used like this :

<my_user_kw1> {

<any_sequence_of_characters>

}

<my_user_kwn> {

<any_sequence_of_characters>

}

The sequences of characters do not matter; we do not use them, but they are surrounded by braces.

I have written a parser production in order to get the user key words and I have tried to play with the tokenizer CommonTokenAction function, but without success.

Could someone help me on this issue, please?

Georges

Re: (jjtree) Problem with TRACK_TOKENS - firstToken is wrong

2009-11-08T01:08:06Z

Hi Paul,

I created issue 232 for it
https://javacc.dev.java.net/issues/show_bug.cgi?id=232

Cheers,
Carsten

Paul Cager wrote:

>> Hi,
>>
>> I have issues concerning TRACK_TOKENS in jjtree as it sets the wrong
>> "firstToken" on some nodes in my case.
>
> .....
>> Is this all a problem with the grammar or a bug in JJTree?
>>
>> TIA
>> Carsten
>
>
> Hi Carsten,
>
> That sounds like a bug in jjtree (I've not checked the details, but using
> getToken(1) doesn't sound right).
>
> Would you mind creating an issue
> (https://javacc.dev.java.net/servlets/ProjectIssues) so that we remember
> to fix it in the next release?
>
> Cheers.
> Paul
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Dynamic tokenizing

2009-11-06T12:55:05Z

This sounds doable. You can probably use LOOKAHEAD with a function call
as parameter instead of a constant value.
See an example here:

<http://old.nabble.com/Problem:-semantic-lookahead-calling-boolean-functions-td23087841.html>

Bachelier, Georges wrote:

>
> Hi!
>
> I have been searching a solution for three days to the following problem:
>
> I am building a JavaCC parser for a language having an original
> feature: the language allows you to define user key words which may be
> found anywhere in the input file after their declaration.
>
> User key words are declared with the following statement :
> UserKeywords <my_user_kw1> … <my_user_kwn> ;
>
> Once keywords have been declared, they may be used like this :
>
> <my_user_kw1> {
>
> <any_sequence_of_characters>
>
> }
>
> <my_user_kwn> {
>
> <any_sequence_of_characters>
>
> }
>
> The sequences of characters do not matter; we do not use them, but
> they are surrounded by braces.
>
> I have written a parser production in order to get the user key words
> and I have tried to play with the tokenizer CommonTokenAction
> function, but without success.
>
> Could someone help me on this issue, please?
>
> Georges
>

--
Regards,
Farrukh

Web: http://www.wellfleetsoftware.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Dynamic tokenizing

2009-11-06T11:13:50Z

Hi!

I have been searching a solution for three days to the following problem:

I am building a JavaCC parser for a language having an original feature: the language allows you to define user key words which may be found anywhere in the input file after their declaration.

User key words are declared with the following statement : UserKeywords <my_user_kw1> … <my_user_kwn> ;

Once keywords have been declared, they may be used like this :

<my_user_kw1> {

<any_sequence_of_characters>

}

<my_user_kwn> {

<any_sequence_of_characters>

}

The sequences of characters do not matter; we do not use them, but they are surrounded by braces.

I have written a parser production in order to get the user key words and I have tried to play with the tokenizer CommonTokenAction function, but without success.

Could someone help me on this issue, please?

Georges

LOOKAHEAD question

2009-11-05T00:47:32Z

Hello,

I do not understand how lookahead works exactly.

In a regex, I can use a expression like this: "A+A", which means to get as many As and finally get another A.

(I know that it is equivalent to "AA+" which is simplier, but the previous is an example to help me understand lookahead in javacc)

I've tried to reproduce the "A+A" example in javacc, but it does not work.

From my understanding, a LOOKAHEAD of 2 could process the input "AAA", and a LOOKAHEAD of 4 could process the input "AAAA".

Why this example is not working?

++++++++++++++++++++++++++++++++++

options { STATIC = false; }

PARSER_BEGIN(B)

import java.io.*;

public class B {

static String test = "AAA";

public static void main(String args[]) throws Exception {

B app = new B();

BTokenManager ttm = new BTokenManager(new SimpleCharStream(new StringReader(test)));

app.ReInit(ttm);

app.Entry();

}

public B() { this((Reader) null); }

}

PARSER_END(B)

TOKEN : { <A: "A"> }

void Entry() : { }

{

( LOOKAHEAD(2) <A> )+

<A>

}

++++++++++++++++++++++++++++++++++

Best regards,

DAvid

Re: Update of the JavaCC cobol grammar

2009-10-30T11:40:07Z

On Oct 28, 2009, at 5:42 AM, thierry blind wrote:

> Hello,
> I would like to contribute to this wonderful project by updating the
> JavaCC cobol grammar that was initially made by Bernard Pinon (and
> that can be found in the repository of JavaCC grammars).
> I've heavily modified and improved this grammar, although it is (and
> i think will) never be perfect. The grammar was well tested with
> java 4.2 in a production environment.
> I've attached the new grammar file to this mail. I left some "TODO"
> comments in this file, for some pending improvements, but those
> "ugly" comments can be removed if necessary.
> I don't know if it is the right address to sind this file, hope I'm
> not wrong ;)
>
> Comments and suggestions are welcome :)

Hi Thierry -

I've uploaded your new grammar here:

https://javacc.dev.java.net/servlets/ProjectDocumentList?folderID=110

Thanks!

Yours,

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Matching productions embedded in arbitrary text

2009-10-30T11:33:21Z

On Oct 29, 2009, at 10:36 AM, Farrukh Najmi wrote:

>>
>> To me the tricky bit here is that you don't know the function names
>> until runtime, so an input string like this:
>>
>> texttext fiddle("a", 1)texttext
>>
>> would only be a FunctionCall if the name "fiddle" is in
>> FUNCTION_NAMES, right?
>>
>> With that constraint, I'm not sure how to get JavaCC to tokenize
>> the input data. I wonder if you might want to extract the function
>> declarations using a regex. You could use a regex like ".* (" +
>> someFunctionName + "(.*))" to extract the function declaration from
>> the surrounding text, and then you could pass the match data to
>> JavaCC to create the parse tree. This would only work if it's just
>> a function call, though, it wouldn't work if the function body were
>> there because that could include nested parenthesis which would
>> confuse the regex.
>
> Hi Tom,
>
> My experience suggests that part seems easy. You just define a
> LOOKAHEAD that uses a function seeFunctionCall() where the function
> looks at next few tokens and decides whether it has a function name
> or not. See example below...

Hm, yup, semantic lookahead could be useful there, good idea. But
what I'm wondering about though... how do you tokenize? Do you just
let every character be a separate token?

Yours,

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

javacc better error messaging

2009-10-30T04:34:12Z

I'd like to know about error messaging capabilities.

javacc 5.0

Assumptions:
I have 5 fields with 5 numbers each ... with some separator.
f.e.:
01234*56789*12391*98765

I cannot change the format and have a specification saying that the first 5 digits are variable say A, the next B ...

| <#NUM_CHAR: ["0"-"9"]>
| <N5: <NUM_CHAR><NUM_CHAR><NUM_CHAR><NUM_CHAR><NUM_CHAR> >
| <DELIMITER: *>

Sometimes I get data violating the format. Example: 4 instead of 5 digits. How can I tell which variable A,B,... is meant?

Parse error says only: <N5> expected, got: 1234

My silly workaround is this:

{ currentField = "A"; } (t = <N5>) { obj.setA(t.image); }

and in the parseexception case I attach the "currentField" value to the error msg. How would a pro do this? :-)

---

first i tried this:
| <A: N5>
| <B: N5>
...

Since <N5> was the first public occurrance ... all <A.>, <B.> (dots for non-html markup) always showed up as <N5> which isnt very distinctive.

---

2)
| <N5: <NUM_CHAR><NUM_CHAR><NUM_CHAR><NUM_CHAR><NUM_CHAR> >

According to the FAQ this should work but doesnt:
| <N5: <NUM_CHAR>{5} >

???

Using JavaCC with spring

2009-10-29T13:14:24Z

I am using JavaCC generated parser with spring framework. I define a
singleton instance of my Parser in spring XMl configuration. When I need
to use the parser I need to set it to a new Reader or InputStream. To do
this I needed to add an argument-less constructor and a setInputStream
method as follows:

public FunctionParser() {
}

public void setInputStream(InputStream stream, String encoding) {
//Following is copied from generated constructor with same args
try { jj_input_stream = new SimpleCharStream(stream, encoding,
1, 1); } catch(java.io.UnsupportedEncodingException e) { throw new
RuntimeException(e); }
token_source = new FunctionParserTokenManager(jj_input_stream);
token = new Token();
jj_ntk = -1;
jj_gen = 0;
for (int i = 0; i < 3; i++) jj_la1[i] = -1;
for (int i = 0; i < jj_2_rtns.length; i++) jj_2_rtns[i] = new
JJCalls();
}

This is a hack because the above code copied from generated constructor
with same args changes as grammar changes.
Is there a better way to do this?

--
Regards,
Farrukh

Web: http://www.wellfleetsoftware.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Matching productions embedded in arbitrary text

2009-10-29T10:03:33Z

Farrukh Najmi wrote:

> Tom Copeland wrote:
>> To me the tricky bit here is that you don't know the function names
>> until runtime, so an input string like this:
>>
>> texttext fiddle("a", 1)texttext
>>
>> would only be a FunctionCall if the name "fiddle" is in
>> FUNCTION_NAMES, right?
>>
>> With that constraint, I'm not sure how to get JavaCC to tokenize the
>> input data. I wonder if you might want to extract the function
>> declarations using a regex. You could use a regex like ".* (" +
>> someFunctionName + "(.*))" to extract the function declaration from
>> the surrounding text, and then you could pass the match data to
>> JavaCC to create the parse tree. This would only work if it's just a
>> function call, though, it wouldn't work if the function body were
>> there because that could include nested parenthesis which would
>> confuse the regex.
>
> Hi Tom,
>
> My experience suggests that part seems easy. You just define a
> LOOKAHEAD that uses a function seeFunctionCall() where the function
> looks at next few tokens and decides whether it has a function name or
> not. See example below...
>
> String textOrFunctionCall():
> {
> String rs = new String("");
> String ts;
> }
> {
> (
> LOOKAHEAD({!seeFunctionCall()}) ts = text() {rs = rs + ts;}
> | LOOKAHEAD({seeFunctionCall()}) ts = FunctionCall() {rs = rs + " "
> + ts + " ";}
> )
> {
> return rs;
> }
> }
>

Now that my parser is working with some place holder production for
matching the arbitrary text I now need to make it work for matching
arbitrary text that is not a function call. To me this is the real
challenge (as I said above the user configured function names is a
simple issue that I already have sorted out).

Let me as a more focused question.

Is it possible to define a grammar that can have occurrence of a
specified production within *arbitrary text*? If so does any one have
some sample of what that might look like?

Thanks.

--
Regards,
Farrukh

Web: http://www.wellfleetsoftware.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Matching productions embedded in arbitrary text

2009-10-29T07:36:51Z

Tom Copeland wrote:

>
> On Oct 28, 2009, at 8:56 PM, Farrukh Najmi wrote:
>
>> Hi Chris and Tom,
>>
>> Thank you for your help. Please see inline below...
>>
>> J.Chris Findlay wrote:
>>> I think it might need to be:
>>> texttexttextfoobar("arg1",2.0)texttexttext
>>
>> Above is more closer to what I intended. The only thing to add is that
>> I think the function name would have to be preceded by white space so
>> here a modified version of your example that would be valid:
>>
>> texttexttext foobar("arg1",2.0)texttexttext
>>
>> BTW, to provide some context, I work on ebXML RegRep specificationm.
>> The spec supports multiple query languages (e.g. SQL, EJBQL, SPARQL,
>> XQuery....) and is defining a syntax for user defined functions
>> (UDF). Since the functions could be embedded in any query language
>> expression I cannot assume a specific grammar.
>> I have made some progress on my parser by temporarily changing the
>> text() production to be a choice of various types of tokens for now.
>> So any suggestion on how to handle the arbitrary text requirement
>> would be great.
>
> To me the tricky bit here is that you don't know the function names
> until runtime, so an input string like this:
>
> texttext fiddle("a", 1)texttext
>
> would only be a FunctionCall if the name "fiddle" is in
> FUNCTION_NAMES, right?
>
> With that constraint, I'm not sure how to get JavaCC to tokenize the
> input data. I wonder if you might want to extract the function
> declarations using a regex. You could use a regex like ".* (" +
> someFunctionName + "(.*))" to extract the function declaration from
> the surrounding text, and then you could pass the match data to JavaCC
> to create the parse tree. This would only work if it's just a
> function call, though, it wouldn't work if the function body were
> there because that could include nested parenthesis which would
> confuse the regex.

Hi Tom,

My experience suggests that part seems easy. You just define a LOOKAHEAD
that uses a function seeFunctionCall() where the function looks at next
few tokens and decides whether it has a function name or not. See
example below...

String textOrFunctionCall():
{
String rs = new String("");
String ts;
}
{
(
LOOKAHEAD({!seeFunctionCall()}) ts = text() {rs = rs + ts;}
| LOOKAHEAD({seeFunctionCall()}) ts = FunctionCall() {rs = rs + " "
+ ts + " ";}
)
{
return rs;
}
}

--
Regards,
Farrukh

Web: http://www.wellfleetsoftware.com

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Matching productions embedded in arbitrary text

2009-10-29T07:26:17Z

On Oct 28, 2009, at 8:56 PM, Farrukh Najmi wrote:

> Hi Chris and Tom,
>
> Thank you for your help. Please see inline below...
>
> J.Chris Findlay wrote:
>> I think it might need to be:
>> texttexttextfoobar("arg1",2.0)texttexttext
>
> Above is more closer to what I intended. The only thing to add is that
> I think the function name would have to be preceded by white space
> so here a modified version of your example that would be valid:
>
> texttexttext foobar("arg1",2.0)texttexttext
>
> BTW, to provide some context, I work on ebXML RegRep specificationm.
> The spec supports multiple query languages (e.g. SQL, EJBQL, SPARQL,
> XQuery....) and is defining a syntax for user defined functions
> (UDF). Since the functions could be embedded in any query language
> expression I cannot assume a specific grammar.
> I have made some progress on my parser by temporarily changing the
> text() production to be a choice of various types of tokens for now.
> So any suggestion on how to handle the arbitrary text requirement
> would be great.

To me the tricky bit here is that you don't know the function names
until runtime, so an input string like this:

texttext fiddle("a", 1)texttext

would only be a FunctionCall if the name "fiddle" is in
FUNCTION_NAMES, right?

With that constraint, I'm not sure how to get JavaCC to tokenize the
input data. I wonder if you might want to extract the function
declarations using a regex. You could use a regex like ".* (" +
someFunctionName + "(.*))" to extract the function declaration from
the surrounding text, and then you could pass the match data to JavaCC
to create the parse tree. This would only work if it's just a
function call, though, it wouldn't work if the function body were
there because that could include nested parenthesis which would
confuse the regex.

Yours,

Tom
http://generatingparserswithjavacc.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Matching productions embedded in arbitrary text

2009-10-28T17:56:39Z

Hi Chris and Tom,

Thank you for your help. Please see inline below...

J.Chris Findlay wrote:
> I think it might need to be:
> texttexttextfoobar("arg1",2.0)texttexttext

Above is more closer to what I intended. The only thing to add is that
I think the function name would have to be preceded by white space so
here a modified version of your example that would be valid:

texttexttext foobar("arg1",2.0)texttexttext

BTW, to provide some context, I work on ebXML RegRep specificationm. The
spec supports multiple query languages (e.g. SQL, EJBQL, SPARQL,
XQuery....) and is defining a syntax for user defined functions (UDF).
Since the functions could be embedded in any query language expression
I cannot assume a specific grammar.

I have made some progress on my parser by temporarily changing the
text() production to be a choice of various types of tokens for now.
So any suggestion on how to handle the arbitrary text requirement would
be great.

Thanks again.

>
> On Thu, Oct 29, 2009 at 11:55 AM, Tom Copeland <tom@...
> <mailto:tom@...>> wrote:
>
>
...

>
> Hi Farrukh -
>
> In this case the error message is indeed accurate; since
> (anything)* is a "zero or more" expansion it can be matched by
> nothing at all. Thus it could just sit there in an infinite loop.
>
> Let's see though... so that I understand it, assuming that
> "foobar" is one of the FUNCTION_NAMES in the HashSet, would this
> be valid input data?
>
> texttexttextfoobar(arg1,arg2)texttexttext
>
> Thanks,
>
> Tom
> http://generatingparserswithjavacc.com/
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> <mailto:users-unsubscribe@...>
> For additional commands, e-mail: users-help@...
> <mailto:users-help@...>
>
>
>
>
> --
> - J.Chris Findlay
> (c:

Re: Matching productions embedded in arbitrary text

2009-10-28T16:04:02Z

I think it might need to be:
texttexttextfoobar("arg1",2.0)texttexttext

On Thu, Oct 29, 2009 at 11:55 AM, Tom Copeland <tom@...> wrote:

On Oct 28, 2009, at 9:50 AM, Farrukh Najmi wrote:

Hi Guys,

I need some help with a grammar (see attached).

The grammar processes text that consists of function calls embedded any where and possibly multiple time within arbitrary text.

The FunctionCall() production is expected to be recognized when a function name (configurable via setFUNCTION_NAMES) is matched in the text.
Arbitrary text is expected to be recognized by the text() production.

When I run javacc I get the following error:

Error: Line 73, Column 5: Expansion within "(...)*" can be matched by empty string.

Can any one help me get over the hump with this problem? If there is a better way to do what I am trying to do please let me know.

Thanks in advance for your help.

Hi Farrukh -

In this case the error message is indeed accurate; since (anything)* is a "zero or more" expansion it can be matched by nothing at all. Thus it could just sit there in an infinite loop.

Let's see though... so that I understand it, assuming that "foobar" is one of the FUNCTION_NAMES in the HashSet, would this be valid input data?

texttexttextfoobar(arg1,arg2)texttexttext

Thanks,

Tom
http://generatingparserswithjavacc.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

--
- J.Chris Findlay
(c:

Re: Matching productions embedded in arbitrary text

2009-10-28T15:55:51Z

On Oct 28, 2009, at 9:50 AM, Farrukh Najmi wrote:

> Hi Guys,
>
> I need some help with a grammar (see attached).
>
> The grammar processes text that consists of function calls embedded
> any where and possibly multiple time within arbitrary text.
>
> The FunctionCall() production is expected to be recognized when a
> function name (configurable via setFUNCTION_NAMES) is matched in the
> text.
> Arbitrary text is expected to be recognized by the text() production.
>
> When I run javacc I get the following error:
>
> Error: Line 73, Column 5: Expansion within "(...)*" can be matched
> by empty string.
>
> Can any one help me get over the hump with this problem? If there is
> a better way to do what I am trying to do please let me know.
>
> Thanks in advance for your help.

Hi Farrukh -

In this case the error message is indeed accurate; since (anything)*
is a "zero or more" expansion it can be matched by nothing at all.
Thus it could just sit there in an infinite loop.

Let's see though... so that I understand it, assuming that "foobar" is
one of the FUNCTION_NAMES in the HashSet, would this be valid input
data?

texttexttextfoobar(arg1,arg2)texttexttext

Thanks,

Tom
http://generatingparserswithjavacc.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Question on examples/Lookahead

2009-10-28T15:14:13Z

On Oct 20, 2009, at 5:41 PM, Riadh Elloumi wrote:

> Hi all,
>
> I am newbie to JavaCC and I started by the tutorials. I tried the
> examples/Lookahead/Example1.jj and I am surprised that the grammar
> does not accept "abc" as it is said on the tutorial. I am using
> JavaCC 5.0 and I compile and run the example with these commands:
>
> riadh@giscard:Lookahead$ ../../bin/javacc Example1.jj
> Java Compiler Compiler Version 5.0 (Parser Generator)
> (type "javacc" with no arguments for help)
> Reading from file Example1.jj . . .
> Warning: Choice conflict in [...] construct at line 60, column 7.
> Expansion nested within construct and expansion following
> construct
> have common prefixes, one of which is: "c"
> Consider using a lookahead of 2 or more for nested expansion.
> File "TokenMgrError.java" is being rebuilt.
> File "ParseException.java" is being rebuilt.
> File "Token.java" is being rebuilt.
> File "SimpleCharStream.java" is being rebuilt.
> Parser generated with 0 errors and 1 warnings.
> riadh@giscard:Lookahead$ javac *.java
> riadh@giscard:Lookahead$ java Example
> abc
> Exception in thread "main" ParseException: Encountered "<EOF>" at
> line 1, column 4.
> Was expecting:
> "c" ...
>
> at Example.generateParseException(Example.java:206)
> at Example.jj_consume_token(Example.java:144)
> at Example.Input(Example.java:12)
> at Example.main(Example.java:6)
> riadh@giscard:Lookahead$
>
> Can you help me to understand why "abc" is not accepted?

Hello Riadh -

I think the idea with examples/Lookahead/Example1.jj is to illustrate
how backtracking affects a grammar. In terms of BNF, Example1.jj can
handle "abc", but since JavaCC parsers don't backtrack, the parser
that JavaCC generates can't. If you look at section 6 of the example/
Lookahead/README file you'll see a discussion of how to solve the
problem in Example1.jj with semantic lookahead.

Hope this helps,

Yours,

Tom
http://generatingparserswithjavacc.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Matching productions embedded in arbitrary text

2009-10-28T10:17:02Z

I would be grateful for *any* general advice on the feasability of my
parser's goals.
Is it even possible to match specific tokens intermixed with arbitrary
text? Has anyone done
a similar parser?

Farrukh Najmi wrote:

> Hi Guys,
>
> I need some help with a grammar (see attached).
>
> The grammar processes text that consists of function calls embedded
> any where and possibly multiple time within arbitrary text.
>
> The FunctionCall() production is expected to be recognized when a
> function name (configurable via setFUNCTION_NAMES) is matched in the
> text.
> Arbitrary text is expected to be recognized by the text() production.
>
> When I run javacc I get the following error:
>
> Error: Line 73, Column 5: Expansion within "(...)*" can be matched by
> empty string.
>
> Can any one help me get over the hump with this problem? If there is a
> better way to do what I am trying to do please let me know.
>
> Thanks in advance for your help.

Re: Matching productions embedded in arbitrary text

2009-10-28T09:53:10Z

The base problem is that in some cases, a tokenizer cannot unambiguously
define the token kind.

This is similar to a parsing problem with PL/I (which has no reserved
keywords). In "IF(...) ... ;" IF can be either a keyword or a variable
name and it cannot be determined until the token after the matching
right paren is examined. There may be any number of tokens (e.g.
expressions) between the parens. One approach that works is to tokenize
IF as a keyword and then use a specialized lookahead routine that alters
the "kind" field in the IF token to "identifier" if necessary (before it
is parsed).

In this case, a similar approach using a specialized lookahead to
identify the token kind (as was pointed out) is reasonably straight
forward.

Farrukh Najmi wrote:

> Tom Copeland wrote:
>>
>> On Oct 28, 2009, at 8:56 PM, Farrukh Najmi wrote:
>>
>>> Hi Chris and Tom,
>>>
>>> Thank you for your help. Please see inline below...
>>>
>>> J.Chris Findlay wrote:
>>>> I think it might need to be:
>>>> texttexttextfoobar("arg1",2.0)texttexttext
>>>
>>> Above is more closer to what I intended. The only thing to add is that
>>> I think the function name would have to be preceded by white space
>>> so here a modified version of your example that would be valid:
>>>
>>> texttexttext foobar("arg1",2.0)texttexttext
>>>
>>> BTW, to provide some context, I work on ebXML RegRep specificationm.
>>> The spec supports multiple query languages (e.g. SQL, EJBQL, SPARQL,
>>> XQuery....) and is defining a syntax for user defined functions
>>> (UDF). Since the functions could be embedded in any query language
>>> expression I cannot assume a specific grammar.
>>> I have made some progress on my parser by temporarily changing the
>>> text() production to be a choice of various types of tokens for now.
>>> So any suggestion on how to handle the arbitrary text requirement
>>> would be great.
>>
>> To me the tricky bit here is that you don't know the function names
>> until runtime, so an input string like this:
>>
>> texttext fiddle("a", 1)texttext
>>
>> would only be a FunctionCall if the name "fiddle" is in
>> FUNCTION_NAMES, right?
>>
>> With that constraint, I'm not sure how to get JavaCC to tokenize the
>> input data. I wonder if you might want to extract the function
>> declarations using a regex. You could use a regex like ".* (" +
>> someFunctionName + "(.*))" to extract the function declaration from
>> the surrounding text, and then you could pass the match data to
>> JavaCC to create the parse tree. This would only work if it's just a
>> function call, though, it wouldn't work if the function body were
>> there because that could include nested parenthesis which would
>> confuse the regex.
>
> Hi Tom,
>
> My experience suggests that part seems easy. You just define a
> LOOKAHEAD that uses a function seeFunctionCall() where the function
> looks at next few tokens and decides whether it has a function name or
> not. See example below...
>
> String textOrFunctionCall():
> {
> String rs = new String("");
> String ts;
> }
> {
> (
> LOOKAHEAD({!seeFunctionCall()}) ts = text() {rs = rs + ts;}
> | LOOKAHEAD({seeFunctionCall()}) ts = FunctionCall() {rs = rs + " "
> + ts + " ";}
> )
> {
> return rs;
> }
> }
>
> ------------------------------------------------------------------------
>
>
> No virus found in this incoming message.
> Checked by AVG - www.avg.com
> Version: 8.5.423 / Virus Database: 270.14.38/2467 - Release Date: 10/29/09 07:38:00
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Matching productions embedded in arbitrary text

2009-10-28T06:50:00Z

Hi Guys,

I need some help with a grammar (see attached).

The grammar processes text that consists of function calls embedded any
where and possibly multiple time within arbitrary text.

The FunctionCall() production is expected to be recognized when a
function name (configurable via setFUNCTION_NAMES) is matched in the text.
Arbitrary text is expected to be recognized by the text() production.

When I run javacc I get the following error:

Error: Line 73, Column 5: Expansion within "(...)*" can be matched by
empty string.

Can any one help me get over the hump with this problem? If there is a
better way to do what I am trying to do please let me know.

Thanks in advance for your help.

--
Regards,
Farrukh

Web: http://www.wellfleetsoftware.com

/**
*
*
*/

options{
STATIC=false ;
IGNORE_CASE=true ;
// DEBUG_LOOKAHEAD= true ;
}

PARSER_BEGIN(FunctionParser)
package com.example.function;

import java.io.Reader;
import java.io.FileInputStream;
import java.util.HashSet;
import java.util.Set;

public class FunctionParser {
protected static Set<String> FUNCTION_NAMES = new HashSet<String>();

public void reInit(Reader input) {
ReInit(input);
}

protected boolean seeFunctionCall() {
return "(".equals(getToken(2).image)
&& FUNCTION_NAMES.contains(getToken(1).image.toUpperCase());
}

public static void setFUNCTION_NAMES(Set<String> functionNames) {
FUNCTION_NAMES = functionNames;
}
}
PARSER_END(FunctionParser)

SKIP:
{
" "
| "\t"
| "\r"
| "\n"
}

TOKEN : /* Numeric Constants */
{
< S_NUMBER: <FLOAT>
| <FLOAT> ( ["e","E"] ([ "-","+"])? <FLOAT> )?
>
| < #FLOAT: <INTEGER>
| <INTEGER> ( "." <INTEGER> )?
| "." <INTEGER>
>
| < #INTEGER: ( <DIGIT> )+ >
| < #DIGIT: ["0" - "9"] >
}

TOKEN:
{
< S_IDENTIFIER: (<LETTER>)+ (<DIGIT> | <LETTER> |<SPECIAL_CHARS>)* >
| < #LETTER: ["a"-"z", "A"-"Z"] >
| < #SPECIAL_CHARS: "_" >
| < S_CHAR_LITERAL: "'" (~["'"])* "'" ("'" (~["'"])* "'")*>
| < S_QUOTED_IDENTIFIER: "\"" (~["\n","\r","\""])* "\"" >
| < S_TEXT: ~[] >
}

void start():
{}
{
(textOrFunctionCall())*

}

void text():
{}
{
( <S_TEXT> )*
}

void textOrFunctionCall():
{}
{
text()
| LOOKAHEAD({seeFunctionCall()}) FunctionCall()
}

/* ---------------- Grammar Productions --------------------- */

void FunctionCall():
{}
{
FunctionReference() "(" (FunctionArgumentList())* ")"
}

void FunctionReference():
{
String name;
}
{
<S_IDENTIFIER> {name = token.image;}

}

void FunctionArgumentList():
{}
{
FunctionArgument() ("," FunctionArgument())*
}

void FunctionArgument():
{}
{
FunctionCall()
| <S_QUOTED_IDENTIFIER>
| <S_NUMBER>
}

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Update of the JavaCC cobol grammar

2009-10-28T02:42:54Z

Hello,
I would like to contribute to this wonderful project by updating the JavaCC cobol grammar that was initially made by Bernard Pinon (and that can be found in the repository of JavaCC grammars).
I've heavily modified and improved this grammar, although it is (and i think will) never be perfect. The grammar was well tested with java 4.2 in a production environment.
I've attached the new grammar file to this mail. I left some "TODO" comments in this file, for some pending improvements, but those "ugly" comments can be removed if necessary.
I don't know if it is the right address to sind this file, hope I'm not wrong ;)

Comments and suggestions are welcome :)

Best regards,
Thierry BLIND.

Nouveau Windows 7 : Trouvez le PC qui vous convient. En savoir plus.
/**
* This is an experimental COBOL-85 grammar for JavaCC.
* <p>References used for this grammar :<ol>
* <li> Ralf Lämmel & Chris Verhoef : VS COBOL II grammar Version 1.0.3
* <li> Fujitsu Cobol compiler reference
* <li> Compaq-HP Non-stop Cobol reference
* </od>
* <p>This grammar is released under the GNU Library Public License (LGPL).
* <p><address>Bernard Pinon, august 2002</address>
* <p><address>Thierry Blind, september 2009</address>
* @see http://www.gnu.org/licenses/lgpl.html
* @see http://www.cwi.nl/
* @see http://www.wins.uva.nl/
* @author Thierry Blind (initial work from Bernard Pinon)
*
* <p>History:<ul>
* <li><date>9-aug-2002</date> [BP] make changes to comply with JTB :<ol>
* <li> Changed Integer() to IntegerConstant() to avoid conflicts with java.lang.Integer
* <li> Removed Java code to make the grammar bare bone
* <li> Fixed Identification Division entries
* </ol>
* <li><date>8-aug-2002</date> [BP] fixed several bugs :<ol>
* <li> INSTALLATION section
* <li> wrong spelling in keywords
* <li> DOT missing in FD/SD entries
* <li> wrong regular expressions for levels
* <li> possibility to have repeated string literals, e.g. "AA" "BB" -> "AABB"
* <li> optimized data division by placing the most common clauses first
* <li> DOT missing in exit statement
* <li> confusion between Identifier and QualifiedDataName (to be checked)
* <li> fixed NumericConstant to use Integer instead of INTEGER
* <li> Correcting AdvancingPhrase so LINE is optional
* <li> Fixed problems with abbreviated conditions
* <li> Fixed Add statement
* </ol>
* <li><date>01-sept-2009</date> :<ol>
* <li> corrected picture values and enhanced linkage section
* <li> corrected relational operators, arithmetic expressions and conditions
* <li> corrected perform, evaluate and call clauses
* <li> corrected cobol word, litteral, identifier, comment declarations
* <li> created Iterables*() variants for some clauses
* <li> added function support from cobol 2000
* <li> added LOOKAHEAD() instructions, with reorderings, to remove parser ambiguities
* <li> ... and many many corrections, new clauses and new tokens relative to cobol 85 and (some from) cobol 2000
* </ol>
* </ul>
*/

/**
* Cobol is not case sensitive. It is not LL(1) neither...
*/

options {
IGNORE_CASE = true;
//FORCE_LA_CHECK = true;
LOOKAHEAD = 3;
//DEBUG_PARSER = true;
//DEBUG_LOOKAHEAD = true;
STATIC = false;
}

/**
* The parser class.
*/

PARSER_BEGIN(CobolParser)
package convex.step2.parser;

public class CobolParser {

}

PARSER_END(CobolParser)

////////////////////////////////////////////////////////////////////////////////
// Lexical structure
////////////////////////////////////////////////////////////////////////////////

SPECIAL_TOKEN :
{
< EOL: "\n" >
| < SPACECHAR: ( " " | "\t" | "\f" | ";" | "\r" )+ >
}

SPECIAL_TOKEN :
{
< COMMENT: ( "*>" | "|" ) (~["\n","\r"])* >
| < PREPROC_COMMENT: "*|" (~["\n","\r"])* >
| < SPACE_SEPARATOR : ( <SPACECHAR> | <EOL> )+ >
| < COMMA_SEPARATOR : "," <SPACE_SEPARATOR> >
}

TOKEN :
{
// TODO : Could be simplified by <DOTCHAR> ?
< DOT : <DOTCHAR> <SPACE_SEPARATOR> >
}

////////////////////////////////////////////////////////////////////////////////
// RESERVED WORDS. Looks like an excerpt of the dictionary of data processing.
// It is very hard to find significative identifiers in COBOL that are not
// reserved...
////////////////////////////////////////////////////////////////////////////////

TOKEN :
{
/* A */
< ACCEPT: "accept" >
| < ACCESS: "access" >
| < ADD: "add" >
| < ADDRESS: "address" > // extension to support pointers
| < ADVANCING: "advancing" >
| < AFTER: "after" >
| < ALL: "all" >
| < ALPHABET: "alphabet" >
| < ALPHABETIC: "alphabetic" >
| < ALPHABETIC_LOWER: "alphabetic-lower" >
| < ALPHABETIC_UPPER: "alphabetic-upper" >
| < ALPHANUMERIC: "alphanumeric" >
| < ALPHANUMERIC_EDITED: "alphanumeric-edited" >
| < ALSO: "also" >
| < ALTER: "alter" >
| < ALTERNATE: "alternate" >
| < AND: "and" >
| < ANY: "any" >
| < APPROXIMATE: "approximate" > // tandem extension
| < ARE: "are" >
| < AREA: "area" >
| < AREAS: "areas" >
| < ASCENDING: "ascending" >
| < ASSIGN: "assign" >
| < AT: "at" >
| < AUTHOR: "author" >
/* B */
| < BEFORE: "before" >
| < BEGINNING: "beginning" > // extension
| < BINARY: "binary" >
| < BLANK: "blank" >
| < BLOCK: "block" >
| < BOTTOM: "bottom" >
| < BY: "by" >
/* C */
| < CALL: "call" >
| < CALL_CONVENTION: "call-convention" >
| < CANCEL: "cancel" >
| < CD: "cd" >
| < CF: "cf" >
| < CH: "ch" >
| < CHARACTER: "character" >
| < CHARACTERS: "characters" >
| < CLASS: "class" >
| < CLOCK_UNITS: "clock-units" >
| < CLOSE: "close" >
//| < COBOL: "cobol" >
| < CODE: "code" >
| < CODE_SET: "code-set" >
| < COLLATING: "collating" >
| < COLUMN: "column" >
| < COMMA: "comma" >
| < COMMON: "common" >
| < COMMUNICATION: "communication" >
| < COMP: "comp" >
| < COMP_1: "comp-1" > // extension in many dialects
| < COMP_2: "comp-2" > // extension in many dialects
| < COMP_3: "comp-3" > // extension in many dialects
| < COMP_4: "comp-4" > // extension in many dialects
| < COMPUTATIONAL: "computational" >
| < COMPUTATIONAL_1: "computational-1" > // extension
| < COMPUTATIONAL_2: "computational-2" > // extension
| < COMPUTATIONAL_3: "computational-3" > // extension
| < COMPUTATIONAL_4: "computational-4" > // extension
| < COMPUTE: "compute" >
| < CONFIGURATION: "configuration" >
| < CONTAINS: "contains" >
| < CONTENT: "content" >
| < CONTINUE: "continue" >
| < CONTROL: "control" >
| < CONTROLS: "controls" >
| < CONVERTING: "converting" >
| < COPY: "copy" >
| < CORR: "corr" >
| < CORRESPONDING: "corresponding" >
| < COUNT: "count" >
| < CURRENCY: "currency" >
/* D */
| < DATA: "data" >
| < DATE: "date" >
| < DATE_COMPILED: "date-compiled" >
| < DATE_WRITTEN: "date-written" >
| < DAY: "day" >
| < DAY_OF_WEEK: "day-of-week" >
| < DBCS: "dbcs" > // extension to support double bytes characters
| < DE: "de" >
| < DEBUG_CONTENTS: "debug-contents" >
| < DEBUG_ITEM: "debug-item" >
| < DEBUG_LINE: "debug-line" >
| < DEBUG_NAME: "debug-name" >
| < DEBUG_SUB_1: "debug-sub-1" >
| < DEBUG_SUB_2: "debug-sub-2" >
| < DEBUG_SUB_3: "debug-sub-3" >
| < DEBUGGING: "debugging" >
| < DECIMAL_POINT: "decimal-point" >
| < DECLARATIVES: "declaratives" >
| < DELETE: "delete" >
| < DELIMITED: "delimited" >
| < DELIMITER: "delimiter" >
| < DEPENDING: "depending" >
| < DESCENDING: "descending" >
| < DESCRIPTOR: "descriptor" >
| < DESTINATION: "destination" >
| < DETAIL: "detail" >
| < DISABLE: "disable" >
| < DISPLAY: "display" >
| < DISPLAY_1: "display-1" > // extension
| < DIVIDE: "divide" >
| < DIVISION: "division" >
| < DOWN: "down" >
| < DUPLICATES: "duplicates" >
| < DYNAMIC: "dynamic" >
/* E */
| < EGCS: "egcs" > // extension
| < EGI: "egi" >
| < ELSE: "else" >
| < EMI: "emi" >
| < ENABLE: "enable" >
| < END: "end" >
| < END_ADD: "end-add" >
| < END_CALL: "end-call" >
| < END_COMPUTE: "end-compute" >
| < END_DELETE: "end-delete" >
| < END_DIVIDE: "end-divide" >
| < END_EVALUATE: "end-evaluate" >
| < END_IF: "end-if" >
| < END_MULTIPLY: "end-multiply" >
| < END_OF_PAGE: "end-of-page" >
| < END_PERFORM: "end-perform" >
| < END_READ: "end-read" >
| < END_RECEIVE: "end-receive" >
| < END_RETURN: "end-return" >
| < END_REWRITE: "end-rewrite" >
| < END_SEARCH: "end-search" >
| < END_START: "end-start" >
| < END_STRING: "end-string" >
| < END_SUBTRACT: "end-subtract" >
| < END_UNSTRING: "end-unstring" >
| < END_WRITE: "end-write" >
| < ENDING: "endinf" >
| < ENTER: "enter" > // not present in all COBOLs, use CALL instead
| < ENTRY: "entry" >
| < ENVIRONMENT: "environment" >
| < EOP: "eop" >
| < EQUAL: "equal" >
| < ERROR: "error" >
| < ESI: "esi" >
| < EVALUATE: "evaluate" >
| < EVERY: "every" >
| < EXCEPTION: "exception" >
/* | < EXCLUSIVE: "exclusive" > tandem extension */
| < EXIT: "exit" >
| < EXTEND: "extend" >
| < EXTERNAL: "external" >
/* F */
| < FALSE: "false" >
| < FD: "fd" >
| < FILE: "file" >
| < FILE_CONTROL: "file-control" >
| < FILLER: "filler" >
| < FINAL: "final" >
| < FIRST: "first" >
| < FOOTING: "footing" >
| < FOR: "for" >
| < FROM: "from" >
| < FUNCTION: "function" >
/* G */
| < GENERATE: "generate" >
| < GOBACK: "goback" > // extension
| < GENERIC: "generic" > // tandem extension
| < GIVING: "giving" >
| < GLOBAL: "global" >
| < GO: "go" >
| < GREATER: "greater" >
| < GROUP: "group" >
/* H */
| < HEADING: "heading" >
| < HIGH_VALUE: "high-value" >
| < HIGH_VALUES: "high-values" >
/* I */
| < I_O: "i-o" >
| < I_O_CONTROL: "i-o-control" >
| < ID: "id" > // extension, synonym for IDENTIFICATION
| < IDENTIFICATION: "identification" >
| < IDENTIFIED: "identified" >
| < IF: "if" >
| < IMPLICIT: "implicit" >
| < IN: "in" >
| < INDEX: "index" >
| < INDEXED: "indexed" >
| < INDICATE: "indicate" >
| < INITIAL: "initial" >
| < INITIALIZE: "initialize" >
| < INITIATE: "initiate" >
| < INPUT: "input" >
| < INPUT_OUTPUT: "input-output" >
| < INSPECT: "inspect" >
| < INSTALLATION: "installation" >
| < INTO: "into" >
| < INVALID: "invalid" >
| < IS: "is" >
/* J */
| < JUST: "just" >
| < JUSTIFIED: "justified" >
| < JUSTIFY: "justify" >
/* K */
| < KANJI: "kanji" > // extension to support Kanji characters (japanese)
| < KEY: "key" >
/* L */
| < LABEL: "label" >
| < LAST: "last" >
| < LEADING: "leading" >
| < LEFT: "left" >
| < LENGTH: "length" >
| < LESS: "less" >
| < LIMIT: "limit" >
| < LIMITS: "limits" >
| < LINAGE: "linage" >
| < LINAGE_COUNTER: "linage_counter" >
| < LINE: "line" >
| < LINES: "lines" >
| < LINE_COUNTER: "line-counter" >
| < LINKAGE: "linkage" >
| < LOCK: "lock" >
| < LOCKFILE: "lockfile" > // tandem extension
| < LOW_VALUE: "low-value" >
| < LOW_VALUES: "low-values" >
/* M */
| < MEMORY: "memory" >
| < MERGE: "merge" >
| < MESSAGE: "message" >
| < MODE: "mode" >
| < MODULES: "modules" >
| < MORE_LABELS: "more-labels" > // IBM extension
| < MOVE: "move" >
| < MULTIPLE: "multiple" >
| < MULTIPLY: "multiply" >
/* N */
| < NATIVE: "native" >
| < NEGATIVE: "negative" >
| < NEXT: "next" >
| < NO: "no" >
| < NOT: "not" >
| < NULL: "null" > // tandem & IBM extension
| < NULLS: "nulls" > // tandem & IBM extension
| < NUMBER: "number" >
| < NUMERIC: "numeric" >
| < NUMERIC_EDITED: "numeric-edited" >
/* O */
| < OBJECT_COMPUTER: "object-computer" >
| < OCCURS: "occurs" >
| < OF: "of" >
| < OFF: "off" >
| < OMITTED: "omitted" >
| < ON: "on" >
| < OPEN: "open" >
| < OPTIONAL: "optional" >
| < OR: "or" >
| < ORDER: "order" >
| < ORGANIZATION: "organization" >
| < OTHER: "other" >
| < OUTPUT: "output" >
| < OVERFLOW: "overflow" >
/* P */
| < PACKED_DECIMAL: "packed-decimal" >
| < PADDING: "padding" >
| < PAGE: "page" >
| < PAGE_COUNTER: "page-counter" >
| < PASSWORD: "password" >
| < PERFORM: "perform" >
| < PF: "pf" >
| < PH: "ph" >
| < PIC: "pic" >
| < PICTURE: "picture" >
| < PLUS: "plus" >
| < POINTER: "pointer" >
| < POSITION: "position" >
| < POSITIVE: "positive" >
| < PRINTING: "printing" >
| < PROCEDURE: "procedure" >
| < PROCEDURES: "procedures" >
| < PROCEDURE_POINTER: "procedure-pointer" >
| < PROCEED: "proceed" >
| < PROGRAM: "program" >
| < PROGRAM_ID: "program-id" >
| < PROGRAM_STATUS: "program-status" > // tandem extension
| < PROMPT: "prompt" > // tandem extension
| < PROTECTED: "protected" > // tandem extension
| < PURGE: "purge" >
/* Q */
| < QUEUE: "queue" >
| < QUOTE: "quote" >
| < QUOTES: "quotes" >
/* R */
| < RANDOM: "random" >
| < RD: "rd" >
| < READ: "read" >
| < RECEIVE: "receive" >
| < RECEIVE_CONTROL: "receive-control" > // tandem extension
| < RECORD: "record" >
| < RECORDING: "recording" > // probably IBM extension
| < RECORDS: "records" >
| < REDEFINES: "redefines" >
| < REEL: "reel" >
| < REFERENCE: "reference" >
| < REFERENCES: "references" >
| < RELATIVE: "relative" >
| < RELEASE: "release" >
| < REMAINDER: "remainder" >
| < REMOVAL: "removal" >
| < RENAMES: "renames" >
| < REPLACE: "replace" >
| < REPLACING: "replacing" >
| < REPLY: "reply" > // tandem extension
| < REPORT: "report" >
| < REPORTING: "reporting" >
| < REPORTS: "reports" >
| < RERUN: "rerun" >
| < RESERVE: "reserve" >
| < RESET: "reset" >
| < RETURN: "return" > // IBM extension - does not do what expected ;-)
| < RETURN_CODE: "return-code" > // special register IBM
| < RETURNED: "returned" >
| < RETURNING: "returning" >
| < REVERSED: "reversed" >
| < REWIND: "rewind" >
| < REWRITE: "rewrite" >
| < RF: "rf" >
| < RH: "rh" >
| < RIGHT: "right" >
| < ROUNDED: "rounded" >
| < RUN: "run" >
/* S */
| < SAME: "same" >
| < SD: "sd" >
| < SEARCH: "search" >
| < SECTION: "section" >
| < SECURITY: "security" >
| < SEGMENT: "segment" >
| < SEGMENT_LIMIT: "segment-limit" >
| < SELECT: "select" >
| < SEND: "send" >
| < SENTENCE: "sentence" >
| < SEPARATE: "separate" >
| < SEQUENCE: "sequence" >
| < SEQUENTIAL: "sequential" >
| < SET: "set" >
| < SHARED: "shared" > // tandem extension, should be in C2000
| < SHIFT_IN: "shift-in" > // IBM special register
| < SHIFT_OUT: "shift-out" > // IBM special register
| < SIGN: "sign" >
| < SIZE: "size" >
| < SORT: "sort" >
| < SORT_CONTROL: "sort-control" > // IBM special register
| < SORT_CORE_SIZE: "sort-core-size" > // IBM special register
| < SORT_FILE_SIZE: "sort-file-size" > // IBM special register
| < SORT_MERGE: "sort-merge" >
| < SORT_MESSAGE: "sort-message" > // IBM special register
| < SORT_MODE_SIZE: "sort-mode-size" > // IBM special register
| < SORT_RETURN: "sort-return" > // IBM special register
| < SOURCE: "source" >
| < SOURCE_COMPUTER: "source-computer" >
| < SPACE: "space" >
| < SPACES: "spaces" >
| < SPECIAL_NAMES: "special-names" >
| < STANDARD: "standard" >
| < STANDARD_1: "standard-1" >
| < STANDARD_2: "standard-2" >
| < START: "start" >
| < STATUS: "status" >
| < STDCALL: "stdcall" >
| < STOP: "stop" >
| < STRING: "string" >
| < SUB_QUEUE_1: "sub-queue-1" >
| < SUB_QUEUE_2: "sub-queue-2" >
| < SUB_QUEUE_3: "sub-queue-3" >
| < SUBTRACT: "subtract" >
| < SUM: "sum" >
| < SUPPRESS: "suppress" >
| < SYMBOLIC: "symbolic" >
| < SYNC: "sync" >
| < SYNCHRONIZED: "synchronized" >
/* T */
| < TABLE: "table" >
| < TALLY: "tally" > // IBM special register
| < TALLYING: "tallying" >
| < TAPE: "tape" >
| < TERMINAL: "terminal" >
| < TERMINATE: "terminate" >
| < TEST: "test" >
| < TEXT: "text" >
| < THAN: "than" >
| < THEN: "then" >
| < THROUGH: "through" >
| < THRU: "thru" >
| < TIME: "time" >
| < TIMES: "times" >
| < TO: "to" >
| < TOP: "top" >
| < TRAILING: "trailing" >
| < TRUE: "true" >
| < TYPE: "type" >
/* U */
| < UNIT: "unit" >
| < UNLOCK: "unlock" > // tandem extension
| < UNLOCKFILE: "unlockfile" > // tandem again
| < UNLOCKRECORD: "unlockrecord" > // guess what
| < UNSTRING: "unstring" >
| < UNTIL: "until" >
| < UP: "up" >
| < UPON: "upon" >
| < USAGE: "usage" >
| < USE: "use" >
| < USING: "using" >
/* V */
| < VALUE: "value" >
| < VALUES: "values" >
| < VARYING: "varying" >
/* W */
| < WHEN: "when" >
| < WHEN_COMPILED: "when-compiled" > // IBM special register
| < WITH: "with" >
| < WORDS: "words" >
| < WORKING_STORAGE: "working-storage" >
| < WRITE: "write" >
/* Z */
| < ZERO: "zero" >
| < ZEROS: "zeros" >
| < ZEROES: "zeroes" >
/* Newly added */
| < ALPHANUMERIC_HASHTABLE: "alphanumeric-hashtable" >
| < BINARY_BYTE: "binary-byte" >
| < BINARY_DOUBLE: "binary-double" >
| < BINARY_LONG: "binary-long" >
| < BINARY_M: "binary-m" >
| < BINARY_REV: "binary-rev" >
| < BINARY_SHORT: "binary-short" >
| < BIT: "bit" >
| < COMPONENT: "component" >
| < COMPUTATIONAL_2_A: "computational-2-a" >
| < COMPUTATIONAL_1_A: "computational-1-a" >
| < COMPUTATIONAL_1_E: "computational-1-e" >
| < COMPUTATIONAL_1_M: "computational-1-m" >
| < COMPUTATIONAL_1_MVS: "computational-1-mvs" >
| < COMPUTATIONAL_1_REV: "computational-1-rev" >
| < COMPUTATIONAL_2_E: "computational-2-e" >
| < COMPUTATIONAL_2_M: "computational-2-m" >
| < COMPUTATIONAL_2_MVS: "computational-2-mvs" >
| < COMPUTATIONAL_2_REV: "computational-2-rev" >
| < COMPUTATIONAL_3_A: "computational-3-a" >
| < COMPUTATIONAL_3_M: "computational-3-m" >
| < COMPUTATIONAL_4_M: "computational-4-m" >
| < COMPUTATIONAL_5: "computational-5" >
| < COMPUTATIONAL_5_M: "computational-5-m" >
| < COMPUTATIONAL_6: "computational-6" >
| < COMPUTATIONAL_A: "computational-a" >
| < COMPUTATIONAL_B: "computational-b" >
| < COMPUTATIONAL_D: "computational-d" >
| < COMPUTATIONAL_M: "computational-m" >
| < COMPUTATIONAL_N: "computational-n" >
| < COMPUTATIONAL_P: "computational-p" >
| < COMPUTATIONAL_S: "computational-s" >
| < COMPUTATIONAL_X: "computational-x" >
| < COMPUTATIONAL_X_REV: "computational-x-rev" >
| < COMP_1_A: "comp-1-a" >
| < COMP_1_E: "comp-1-e" >
| < COMP_1_M: "comp-1-m" >
| < COMP_1_MVS: "comp-1-mvs" >
| < COMP_1_REV: "comp-1-rev" >
| < COMP_2_A: "comp-2-a" >
| < COMP_2_E: "comp-2-e" >
| < COMP_2_M: "comp-2-m" >
| < COMP_2_MVS: "comp-2-mvs" >
| < COMP_2_REV: "comp-2-rev" >
| < COMP_3_A: "comp-3-a" >
| < COMP_3_M: "comp-3-m" >
| < COMP_4_M: "comp-4-m" >
| < COMP_5: "comp-5" >
| < COMP_5_M: "comp-5-m" >
| < COMP_6: "comp-6" >
| < COMP_A: "comp-a" >
| < COMP_B: "comp-b" >
| < COMP_D: "comp-d" >
| < COMP_M: "comp-m" >
| < COMP_N: "comp-n" >
| < COMP_P: "comp-p" >
| < COMP_S: "comp-s" >
| < COMP_X: "comp-x" >
| < COMP_X_REV: "comp-x-rev" >
| < DEFAULT_FONT: "default-font" >
| < DISPLAY_WS: "display-ws" >
| < DOUBLE: "double" >
| < EXTERNAL_FORM: "external-form" >
| < FIXED_FONT: "fixed-font" >
| < FLOAT: "float" >
| < FLOAT_LONG: "float-long" >
| < FLOAT_SHORT: "float-short" >
| < FONT: "font" >
| < HANDLE: "handle" >
| < JBOOLEAN: "jboolean" >
| < JBYTE: "jbyte" >
| < JCHAR: "jchar" >
| < JDOUBLE: "jdouble" >
| < JFLOAT: "jfloat" >
| < JINT: "jint" >
| < JLONG: "jlong" >
| < JPACKED_DECIMAL: "jpacked-decimal" >
| < JSHORT: "jshort" >
| < JSTRING: "jstring" >
| < LARGE_FONT: "large-font" >
| < MEDIUM_FONT: "medium-font" >
| < NATIONAL: "national" >
| < NUMERIC_HASHTABLE: "numeric-hashtable" >
| < OBJECT: "object" >
| < OBJECT_HASHTABLE: "object-hashtable" >
| < PACKED_DECIMAL_A: "packed-decimal-a" >
| < PACKED_DECIMAL_E: "packed-decimal-e" >
| < PACKED_DECIMAL_H: "packed-decimal-h" >
| < PACKED_DECIMAL_I: "packed-decimal-i" >
| < PACKED_DECIMAL_M: "packed-decimal-m" >
| < SIGNED_INT: "signed-int" >
| < SIGNED_LONG: "signed-long" >
| < SIGNED_SHORT: "signed-short" >
| < SMALL_FONT: "small-font" >
| < SQLIND: "sqlind" >
| < THREAD: "thread" >
| < TRADITIONAL_FONT: "traditional-font" >
| < UNSIGNED_INT: "unsigned-int" >
| < UNSIGNED_LONG: "unsigned-long" >
| < UNSIGNED_SHORT: "unsigned-short" >
| < WINDOW: "window" >
}

TOKEN :
{
< LEVEL_66: "66" >
| < LEVEL_77: "77" >
| < LEVEL_88: "88" >
| < LEVEL_NUMBER: ( (("0")? ["1"-"9"]) | (["1"-"4"]["0"-"9"]) | "78" ) >
| < INTEGER: (["0"-"9"])+ >
| < MINUSCHAR: "-" > // a.k.a. dash
| < LPARENCHAR: "(" >
| < RPARENCHAR: ")" >
| < COLONCHAR: ":" >
| < DOTCHAR: "." >
| < COMMACHAR: "," >
| < DOUBLEDQUOTECHAR: "\"\"" >
| < QUOTECHAR: "\"" >
| < DOUBLEDAPOSTROPHE: "''" >
| < APOSTROPHE: "'" >
| < PLUSCHAR: "+" >
| < ASTERISKCHAR: "*" >
| < POWEROF: "**" >
| < SLASHCHAR: "/" >
| < DOLLARCHAR: "$" >
| < LESSTHANOREQUAL: "<=" >
| < LESSTHANCHAR: "<" >
| < MORETHANOREQUAL: ">=" >
| < MORETHANCHAR: ">" >
| < EQUALCHAR: "=" >
| < NOTEQUAL: "<>" >
| < HEXNUMBER: ["h","x"] ( ( <QUOTECHAR> ( ["0"-"9","a"-"f"] )+ <QUOTECHAR> )
| ( <APOSTROPHE> ( ["0"-"9","a"-"f"] )+ <APOSTROPHE> )
)
>
| < QUOTEDSTRING: ( <QUOTECHAR> (~["\""] | <DOUBLEDQUOTECHAR> )* <QUOTECHAR>
| <APOSTROPHE> (~["'"] | <DOUBLEDAPOSTROPHE> )* <APOSTROPHE>
)
>
// NB : COBOL_WORD can be defined as an INTEGER (or a LEVEL_66* or a LEVEL_NUMBER) too !
| < COBOL_WORD: (["a"-"z","0"-"9"])+ ( (<MINUSCHAR> | "_" )+ (["a"-"z","0"-"9"])+ )* >
| < OTHER_CHARS: ~[] >
}

// Contains at least one alphabetic, max. 30 char
void CobolWord() :
{}
{
<COBOL_WORD>
}

void QuotedText() :
{}
{
( IterableQuotedText() )+
}

void IterableQuotedText() :
{}
{
<QUOTEDSTRING> | <DOUBLEDQUOTECHAR> | <DOUBLEDAPOSTROPHE>
}

void IntegerConstant() :
{}
{
<LEVEL_66> | <LEVEL_77> | <LEVEL_88> | <LEVEL_NUMBER> | <INTEGER>
}

void NumericConstant() :
{}
{
[ <PLUSCHAR> | <MINUSCHAR> ] ( ( <DOTCHAR> | <COMMACHAR> ) IntegerConstant() | IntegerConstant() [ ( <DOTCHAR> | <COMMACHAR> ) IntegerConstant() ] )
}

void LevelNumber() :
{}
{
<LEVEL_NUMBER>
}

void FigurativeConstant() :
{}
{
<ZERO> | <ZEROS> | <ZEROES>
| <SPACE> | <SPACES>
| <HIGH_VALUE> | <HIGH_VALUES>
| <LOW_VALUE> | <LOW_VALUES>
| <QUOTE> | <QUOTES>
| <NULL> | <NULLS>
}

void NonNumericConstant() :
{}
{
( QuotedText()
| <HEXNUMBER>
)
}

void IterableNonNumericConstant() :
{}
{
( IterableQuotedText()
| <HEXNUMBER>
)
}

void Literal() :
{}
{
[ <ALL> ] ( NonNumericConstant()
| NumericConstant()
/* | <DBCS> */
| FigurativeConstant()
)
}

void IterableLiteral() :
{}
{
[ <ALL> ] ( IterableNonNumericConstant()
| NumericConstant()
/* | <DBCS> */
| FigurativeConstant()
)
}

////////////////////////////////////////////////////////////////////////////////
// LOGICAL EXPRESSIONS (e.g. in IF statements)
// one of the nice feature of COBOL is to allow for abbeviated conditions
// such as : if X less than 100 and more than 10 then ...
////////////////////////////////////////////////////////////////////////////////

void IterableCondition() :
{}
{
// TODO : This step was created because of the ambiguite of the rule ClassCondition() inside of RelationCondition() :
// CobolWord() CobolWord() === ArithmeticExpression() ClassCondition()
LOOKAHEAD(ArithmeticExpression())
ArithmeticExpression()
| <LPARENCHAR> Condition() <RPARENCHAR>
}

void Condition() :
{}
{
CombinableCondition() ( ( <AND> | <OR> ) ( LOOKAHEAD(CombinableCondition()) CombinableCondition() | AbbreviationRest() ) )*
}

void CombinableCondition() :
{}
{
[ <NOT> ] SimpleCondition()
}

void SimpleCondition() :
{}
{
LOOKAHEAD(RelationCondition())
RelationCondition()
| LOOKAHEAD(ArithmeticExpression())
ArithmeticExpression()
| <LPARENCHAR> Condition() <RPARENCHAR>
}

void ClassCondition() :
{}
{
[ <IS> ] [ <NOT> ] ( <NUMERIC> | <ALPHABETIC> | <ALPHABETIC_LOWER> | <ALPHABETIC_UPPER> | ClassName() | <DBCS> | <KANJI> )
}

void RelationCondition() :
{}
{
ArithmeticExpression() ( LOOKAHEAD(AbbreviationRest())
AbbreviationRest()
| LOOKAHEAD(SignCondition())
SignCondition()
| ClassCondition()
)
}

void SignCondition() :
{}
{
[ <IS> ] [ <NOT> ] ( <POSITIVE> | <NEGATIVE> | <ZERO> | <ZEROS> | <ZEROES> )
}

void RelationalOperator() :
{}
{
[ <IS> ] [ <NOT> ] ( <GREATER> [ <THAN> ] [ <OR> <EQUAL> [ <TO> ] ]
| <MORETHANCHAR>
| <LESS> [ <THAN> ] [ <OR> <EQUAL> [ <TO> ] ]
| <LESSTHANCHAR>
| <EQUAL> [ <TO> ]
| <EQUALCHAR>
| <NOTEQUAL>
| <MORETHANOREQUAL>
| <LESSTHANOREQUAL>
)
}

void AbbreviationLeaf() :
{}
{
LOOKAHEAD(ArithmeticExpression())
ArithmeticExpression()
// TODO : Is there another abbreviation rule ?
| <LPARENCHAR> AbbreviationLeaf() ( <OR> AbbreviationLeaf() )+ <RPARENCHAR>
}

void AbbreviationRest() :
{}
{
RelationalOperator() AbbreviationLeaf()
}

////////////////////////////////////////////////////////////////////////////////
// VARIOUS TYPES OF IDENTIFIERS
////////////////////////////////////////////////////////////////////////////////

void ProcedureName() :
{}
{
( ParagraphName() [ ( <IN> | <OF> ) SectionName() ]
| SectionName() // Useless here ...
)
}

void Subscript() :
{}
{
<LPARENCHAR> ( ArithmeticExpression() )+ <RPARENCHAR>
}

void FunctionClause() :
{}
{
<FUNCTION> FunctionName() [ LOOKAHEAD(Subscript()) Subscript() ] [ <LPARENCHAR> LeftmostCharacterPosition() <COLONCHAR> [ Length() ] <RPARENCHAR> ]
}

void Identifier() :
{}
{
( QualifiedDataName() ( LOOKAHEAD(Subscript()) Subscript() )* [ <LPARENCHAR> LeftmostCharacterPosition() <COLONCHAR> [ Length() ] <RPARENCHAR> ]
| <LINAGE_COUNTER> [ ( <IN> | <OF> ) FileName() ]
| <PROGRAM_STATUS>
)
}

void QualifiedDataName() :
{}
{
( DataName() ( ( <IN> | <OF> ) DataName() )*
[ ( <IN> | <OF> ) FileName() ] // Useless part here ...
| SpecialRegister() // Useless here ...
)
}

void Length() :
{}
{
ArithmeticExpression()
}

void LeftmostCharacterPosition() :
{}
{
ArithmeticExpression()
}

void Mode() :
{}
{
CobolWord()
}

void AlphabetName() :
{}
{
CobolWord()
}

void ClassName() :
{}
{
CobolWord()
}

void ConditionName() :
{}
{
CobolWord()
}

void DataName() :
{}
{
CobolWord()
}

void FileName() :
{}
{
CobolWord()
}

void IndexName() :
{}
{
CobolWord()
}

void MnemonicName() :
{}
{
CobolWord()
}

void RecordName() :
{}
{
Identifier()
}

void RoutineName() :
{}
{
CobolWord()
}

void SymbolicCharacter() :
{}
{
CobolWord()
}

void LibraryName() :
{}
{
CobolWord()
}

void ProgramName() :
{}
{
CobolWord()
}

void SectionName() :
{}
{
// We force IntegerConstant() recognization here ...
IntegerConstant() | CobolWord()
}

void ParagraphName() :
{}
{
// We force IntegerConstant() recognization here ...
IntegerConstant() | CobolWord()
}

void SystemName() :
{}
{
CobolWord()
}

void ComputerName() :
{}
{
SystemName()
}

void LanguageName() :
{}
{
SystemName()
}

void EnvironmentName() :
{}
{
SystemName()
}

void AssignmentName() :
{}
{
SystemName()
}

void BasisName() :
{}
{
ProgramName()
}

void FunctionName() :
{}
{
( CobolWord()
| <LENGTH> // To avoid conflicts with token having same name ...
| <WHEN_COMPILED>
| <SUM>
| <RANDOM>
)
}

void SpecialRegister() :
{}
{
( <ADDRESS> <OF> DataName()
| <DEBUG_ITEM>
| <LENGTH> <OF> Identifier()
| <RETURN_CODE>
| <SHIFT_OUT>
| <SHIFT_IN>
| <SORT_CONTROL>
| <SORT_CORE_SIZE>
| <SORT_FILE_SIZE>
| <SORT_MESSAGE>
| <SORT_MODE_SIZE>
| <SORT_RETURN>
| <TALLY>
| <WHEN_COMPILED>
)
}

void CdName() :
{}
{
CobolWord()
}

void ConventionName() :
{}
{
CobolWord()
}

void HandleComponent() :
{}
{
CobolWord()
}

void ImplicitTitle() :
{}
{
QuotedText()
}

void LevelName66() :
{}
{
DataName()
}

void LevelName77() :
{}
{
DataName()
}

void LevelName88() :
{}
{
ConditionName()
}

void LevelName() :
{}
{
[ DataName() | <FILLER> | <REPLY> | <PROGRAM_STATUS> ]
}

////////////////////////////////////////////////////////////////////////////////
// ARITHMETIC
////////////////////////////////////////////////////////////////////////////////

void ArithmeticExpression() :
{}
{
TimesDiv() ( ( <PLUSCHAR> | <MINUSCHAR> ) TimesDiv() )*
}

void TimesDiv() :
{}
{
Power() ( ( <ASTERISKCHAR> | <SLASHCHAR> ) Power() )*
}

void Power() :
{}
{
[ ( <PLUSCHAR> | <MINUSCHAR> ) ] Basis() ( <POWEROF> Basis() )*
}

void Basis() :
{}
{
// IMPORTANT : From more complex to most specific :
// TODO : Dirty hack : is <ALL> a valid value ?
( FunctionClause() | Identifier() | Literal() | <ALL> | <LPARENCHAR> ArithmeticExpression() <RPARENCHAR> )
}

void CommentLine() :
{}
{
( ( NonDotChars() | Literal() )+ [ <DOT> ] )+
}

////////////////////////////////////////////////////////////////////////////////
// COMPILATION UNIT.
////////////////////////////////////////////////////////////////////////////////

void CompilationUnit() :
{}
{
ProgramUnit()
( NestedProgramUnit() EndProgramStatement() )*
[ EndProgramStatement() ( CompilationUnit() )* ]
<EOF>
}

void ProgramUnit() :
{}
{
IdentificationDivision()
[ EnvironmentDivision() ]
[ DataDivision() ]
[ ProcedureDivision() ]
}

void NestedProgramUnit() :
{}
{
NestedIdentificationDivision()
[ EnvironmentDivision() ]
[ DataDivision() ]
[ ProcedureDivision() ]
}

void EndProgramStatement() :
{}
{
<END> <PROGRAM> ProgramName() <DOT>
}

void CopyBookUnit() :
{}
{
LinkageSectionEntry()
<EOF>
}

////////////////////////////////////////////////////////////////////////////////
// IDENTIFICATION DIVISION.
////////////////////////////////////////////////////////////////////////////////

void IdentificationDivision() :
{}
{
( <IDENTIFICATION> | <ID> ) <DIVISION> <DOT>
ProgramIdParagraph()
( IdentificationDivisionParagraph() )*
}

void NestedIdentificationDivision() :
{}
{
( <IDENTIFICATION> | <ID> ) <DIVISION> <DOT>
NestedProgramIdParagraph()
( IdentificationDivisionParagraph() )*
}

void IdentificationDivisionParagraph() :
{}
{
AuthorParagraph()
| InstallationParagraph()
| DateWrittenParagraph()
| DateCompiledParagraph()
| SecurityParagraph()
}

void ProgramIdParagraph() :
{}
{
<PROGRAM_ID> <DOT> ProgramName() [ [ <IS> ] <INITIAL> [ <PROGRAM> ] ] <DOT>
}

void NestedProgramIdParagraph() :
{}
{
<PROGRAM_ID> <DOT> ProgramName()
[ [ <IS> ] InitialOrCommon() [ <PROGRAM> ] ] <DOT>
}

void InitialOrCommon() :
{}
{
( <INITIAL> [ <COMMON> ]
| <COMMON> [ <INITIAL> ]
)
}

void AuthorParagraph() :
{ /* System.out.println("Deprecated keyword : author - use a comment"); */ }
{
<AUTHOR> <DOT> [ CommentLine() ]
}

void InstallationParagraph() :
{ /* System.out.println("Deprecated keyword : installation - use a comment"); */ }
{
<INSTALLATION> <DOT> [ CommentLine() ]
}

void DateWrittenParagraph() :
{ /* System.out.println("Deprecated keyword : date-written - use a comment"); */ }
{
<DATE_WRITTEN> <DOT> [ CommentLine() ]
}

void DateCompiledParagraph() :
{ /* System.out.println("Deprecated keyword : date-compiled - use a comment"); */ }
{
<DATE_COMPILED> <DOT> [ CommentLine() ]
}

void SecurityParagraph() :
{ /* System.out.println("Deprecated keyword : security - use a comment"); */ }
{
<SECURITY> <DOT> [ CommentLine() ]
}

////////////////////////////////////////////////////////////////////////////////
// ENVIRONMENT DIVISION.
////////////////////////////////////////////////////////////////////////////////

void EnvironmentDivision() :
{}
{
<ENVIRONMENT> <DIVISION> <DOT>
( EnvironmentSection() )*
}

void EnvironmentSection() :
{}
{
ConfigurationSection()
| InputOutputSection()
}

//------------------------------------------------------------------------------
// CONFIGURATION SECTION
//------------------------------------------------------------------------------

void ConfigurationSection() :
{}
{
<CONFIGURATION> <SECTION> <DOT>
( ConfigurationSectionParagraph() )*
}

void ConfigurationSectionParagraph() :
{}
{
SourceComputerParagraph()
| ObjectComputerParagraph()
| SpecialNamesParagraph()
}

void SourceComputerParagraph() :
{}
{
<SOURCE_COMPUTER> <DOT> ComputerName() [ [ <WITH> ] <DEBUGGING> <MODE> ] <DOT>
}

void ObjectComputerParagraph() :
{}
{
<OBJECT_COMPUTER> <DOT>
ComputerName() ( ObjectComputerClause() )* <DOT>
}

void ObjectComputerClause() :
{}
{
MemorySizeClause()
| CollatingSequenceClause()
| SegmentLimitClause()
| CharacterSetClause()
}

void MemorySizeClause() :
{ /* System.out.println("Deprecated keyword : memory - use a comment"); */ }
{
<MEMORY> [ <SIZE> ] IntegerConstant() [ <WORDS> | <CHARACTERS> | <MODULES> ]
}

void CollatingSequenceClause() :
{ /* System.out.println("Deprecated keyword : collating sequence - use a comment"); */ }
{
[ <PROGRAM> ] [ <COLLATING> ] <SEQUENCE> [ <IS> ] AlphabetName()
}

void SegmentLimitClause() :
{ /* System.out.println("Deprecated keyword : segment unit - use a comment"); */ }
{
( <SEGMENT_LIMIT> | <SEGMENT> <LIMIT> ) [ <IS> ] IntegerConstant()
}

// Tandem ???
void CharacterSetClause() :
{}
{
<CHARACTER> <SET> [ CommentLine() ]
}

void SpecialNamesParagraph() :
{}
{
<SPECIAL_NAMES> <DOT>
[ ( SpecialNameClause() )+ <DOT> ]
}

void SpecialNameClause() :
{}
{
AlphabetClause()
| ClassClause()
| CurrencySignClause()
| DecimalPointClause()
| SymbolicCharactersClause()
| CallConventionClause()
| EnvironmentNameIsMnemonicNameClause()
}

void IterableAlphabetPhrase() :
{}
{
( <THROUGH> | <THRU> ) IterableLiteral()
| ( <ALSO> IterableLiteral() )+
}

void AlphabetClause() :
{}
{
<ALPHABET> AlphabetName() [ <IS> ]
( <STANDARD_1>
| <STANDARD_2>
| <NATIVE>
| CobolWord()
| ( IterableLiteral() [ IterableAlphabetPhrase() ] )+
)
}

void ClassClause() :
{}
{
// TODO (SUPPRESS) : Dirty hack : IterableLiteral() --> Identifier() | IterableLiteral()
<CLASS> ClassName() [ <IS> ] ( ( Identifier() | IterableLiteral() ) [ ( <THROUGH> | <THRU> ) ( Identifier() | IterableLiteral() ) ] )+
}

void CurrencySignClause() :
{}
{
<CURRENCY> [ <SIGN> ] [ <IS> ] Literal()
}

void DecimalPointClause() :
{}
{
<DECIMAL_POINT> [ <IS> ] <COMMA>
}

void SymbolicCharactersClause() :
{}
{
<SYMBOLIC> [ <CHARACTERS> ] ( ( SymbolicCharacter() )+ [ ( <ARE> | <IS> ) ] ( NumericConstant() | <HEXNUMBER> )+ )+ [ <IN> AlphabetName() ]
}

void CallConventionClause() :
{}
{
<CALL_CONVENTION> IntegerConstant() [ <IS> ] ConventionName()
}

void EnvironmentNameIsMnemonicNameClause() :
{}
{
EnvironmentName()
( [ <IS> ] MnemonicName() [ SpecialNamesParagraphStatusPhrase() ]
| SpecialNamesParagraphStatusPhrase()
)
}

void SpecialNamesParagraphStatusPhrase() :
{}
{
( <ON> [ <STATUS> ] [ <IS> ] IterableCondition() [ <OFF> [ <STATUS> ] [ <IS> ] IterableCondition() ]
| <OFF> [ <STATUS> ] [ <IS> ] IterableCondition() [ <ON> [ <STATUS> ] [ <IS> ] IterableCondition() ]
)
}

//------------------------------------------------------------------------------
// INPUT-OUTPUT SECTION
//------------------------------------------------------------------------------

void InputOutputSection() :
{}
{
<INPUT_OUTPUT> <SECTION> <DOT>
( InputOutputSectionParagraph() )*
}

void InputOutputSectionParagraph() :
{}
{
FileControlParagraph()
| IOControlParagraph()
}

void FileControlParagraph() :
{}
{
<FILE_CONTROL> ( LOOKAHEAD([ <DOT> ] FileControlEntry()) [ <DOT> ] FileControlEntry() )* <DOT>
}

void FileControlEntry() :
{}
{
SelectClause()
( FileControlClause() )*
}

void FileControlClause() :
{}
{
AssignClause()
| ReserveClause()
| OrganizationClause()
| PaddingCharacterClause()
| RecordDelimiterClause()
| AccessModeClause()
| RecordKeyClause()
| AlternateRecordKeyClause()
| FileStatusClause()
| PasswordClause()
}

void SelectClause() :
{}
{
<SELECT> [ <OPTIONAL> ] FileName()
}

void AssignClause() :
{}
{
<ASSIGN> [ <TO> ] ( AssignmentName() | Literal() ) [ <USING> ( AssignmentName() | Literal() ) ]
}

void ReserveClause() :
{}
{
<RESERVE> IntegerConstant() [ <AREA> | <AREAS> ]
}

void OrganizationClause() :
{}
{
[ <ORGANIZATION> [ <IS> ] ]
( SequentialOrganizationClause()
| LineSequentialOrganizationClause()
| RelativeOrganizationClause()
| IndexedOrganizationClause()
)
}

void SequentialOrganizationClause() :
{}
{
<SEQUENTIAL>
}

void LineSequentialOrganizationClause() :
{}
{
<LINE> <SEQUENTIAL>
}

void RelativeOrganizationClause() :
{}
{
<RELATIVE> [ Identifier() ]
}

void IndexedOrganizationClause() :
{}
{
<INDEXED>
}

void PaddingCharacterClause() :
{}
{
<PADDING> [ <CHARACTER> ] [ <IS> ] ( Identifier() | Literal() )
}

void RecordDelimiterClause() :
{}
{
<RECORD> <DELIMITER> [ <IS> ] ( <STANDARD_1> | <IMPLICIT> | AssignmentName() )
}

void AccessModeClause() :
{}
{
<ACCESS> [ <MODE> ] [ <IS> ]
( SequentialAccessMode()
| RandomAccessMode()
| DynamicAccessMode()
)
}

void SequentialAccessMode() :
{}
{
<SEQUENTIAL> [ RelativeKeyClause() ]
}

void RandomAccessMode() :
{}
{
<RANDOM> [ RelativeKeyClause() ]
}

void DynamicAccessMode() :
{}
{
<DYNAMIC> [ RelativeKeyClause() ]
}

void RelativeKeyClause() :
{}
{
<RELATIVE> [ <KEY> ] [ <IS> ] Identifier()
}

void RecordKeyClause() :
{}
{
<RECORD> [ <KEY> ] [ <IS> ] Identifier() [ PasswordClause() ] [ [ <WITH> ] <DUPLICATES> ]
}

void AlternateRecordKeyClause() :
{}
{
<ALTERNATE> <RECORD> [ <KEY> ] [ <IS> ] Identifier() [ PasswordClause() ] [ [ <WITH> ] <DUPLICATES> ]
}

void PasswordClause() :
{}
{
<PASSWORD> [ <IS> ] DataName()
}

void FileStatusClause() :
{}
{
[ <FILE> ] <STATUS> [ <IS> ] Identifier() [ Identifier() ]
}

void IOControlParagraph() :
{}
{
<I_O_CONTROL>
( LOOKAHEAD([ <DOT> ] IOControlClause())
[ <DOT> ] IOControlClause()
)*
<DOT>
}

void IOControlClause() :
{}
{
RerunClause()
| SameAreaClause()
| MultipleFileClause()
}

void RerunClause() :
{ /* System.out.println("Deprecated keyword : rerun"); */ }
{
<RERUN> [ <ON> ( AssignmentName() | FileName() ) ] <EVERY> ( LOOKAHEAD(Rerun2()) Rerun2() | IntegerConstant() [ <CLOCK_UNITS> ] )
}

void Rerun2() :
{}
{
IntegerConstant() <RECORDS>
| [ <END> ] [ <OF> ] ( <REEL> | <UNIT> ) <OF> FileName()
}

void SameAreaClause() :
{}
{
// TODO : Dirty hack : Are <SAME> <AREA> ... or <SAME> ... valid values ?
<SAME> [ <RECORD> | <SORT> | <SORT_MERGE> ] [ <AREA> ] [ <FOR> ] ( FileName() )+
//| <SAME> <AREA> [ <FOR> ] ( FileName() )+
}

void MultipleFileClause() :
{ /* System.out.println("Deprecated keyword : multiple file"); */ }
{
<MULTIPLE> <FILE> [ <TAPE> ] [ <CONTAINS> ] ( FileName() [ [ <POSITION> ] IntegerConstant() ] )+
}

////////////////////////////////////////////////////////////////////////////////
// DATA DIVISION.
////////////////////////////////////////////////////////////////////////////////

void DataDivision() :
{}
{
<DATA> <DIVISION> <DOT>
( DataDivisionSection() )*
}

void DataDivisionSection() :
{}
{
FileSection()
| WorkingStorageSection()
| LinkageSection()
| CommunicationSection()
}

void FileSection() :
{}
{
<FILE> <SECTION> <DOT>
( FileAndSortDescriptionEntry() ( DataDescriptionEntry() )* )*
}

void FileAndSortDescriptionEntry() :
{}
{
( <FD> | <SD> ) FileName()
(
LOOKAHEAD([ <DOT> ] FileAndSortDescriptionEntryClause())
[ <DOT> ] FileAndSortDescriptionEntryClause()
)* <DOT>
}

void FileAndSortDescriptionEntryClause() :
{}
{
ExternalClause()
| GlobalClause()
| BlockContainsClause()
| RecordContainsClause()
| LabelRecordsClause()
| ValueOfClause()
| DataRecordClause()
| LinageClause()
| CodeSetClause()
| ReportClause()
| RecordingModeClause()
}

void ExternalClause() :
{}
{
[ <IS> ] <EXTERNAL>
}

void GlobalClause() :
{}
{
[ <IS> ] <GLOBAL>
}

void BlockContainsClause() :
{}
{
<BLOCK> [ <CONTAINS> ] IntegerConstant() [ <TO> IntegerConstant() ] [ <RECORDS> | <CHARACTERS> ]
}

void RecordContainsClause() :
{}
{
<RECORD> [ <CONTAINS> ]
( IntegerConstant() [ <TO> IntegerConstant() ] [ <CHARACTERS> ]
| [ <IS> ] <VARYING> [ <IN> ] [ <SIZE> ]
[ [ <FROM> ] IntegerConstant() [ <TO> IntegerConstant() ] [ <CHARACTERS> ] ]
[ <DEPENDING> [ <ON> ] Identifier() ]
)
}

void LabelRecordsClause() :
{}
{
<LABEL> ( <RECORD> [ <IS> ]
| <RECORDS> [ <ARE> ]
) ( <OMITTED> | <STANDARD> | ( DataName() )+ )
}

void ValueOfClause() :
{ /* System.out.println("Deprecated clause : value of"); */ }
{
<VALUE> <OF> ( SystemName() <IS> ( Identifier() | Literal() ) )+
}

void DataRecordClause() :
{ /* System.out.println("Deprecated clause : data record"); */ }
{
<DATA> ( <RECORD> [ <IS> ]
| <RECORDS> [ <ARE> ]
) ( DataName() )+
}

void LinageClause() :
{}
{
<LINAGE> [ <IS> ] ( DataName() | IntegerConstant() ) [ <LINES> ]
( [ <WITH> ] <FOOTING> [ <AT> ] ( DataName() | IntegerConstant() )
| [ <LINES> ] [ <AT> ] ( <TOP> | <BOTTOM> ) ( DataName() | IntegerConstant() )
)*
}

void RecordingModeClause() :
{}
{
<RECORDING> [ <MODE> ] [ <IS> ] Mode()
}

void CodeSetClause() :
{}
{
<CODE_SET> [ <IS> ] AlphabetName()
}

void ReportClause() :
{}
{
( <REPORT> [ <IS> ] | <REPORTS> [ <ARE> ] ) ( Identifier() )+
}

void DataDescriptionEntry() :
{}
{
( <LEVEL_66> LevelName66() RenamesClause() <DOT>
| <LEVEL_77> LevelName77() ( LOOKAHEAD(DataDescriptionEntryClause()) DataDescriptionEntryClause() )* <DOT>
| <LEVEL_88> LevelName88() ConditionValueClause() <DOT>
| LevelNumber() LevelName() ( LOOKAHEAD(DataDescriptionEntryClause()) DataDescriptionEntryClause() )* <DOT>
)
}

void DataDescriptionEntryClause() :
{}
{
DataPictureClause()
| DataValueClause()
| DataUsageClause()
| DataRedefinesClause()
| DataExternalClause()
| DataGlobalClause()
| DataIdentifiedClause()
| DataSignClause()
| DataOccursClause()
| DataSynchronizedClause()
| DataJustifiedClause()
| DataBlankWhenZeroClause()
}

void DataRedefinesClause() :
{}
{
<REDEFINES> DataName() [ DataPictureClause() ]
}

void DataBlankWhenZeroClause() :
{}
{
<BLANK> [ <WHEN> ] ( <ZERO> | <ZEROS> | <ZEROES> )
}

void DataJustifiedClause() :
{}
{
( <JUSTIFIED> | <JUST> ) [ <RIGHT> ]
}

void DataOccursClause() :
{}
{
<OCCURS> IntegerConstant() [ <TO> IntegerConstant() ] [ <TIMES> ]
[ <DEPENDING> [ <ON> ] Identifier() ]
( ( <ASCENDING> | <DESCENDING> ) [ <KEY> ] [ <IS> ] ( Identifier() )+ )*
[ <INDEXED> [ <BY> ] ( IndexName() )+ ]
}

void DataPictureClause() :
{}
{
( <PICTURE> | <PIC> ) [ <IS> ] PictureString()
}

void PictureString() :
{}
{
PictureOccurence() ( LOOKAHEAD(<DOTCHAR> PictureOccurence()) <DOTCHAR> PictureOccurence() | PictureOccurence() )*
// There can be additional dots at end of picture strings ...
( <DOTCHAR> )*
}

void PictureOccurence() :
{}
{
( NonDotChars() )+ [ <LPARENCHAR> IntegerConstant() <RPARENCHAR> ]
| <DOTCHAR> ( <LPARENCHAR> IntegerConstant() <RPARENCHAR> | NonDotChars() )
}

void PicturePunctuation() :
{}
{
<SLASHCHAR> | <COMMACHAR> | <COLONCHAR> | <ASTERISKCHAR> | <MINUSCHAR> | <PLUSCHAR> |
<POWEROF> | <LESSTHANOREQUAL> | <LESSTHANCHAR> | <MORETHANOREQUAL> | <MORETHANCHAR> |
<EQUALCHAR> | <NOTEQUAL>
}

void PictureCurrency() :
{}
{
<DOLLARCHAR> // to be completed
}

void NonDotChars() :
{}
{
IntegerConstant() | CobolWord() | PicturePunctuation() | PictureCurrency()
}

void DataExternalClause() :
{}
{
[ <IS> ] <EXTERNAL> [ <BY> Literal() ]
}

void DataGlobalClause() :
{}
{
[ <IS> ] <GLOBAL>
}

void DataIdentifiedClause() :
{}
{
[ <IS> ] <IDENTIFIED> <BY> Literal()
}

void DataUsageClause() :
{}
{
[ <USAGE> [ <IS> ] ] ( MemoryUsage() | ExternalUsage() | PropertyObjectModifier() )
}

void MemoryUsage() :
{}
{
<DISPLAY> | <DISPLAY_1>
| <COMP> | <COMPUTATIONAL>
| <COMP_N> | <COMPUTATIONAL_N>
| <COMP_1> | <COMPUTATIONAL_1>
| <COMP_1_REV> | <COMPUTATIONAL_1_REV> | <COMP_1_M> | <COMPUTATIONAL_1_M>
| <COMP_1_MVS> | <COMPUTATIONAL_1_MVS> | <COMP_1_E> | <COMPUTATIONAL_1_E>
| <COMP_2> | <COMPUTATIONAL_2>
| <COMP_2_REV> | <COMPUTATIONAL_2_REV> | <COMP_2_M> | <COMPUTATIONAL_2_M>
| <COMP_2_MVS> | <COMPUTATIONAL_2_MVS> | <COMP_2_E> | <COMPUTATIONAL_2_E>
| <COMP_3> | <COMPUTATIONAL_3>
| <COMP_4> | <COMPUTATIONAL_4>
| <COMP_5> | <COMPUTATIONAL_5>
| <COMP_6> | <COMPUTATIONAL_6>
| <COMP_S> | <COMPUTATIONAL_S> | <COMP_1_A> | <COMPUTATIONAL_1_A>
| <COMP_D> | <COMPUTATIONAL_D> | <COMP_2_A> | <COMPUTATIONAL_2_A>
| <PACKED_DECIMAL> | <COMP_P> | <COMPUTATIONAL_P>
| <PACKED_DECIMAL_A> | <COMP_3_A> | <COMPUTATIONAL_3_A>
| <PACKED_DECIMAL_I> | <PACKED_DECIMAL_E> | <PACKED_DECIMAL_H> | <COMP_3_M> | <COMPUTATIONAL_3_M> | <PACKED_DECIMAL_M>
| <BINARY> | <BINARY_M> | <COMP_M> | <COMPUTATIONAL_M> | <COMP_A> | <COMPUTATIONAL_A> | <COMP_4_M> | <COMPUTATIONAL_4_M> | <COMP_B> | <COMPUTATIONAL_B>
| <BINARY_REV>
| <SQLIND>
| <COMP_X> | <COMP_X> | <COMPUTATIONAL_X> | <COMPUTATIONAL_X> | <COMP_5_M> | <COMPUTATIONAL_5_M>
| <COMP_X_REV> | <COMPUTATIONAL_X_REV>
| <DISPLAY_WS>
| <EXTERNAL_FORM>
| <NATIONAL>
| <KANJI>
| <SIGNED_SHORT>
| <UNSIGNED_SHORT>
| <SIGNED_INT>
| <UNSIGNED_INT>
| <SIGNED_LONG>
| <UNSIGNED_LONG>
| <FLOAT>
| <DOUBLE>
| <HANDLE> <OF> <THREAD> // <OF> is not optional here
| <HANDLE> [ [ <OF> ] HandleComponent() [ ImplicitTitle() ] ]
| <HANDLE> [ <OF> ] <FONT> [ <FIXED_FONT> | <TRADITIONAL_FONT> | <DEFAULT_FONT> | <SMALL_FONT> | <MEDIUM_FONT> | <LARGE_FONT> ]
| <HANDLE> [ <OF> ] <WINDOW> [ ImplicitTitle() ]
| <WINDOW>
}

void ExternalUsage() :
{}
{
<INDEX>
| <POINTER>
| <PROCEDURE_POINTER> | <PROCEDURE> <POINTER>
| <STRING>
| <JPACKED_DECIMAL>
| <JSTRING>
| <JBYTE>
| <JSHORT>
| <JINT>
| <JLONG>
| <JBOOLEAN>
| <JFLOAT>
| <JDOUBLE>
| <JCHAR>
| <ALPHANUMERIC_HASHTABLE>
| <NUMERIC_HASHTABLE>
| <OBJECT_HASHTABLE>
| <OBJECT> <REFERENCE> [ [ <OF> ] ( ClassName() | Literal() ) ]
| ( <OBJECT> | <COMPONENT> ) [ [ <OF> ] ClassName() ]
}

void PropertyObjectModifier() :
{}
{
<CHARACTER>
| <BINARY_BYTE>
| <BINARY_SHORT>
| <BINARY>
| <BINARY_LONG>
| <BINARY_DOUBLE>
| <FLOAT_SHORT>
| <FLOAT_LONG>
| <BIT>
}

void DataSignClause() :
{}
{
[ <SIGN> [ <IS> ] ] ( <LEADING> | <TRAILING> ) [ <SEPARATE> [ <CHARACTER> ] ]
}

void DataSynchronizedClause() :
{}
{
( <SYNCHRONIZED> | <SYNC> ) [ ( <LEFT> | <RIGHT> ) ]
}

void DataValueClause() :
{}
{
// TODO (SUPPRESS) : Dirty hack : Literal() --> PictureString() | IterableLiteral()
( <VALUE> [ <IS> ] | <VALUES> [ <ARE> ] ) ( ( LOOKAHEAD(PictureString()) PictureString() | IterableLiteral() ) [ ( <THROUGH> | <THRU> ) IterableLiteral() ] )+
}

void ConditionValueClause() :
{}
{
DataValueClause()
}

void RenamesClause() :
{}
{
<RENAMES> Identifier() [ ( <THROUGH> | <THRU> ) Identifier() ]
}

////////////////////////////////////////////////////////////////////////////////
// DATA DIVISION : COMMUNICATION SECTION.
////////////////////////////////////////////////////////////////////////////////

void CommunicationDescriptionEntry() :
{}
{
(
LOOKAHEAD(CommunicationInputEntry())
CommunicationInputEntry()
| LOOKAHEAD(CommunicationOutputEntry())
CommunicationOutputEntry()
| LOOKAHEAD(CommunicationIOEntry())
CommunicationIOEntry()
) <DOT>
}

void CommunicationInputEntry() :
{}
{
<CD> CdName() [ <FOR> ] [ <INITIAL> ] <INPUT> ( CommunicationInputClause() )* ( DataName() )*
}

void CommunicationOutputEntry() :
{}
{
<CD> CdName() [ <FOR> ] <OUTPUT> ( CommunicationOutputClause() )*
}

void CommunicationIOEntry() :
{}
{
<CD> CdName() [ <FOR> ] [ <INITIAL> ] <I_O> ( CommunicationIOClause() )* ( DataName() )*
}

void CommunicationInputClause() :
{}
{
<MESSAGE> ( <DATE> | <TIME> | <COUNT> ) [ <IS> ] DataName()
| <TEXT> <LENGTH> [ <IS> ] DataName()
| <END> <KEY> [ <IS> ] DataName()
| <STATUS> <KEY> [ <IS> ] DataName()
| <COUNT> [ <IS> ] DataName()
| [ <SYMBOLIC> ] ( <QUEUE> | <SUB_QUEUE_1> | <SUB_QUEUE_2> | <SUB_QUEUE_3> | <SOURCE> ) [ <IS> ] DataName()
}

void CommunicationOutputClause() :
{}
{
<DESTINATION> <COUNT> [ <IS> ] DataName()
| <TEXT> <LENGTH> [ <IS> ] DataName()
| <STATUS> <KEY> [ <IS> ] DataName()
| LOOKAHEAD([ <SYMBOLIC> ] <DESTINATION> [ <IS> ] DataName())
[ <SYMBOLIC> ] <DESTINATION> [ <IS> ] DataName()
| <DESTINATION> <TABLE> <OCCURS> IntegerConstant() [ <TIMES> ] [ <INDEXED> [ <BY> ] ( IndexName() )+ ]
| <ERROR> <KEY> [ <IS> ] DataName()
}

void CommunicationIOClause() :
{}
{
<MESSAGE> ( <DATE> | <TIME> ) [ <IS> ] DataName()
| <TEXT> <LENGTH> [ <IS> ] DataName()
| <END> <KEY> [ <IS> ] DataName()
| <STATUS> <KEY> [ <IS> ] DataName()
| [ <SYMBOLIC> ] <TERMINAL> [ <IS> ] DataName()
}

//------------------------------------------------------------------------------
// WORKING STORAGE SECTION, LINKAGE SECTION.
//------------------------------------------------------------------------------

void WorkingStorageSection() :
{}
{
<WORKING_STORAGE> <SECTION> <DOT>
( DataDescriptionEntry() )*
}

void LinkageSectionEntry() :
{}
{
( DataDescriptionEntry() )*
}

void LinkageSection() :
{}
{
<LINKAGE> <SECTION> <DOT>
LinkageSectionEntry()
}

void CommunicationSection() :
{}
{
<COMMUNICATION> <SECTION> <DOT>
( CommunicationDescriptionEntry() ( DataDescriptionEntry() )* )*
}

////////////////////////////////////////////////////////////////////////////////
// PROCEDURE DIVISION also known as "spaghetti division".
////////////////////////////////////////////////////////////////////////////////

void ProcedureDivision() :
{}
{
<PROCEDURE> <DIVISION> [ ConventionName() ] [ <USING> ( DataName() )+ ] <DOT>
[ Declaratives() ]
ProcedureBody()
}

void Declaratives() :
{}
{
<DECLARATIVES> <DOT>
( SectionHeader() <DOT>
UseStatement() <DOT>
Paragraphs()
)+
<END> <DECLARATIVES> <DOT>
}

void ProcedureBody() :
{}
{
Paragraphs() ( ProcedureSection() )*
}

void ProcedureSection() :
{}
{
SectionHeader() <DOT> Paragraphs()
}

void SectionHeader() :
{}
{
SectionName() <SECTION> [ IntegerConstant() ]
}

void Paragraphs() :
{}
{
( Sentence() )* ( Paragraph() )*
}

void Paragraph() :
{}
{
ParagraphName() <DOT>
( ExitStatement()
| AlteredGoto()
| ( Sentence() )*
)
}

void Sentence() :
{}
{
StatementList() <DOT>
}

void StatementList() :
{}
{
( Statement() )+
}

void Statement() :
{}
{
( AcceptStatement()
| AddStatement()
| AlterStatement()
| CallStatement()
| CancelStatement()
| CloseStatement()
| ComputeStatement()
| ContinueStatement()
| DeleteStatement()
| DisplayStatement()
| DivideStatement()
| EntryStatement()
| EvaluateStatement()
/* | ExitStatement() */
| ExitProgramStatement()
| GobackStatement()
| GotoStatement()
| IfStatement()
| InitializeStatement()
| InspectStatement()
| MergeStatement()
| MoveStatement()
| MultiplyStatement()
| OpenStatement()
| PerformStatement()
| ReadStatement()
| ReleaseStatement()
| ReturnStatement()
| RewriteStatement()
| SearchStatement()
| SetStatement()
| SortStatement()
| StartStatement()
| StopStatement()
| StringStatement()
| SubtractStatement()
| UnstringStatement()
| WriteStatement() )
}

void AcceptStatement() :
{}
{
<ACCEPT> ( <REPLY> | <OMITTED> | Identifier() )
[ <FROM>
( MnemonicName()
| EnvironmentName()
| <DATE>
| <DAY>
| <DAY_OF_WEEK>
| <TIME>
)
]
}

void AddStatement() :
{}
{
<ADD>
( ( <CORRESPONDING> | <CORR> ) Identifier() <TO> Identifier() [ <ROUNDED> ]
| ( Identifier() | Literal() )+ [ <TO> ( Identifier() [ <ROUNDED> ] )+ ]
[ <GIVING> ( Identifier() [ <ROUNDED> ] )+ ]
)
[ [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <NOT> [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <END_ADD> ]
}

void AlteredGoto() :
{ /* System.out.println("***ALTER ALERT***"); */ }
{
<GO> [ <TO> ] <DOT>
}

/**
* ALTER is one of the most dangerous statement ever invented by man.
* ALTER is to GOTO what an atomic bomb is to firecrackers.
*/

void AlterStatement() :
{ /* System.out.println("***ALTER ALERT***"); */ }
{
<ALTER> ( ProcedureName() <TO> [ <PROCEED> <TO> ] ProcedureName() )+
}

void CallByContent() :
{}
{
[ <BY> ] <CONTENT> ( <ADDRESS> <OF> Identifier() | [ <LENGTH> <OF> ] Identifier() | Literal() | <OMITTED> )+
}

void CallByReference() :
{}
{
[ [ <BY> ] <REFERENCE> ] ( <ADDRESS> <OF> Identifier() | Identifier() | FileName() | <OMITTED> )+
}

void CallByDescriptor() :
{}
{
[ <BY> ] <DESCRIPTOR> ( <ADDRESS> <OF> Identifier() | Identifier() | FileName() | <OMITTED> )+
}

void CallByValue() :
{}
{
[ <BY> ] <VALUE> ( <ADDRESS> <OF> Identifier() | [ <LENGTH> <OF> ] Identifier() | Literal() | <OMITTED> )+
}

void CallStatement() :
{}
{
<CALL>
(
LOOKAHEAD(ConventionName() ( Identifier() | Literal() ))
ConventionName() ( Identifier() | Literal() )
| ( Identifier() | Literal() )
)
[ [ <WITH> ] <STDCALL> [ <LINKAGE> ] ] // Fijutsu Cobol
[ <USING>
(
LOOKAHEAD(CallByContent())
CallByContent()
| LOOKAHEAD(CallByDescriptor())
CallByDescriptor()
| LOOKAHEAD(CallByValue())
CallByValue()
| LOOKAHEAD(CallByReference())
CallByReference()
)+
]
[ ( <GIVING> | <RETURNING> ) Identifier() ]
[ LOOKAHEAD([ <ON> ] <EXCEPTION>) [ <ON> ] <EXCEPTION> StatementList() ]
[ LOOKAHEAD(<NOT> [ <ON> ] <EXCEPTION>) <NOT> [ <ON> ] <EXCEPTION> StatementList() ]
[ [ <ON> ] <OVERFLOW> StatementList() ]
[ <NOT> [ <ON> ] <OVERFLOW> StatementList() ]
[ <END_CALL> ]
}

void CancelStatement() :
{}
{
<CANCEL> ( Identifier() | Literal() )+
}

void CloseStatement() :
{}
{
<CLOSE>
( FileName()
[ ( ( <REEL> | <UNIT> ) [ ( [ <FOR> ] <REMOVAL>
| [ <WITH> ] <NO> <REWIND>
)
]
| [ <WITH> ] ( <NO> <REWIND> | <LOCK> )
)
]
)+
}

void ComputeStatement() :
{}
{
<COMPUTE> ( Identifier() [ <ROUNDED> ] )+
( <EQUALCHAR> | <EQUAL> )
ArithmeticExpression()
[ [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <NOT> [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <END_COMPUTE> ]
}

void ContinueStatement() :
{}
{
<CONTINUE>
}

void DeleteStatement() :
{}
{
<DELETE> FileName() [ <RECORD> ]
[ <INVALID> [ <KEY> ] StatementList() ]
[ <NOT> <INVALID> [ <KEY> ] StatementList() ]
[ <END_DELETE> ]
}

void DisplayStatement() :
{}
{
<DISPLAY> ( Identifier() | Literal() )+
[ <UPON> ( MnemonicName() | EnvironmentName() ) ]
[ [ <WITH> ] <NO> <ADVANCING> ]
}

void DivideStatement() :
{}
{
<DIVIDE> ( Identifier() | Literal() )
( <INTO> ( Identifier() | Literal() ) [ <ROUNDED> ] ( <GIVING> ( Identifier() [ <ROUNDED> ] )+ | ( ( Identifier() | Literal() ) [ <ROUNDED> ] )* )
| <BY> ( Identifier() | Literal() ) [ <GIVING> ( Identifier() [ <ROUNDED> ] )+ ]
)
[ <REMAINDER> Identifier() ]
[ [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <NOT> [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <END_DIVIDE> ]
}

void EntryStatement() :
{}
{
<ENTRY> Literal() [ <USING> ( Identifier() )+ ]
}

void EvaluateStatement() :
{}
{
<EVALUATE> EvaluateValue()
( <ALSO> EvaluateValue() )*
( ( EvaluateWhenClause() )+ [ <THEN> ] StatementList() )+
[ <END_EVALUATE> ]

}

void EvaluateValue() :
{}
{
( Condition()
| <TRUE> // Useless here ...
| <FALSE> // Useless here ...
)
}

void EvaluateWhenClause() :
{}
{
( LOOKAHEAD(<WHEN> <OTHER>)
<WHEN> <OTHER>
| <WHEN> EvaluatePhrase() ( <ALSO> EvaluatePhrase() )*
)
}

void EvaluateThruPhrase() :
{}
{
[ <NOT> ] ArithmeticExpression() ( <THROUGH> | <THRU> ) ArithmeticExpression()
}

void EvaluatePhrase() :
{}
{
( LOOKAHEAD(EvaluateThruPhrase())
EvaluateThruPhrase()
| Condition()
| <ANY>
| <TRUE> // Useless here ...
| <FALSE> // Useless here ...
)
}

void ExitStatement() :
{}
{
<EXIT> <DOT>
}

void ExitProgramStatement() :
{}
{
<EXIT> <PROGRAM>
}

void GobackStatement() :
{}
{
<GOBACK>
}

void GotoStatement() :
{}
{
<GO> [ <TO> ]
( ( ProcedureName() )+ [ <DEPENDING> [ <ON> ] Identifier() ]
| <MORE_LABELS> // ??? IBM extension ???
)
}

void IfStatement() :
{}
{
<IF> Condition() [ <THEN> ] ( ( Statement() )+ | <NEXT> <SENTENCE> )
[ <ELSE> ( ( Statement() )+ | <NEXT> <SENTENCE> ) ]
[ <END_IF> ]

}

void InitializeStatement() :
{}
{
<INITIALIZE> ( Identifier() )+
[ <REPLACING> ( ( <ALPHABETIC>
| <ALPHANUMERIC>
| <NUMERIC>
| <ALPHANUMERIC_EDITED>
| <NUMERIC_EDITED>
| <DBCS>
| <EGCS>
) [ <DATA> ] <BY> ( Identifier() | Literal() )
)+
]

}

void InspectStatement() :
{}
{
<INSPECT> Identifier()
( TallyingPhrase()
| ConvertingPhrase()
| ReplacingPhrase()
)
}

void IterableTallyingClause() :
{}
{
<CHARACTERS> ( IterableBeforeAfterPhrase() )*
}

void RecursiveTallyingClause() :
{}
{
( <ALL> | <LEADING> ) IterableBeforeAfterClause()
(
LOOKAHEAD(Identifier() <FOR>)
( Identifier() <FOR> ( RecursiveTallyingClause() | IterableTallyingClause() )+ )+
| IterableBeforeAfterClause()
)*
}

void TallyingPhrase() :
{}
{
<TALLYING>
( Identifier() <FOR> ( RecursiveTallyingClause() | IterableTallyingClause() )+ )+
[ ReplacingPhrase() ]
}

void ConvertingPhrase() :
{}
{
<CONVERTING> ( Identifier() | Literal() )
<TO> IterableBeforeAfterClause()
}

void ReplacingPhrase() :
{}
{
<REPLACING>
( <CHARACTERS> <BY> IterableBeforeAfterClause()
| ( <ALL> | <LEADING> | <FIRST> ) ( LOOKAHEAD(( Identifier() | Literal() ) <BY> IterableBeforeAfterClause())
( Identifier() | Literal() ) <BY> IterableBeforeAfterClause()
)+
)+
}

void IterableBeforeAfterClause() :
{}
{
( Identifier() | IterableLiteral() ) ( IterableBeforeAfterPhrase() )*
}

void IterableBeforeAfterPhrase() :
{}
{
( <BEFORE> | <AFTER> ) [ <INITIAL> ] ( Identifier() | IterableLiteral() )
}

void MergeStatement() :
{}
{
<MERGE> FileName()
( [ <ON> ] ( <ASCENDING> | <DESCENDING> ) [ <KEY> ] ( Identifier() )+ )+
[ [ <COLLATING> ] <SEQUENCE> [ <IS> ] AlphabetName() ] <USING> FileName() ( FileName() )+
( <OUTPUT> <PROCEDURE> [ <IS> ] ProcedureName() [ ( <THROUGH> | <THRU> ) ProcedureName() ]
| <GIVING> ( FileName() )+
)
}

/**
* MOVE is probably the first polymorphic statement in the history of computing.
*/

void MoveStatement() :
{}
{
// <MOVE> FunctionClause() : NON-CONFORMING STANDARD
<MOVE>
( ArithmeticExpression() <TO> ( Identifier() )+
| ( <CORRESPONDING> | <CORR> ) Identifier() <TO> ( Identifier() )+
)
}

void MultiplyStatement() :
{}
{
<MULTIPLY> ( Identifier() | Literal() ) <BY> ( ( Identifier() | Literal() ) [ <ROUNDED> ] )+
[ <GIVING> ( Identifier() [ <ROUNDED> ] )+ ]
[ [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <NOT> [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <END_MULTIPLY> ]
}

void OpenStatement() :
{}
{
<OPEN> ( <INPUT> ( FileName() [ ( <REVERSED> | [ <WITH> ] <NO> <REWIND> ) ] )+
| <OUTPUT> ( FileName() [ [ <WITH> ] <NO> <REWIND> ] )+
| <I_O> ( FileName() )+
| <EXTEND> ( FileName() )+
)+
}

/**
* The "one-size-fits-all" statement of COBOL: it replaces for, while, call...
*/

void PerformStatement() :
{}
{
// TODO : Rule "PERFORM <Identifier Range> ***<With Test>*** <Numeric> TIMES" doesn't exist.
<PERFORM>
( LOOKAHEAD(PerformFlavour() StatementList() <END_PERFORM>)
PerformFlavour() StatementList() <END_PERFORM>
| LOOKAHEAD(PerformProcedureScopeClause() PerformFlavour())
PerformProcedureScopeClause() PerformFlavour()
| LOOKAHEAD(StatementList() <END_PERFORM>)
StatementList() <END_PERFORM>
| PerformProcedureScopeClause()
)
}

void PerformFlavour() :
{}
{
PerformTimeClause()
| [ PerformTestPositionClause() ] ( PerformUntilClause() | PerformVaryingWithAfterClause() )
}

void PerformVaryingWithAfterClause() :
{}
{
<VARYING> PerformVaryingPhrase() ( <AFTER> PerformVaryingPhrase() )*
}

void PerformVaryingPhrase() :
{}
{
Identifier()
<FROM> ( Identifier() | Literal() )
<BY> ( Identifier() | Literal() )
PerformUntilClause()
}

void PerformUntilClause() :
{}
{
<UNTIL> Condition()
}

void PerformTestPositionClause() :
{}
{
[ <WITH> ] <TEST> ( <BEFORE> | <AFTER> )
}

void PerformProcedureScopeClause() :
{}
{
ProcedureName() [ ( <THROUGH> | <THRU> ) ProcedureName() ]
}

void PerformTimeClause() :
{}
{
( Identifier() | IntegerConstant() ) <TIMES>
}

void ReadStatement() :
{}
{
<READ> FileName() [ <NEXT> ] [ <RECORD> ]
[ <INTO> Identifier() ]
[ <KEY> [ <IS> ] Identifier() ]
[ <INVALID> [ <KEY> ] StatementList() ]
[ <NOT> <INVALID> [ <KEY> ] StatementList() ]
[ [ <AT> ] <END> StatementList() ]
[ <NOT> [ <AT> ] <END> StatementList() ]
[ <END_READ> ]
}

void ReleaseStatement() :
{}
{
<RELEASE> RecordName() [ <FROM> Identifier() ]
}

void ReturnStatement() :
{}
{
<RETURN> FileName() [ <RECORD> ] [ <INTO> Identifier() ]
[ <AT> ] <END> StatementList()
[ <NOT> [ <AT> ] <END> StatementList() ]
[ <END_RETURN> ]
}

void RewriteStatement() :
{}
{
<REWRITE> RecordName() [ <FROM> Identifier() ]
[ <INVALID> [ <KEY> ] StatementList() ]
[ <NOT> <INVALID> [ <KEY> ] StatementList() ]
[ <END_REWRITE> ]
}

void SearchStatement() :
{}
{
<SEARCH> [ <ALL> ] Identifier()
[ <VARYING> Identifier() ]
[ [ <AT> ] <END> StatementList() ]
( <WHEN> Condition() ( StatementList() | <NEXT> <SENTENCE> ) )+
[ <END_SEARCH> ]
}

void SetStatement() :
{}
{
<SET> [ <CONFIGURATION> ] ( Identifier() | Literal() )+
( <TO> [ <CONFIGURATION> ] ( Identifier() | <TRUE> | <FALSE> | <ON> | <OFF> | [ <ENTRY> ] Literal() )
| ( <UP> | <DOWN> ) [ <BY> ] ( Identifier() | Literal() )
)
}

void SortStatement() :
{}
{
<SORT> FileName()
( [ <ON> ] ( <ASCENDING> | <DESCENDING> ) [ <KEY> ] ( Identifier() )+ )+
[ [ <WITH> ] <DUPLICATES> [ <IN> ] [ <ORDER> ] ]
[ [ <COLLATING> ] <SEQUENCE> [ <IS> ] AlphabetName() ]
( <USING> ( FileName() )+ | <INPUT> <PROCEDURE> [ <IS> ] ProcedureName() [ ( <THROUGH> | <THRU> ) ProcedureName() ] )
( <GIVING> ( FileName() )+ | <OUTPUT> <PROCEDURE> [ <IS> ] ProcedureName() [ ( <THROUGH> | <THRU> ) ProcedureName() ] )
}

void StartStatement() :
{}
{
<START> FileName()
[ <KEY> RelationalOperator() Identifier() ]
[ <INVALID> [ <KEY> ] StatementList() ]
[ <NOT> <INVALID> [ <KEY> ] StatementList() ]
[ <END_START> ]
}

void StopStatement() :
{}
{
<STOP> ( <RUN> | Literal() )
}

void StringStatement() :
{}
{
<STRING>
( ( Identifier() | IterableLiteral() )+ <DELIMITED> [ <BY> ] ( Identifier() | IterableLiteral() | <SIZE> ) )+
<INTO> Identifier()
[ [ <WITH> ] <POINTER> Identifier() ]
[ [ <ON> ] <OVERFLOW> StatementList() ]
[ <NOT> [ <ON> ] <OVERFLOW> StatementList() ]
[ <END_STRING> ]
}

void SubtractStatement() :
{}
{
<SUBTRACT>
( ( Identifier() | Literal() )+ <FROM> ( ( Identifier() | Literal() ) [ <ROUNDED> ] )+ [ <GIVING> ( Identifier() [ <ROUNDED> ] )+ ]
| ( <CORRESPONDING> | <CORR> ) Identifier() <FROM> Identifier() [ <ROUNDED> ]
)
[ [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <NOT> [ <ON> ] <SIZE> <ERROR> StatementList() ]
[ <END_SUBTRACT> ]
}

void UnstringStatement() :
{}
{
<UNSTRING> Identifier()
[ <DELIMITED> [ <BY> ] [ <ALL> ] ( Identifier() | Literal() ) ( <OR> [ <ALL> ] ( Identifier() | Literal() ) )* ]
<INTO>
( Identifier() [ <DELIMITER> [ <IN> ] Identifier() ] [ <COUNT> [ <IN> ] Identifier() ] )+
[ [ <WITH> ] <POINTER> Identifier() ] [ <TALLYING> [ <IN> ] Identifier() ]
[ [ <ON> ] <OVERFLOW> StatementList() ]
[ <NOT> [ <ON> ] <OVERFLOW> StatementList() ]
[ <END_UNSTRING> ]
}

void UseStatement() :
{}
{
<USE> ( [ <FOR> ] <DEBUGGING> [ <ON> ] ( ( ProcedureName() )+ | <ALL> ( <PROCEDURES> | <REFERENCES> <OF> DataName() ) )
| [ <GLOBAL> ] <AFTER> [ <STANDARD> ]
( ( <EXCEPTION> | <ERROR> )
| [ ( <BEGINNING> | <ENDING> ) ] [ ( <FILE> | <REEL> | <UNIT> ) ] <LABEL>
)
<PROCEDURE> [ <ON> ] ( ( FileName() )+ | <INPUT> | <OUTPUT> | <I_O> | <EXTEND> )
)
}

void WriteStatement() :
{}
{
<WRITE> RecordName() [ <FROM> Identifier() ]
[ AdvancingPhrase() ]
[ [ <AT> ] ( <END_OF_PAGE> | <EOP> ) StatementList() ]
[ <NOT> [ <AT> ] ( <END_OF_PAGE> | <EOP> ) StatementList() ]
[ <INVALID> [ <KEY> ] StatementList() ]
[ <NOT> <INVALID> [ <KEY> ] StatementList() ]
[ <END_WRITE> ]
}

void AdvancingPhrase() :
{}
{
( <BEFORE> | <AFTER> ) [ <ADVANCING> ]
( <PAGE>
| ( IntegerConstant() | Identifier() | Literal() ) [ ( <LINE> | <LINES> ) ]
| MnemonicName() // Useless here ...
)
}

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Generating Parsers with JavaCC, Second Edition

2009-10-27T09:27:29Z

This is a plug... stop reading now if you don't like those!

That said, we're now shipping the second edition of my JavaCC book,
Generating Parsers with JavaCC!

Creating this second edition was a labor of love. I went through every
page of the first edition and found all the places where JavaCC had
moved out from under the book. I've brought those bits up to date,
added new sections all over, fixed numerous typos, added more index
entries, and generally given it a thorough house cleaning. This second
edition also benefits from the intervening two years during which I
worked on the JavaCC code and answered questions from folks on the
javacc-users mailing list. And I've generally had a little more time
to digest some of the information in the first edition, so I think the
second edition is improved because of that.

So, what does the second edition include? Here's a sampling:

Discussion of new JavaCC features like TRACK_TOKENS, TOKEN_EXTENDS,
SUPPORT_CLASS_VISIBILITY_PUBLIC, TOKEN_FACTORY, and more.
Details of new JJTree features like NODE_CLASS, VISITOR_DATA_TYPE,
VISITOR_RETURN_TYPE, and more.
Updated examples to reflect JavaCC's current defaults - e.g.,
JDK_VERSION is now 1.5 by default, the options block is now optional
if empty, and more.
Updated chapter 8 (Case Study: The JavaCC Grammar) to reflect
latest JavaCC grammar.
Update software versions used throughout - the second edition uses
much more recent versions of Java, Maven, JUnit, Jaxen.
Changes since JavaCC 4.0 are flagged in the index for easy lookup.
Many small fixes to the text, to examples, to code samples.

I'm excited to get this second edition out there; I think folks will
find that it's a nice refresh. Enjoy!

http://generatingparserswithjavacc.com/

Yours,

Tom

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: (jjtree) Problem with TRACK_TOKENS - firstToken is wrong

2009-10-26T17:31:11Z

> Hi,
>
> I have issues concerning TRACK_TOKENS in jjtree as it sets the wrong
> "firstToken" on some nodes in my case.

.....
>
> Is this all a problem with the grammar or a bug in JJTree?
>
> TIA
> Carsten

Hi Carsten,

That sounds like a bug in jjtree (I've not checked the details, but using
getToken(1) doesn't sound right).

Would you mind creating an issue
(https://javacc.dev.java.net/servlets/ProjectIssues) so that we remember
to fix it in the next release?

Cheers.
Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Token Interface and JFlex

2009-10-26T17:30:16Z

KawaDD is now known as FreeCC I believe.

On Tue, Oct 27, 2009 at 10:45 AM, Paul Cager <paul-debian@...> wrote:

> I also would like to vote for getting rid of the "wired in" access to
> the Token class, and think providing an IToken interface would be better.
>

....

> Some tinkering was necessary to make it work, and I believe the task
> would have been a bit easier if an IToken interface had been used by the
> generated code. Are there currently any plans to move in that direction?
>
> Bill Fenlason

Yes, modernising JavaCC is probably high on the "to do" list. There are
also two forks of JavaCC with a more modern API (but not backwards
compatible):

https://javacc.dev.java.net/servlets/ReadMsg?list=users&msgNo=2923

and

KawaDD.

Cheers,
Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

--
- J.Chris Findlay
(c:

Re: Token Interface and JFlex

2009-10-26T14:45:34Z

> I also would like to vote for getting rid of the "wired in" access to
> the Token class, and think providing an IToken interface would be better.
>
....

> Some tinkering was necessary to make it work, and I believe the task
> would have been a bit easier if an IToken interface had been used by the
> generated code. Are there currently any plans to move in that direction?
>
> Bill Fenlason

Yes, modernising JavaCC is probably high on the "to do" list. There are
also two forks of JavaCC with a more modern API (but not backwards
compatible):

https://javacc.dev.java.net/servlets/ReadMsg?list=users&msgNo=2923

and

KawaDD.

Cheers,
Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Re: Error Recovery

2009-10-26T13:59:33Z

There are examples in the docs for both shallow and deep error recovery.

On Tue, Oct 27, 2009 at 9:57 AM, Kurt Guenther <kurtg@...> wrote:

Are there any patterns for doing error recovery (to report more then one error to the user) ?

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

--
- J.Chris Findlay
(c:

Error Recovery

2009-10-26T13:57:51Z

Are there any patterns for doing error recovery (to report more then one
error to the user) ?

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

RE: Parsing strings with quotes....

2009-10-26T01:14:51Z

Try this ?

<QUOTED_STRING: "\'" (~["\'"])* ( "\'\'\'" (~["\'"])* )* "\'">

You should also extend this if you can have two apostrophes in a row

Marc MAZAS

-----Message d'origine-----
De : Alexander James [mailto:earwigboy@...]
Envoyé : dimanche 25 octobre 2009 17:55
À : users@...
Objet : [JavaCC] Parsing strings with quotes....

Hi All,

I'm trying to parse a language where single quotes are allowed within strings that are themselves enclosed within the same quotes without an escape character being used.
E.g. 'This is how you'''d use an apostrophe'

At the moment I'm defining a string within single quotes as the
following:

<QUOTED_STRING: "\'" (~["\'"])* "\'">

Has anyone got an suggestions on I could handle this?

Cheers,

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Parsing strings with quotes....

2009-10-25T09:54:55Z

Hi All,

I'm trying to parse a language where single quotes are allowed within
strings that are themselves enclosed within the same quotes without an
escape character being used.
E.g. 'This is how you'''d use an apostrophe'

At the moment I'm defining a string within single quotes as the
following:

<QUOTED_STRING: "\'" (~["\'"])* "\'">

Has anyone got an suggestions on I could handle this?

Cheers,

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Token Interface and JFlex

2009-10-22T08:37:02Z

I also would like to vote for getting rid of the "wired in" access to
the Token class, and think providing an IToken interface would be better.

I used JavaCC and a JFlex generated scanner in my PL/I Eclipse plugin.
I have not seen the JavaCC / JFlex combination mentioned in this forum
and don't know if it is common. In my case, several reasons made it an
effective solution. PL/I has hundreds of keywords (1200+ line lex file)
and I had a flex definition file to start with, scanning and parsing run
in different threads, etc. (details on request)

Some tinkering was necessary to make it work, and I believe the task
would have been a bit easier if an IToken interface had been used by the
generated code. Are there currently any plans to move in that direction?

Bill Fenlason

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

New version of JavaCC uploaded

2009-10-20T16:25:20Z

Hi,

I have uploaded a new release of JavaCC 5.0 to the Documents and Files
secion. This fixes a rather embarrassing problem with the previous release
where the "bin/javacc.bat" file was missing.

Sorry for the inconvenience.

Cheers,
Paul

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

Question on examples/Lookahead

2009-10-20T12:47:30Z

Hi all,

I am newbie to JavaCC and I started by the tutorials. I tried the
examples/Lookahead/Example1.jj and I am surprised that the grammar does
not accept "abc" as it is said on the tutorial. I am using JavaCC 5.0
and I compile and run the example with these commands:

riadh@giscard:Lookahead$ ../../bin/javacc Example1.jj
Java Compiler Compiler Version 5.0 (Parser Generator)
(type "javacc" with no arguments for help)
Reading from file Example1.jj . . .
Warning: Choice conflict in [...] construct at line 60, column 7.
Expansion nested within construct and expansion following construct
have common prefixes, one of which is: "c"
Consider using a lookahead of 2 or more for nested expansion.
File "TokenMgrError.java" is being rebuilt.
File "ParseException.java" is being rebuilt.
File "Token.java" is being rebuilt.
File "SimpleCharStream.java" is being rebuilt.
Parser generated with 0 errors and 1 warnings.
riadh@giscard:Lookahead$ javac *.java
riadh@giscard:Lookahead$ java Example
abc
Exception in thread "main" ParseException: Encountered "<EOF>" at line
1, column 4.
Was expecting:
"c" ...

at Example.generateParseException(Example.java:206)
at Example.jj_consume_token(Example.java:144)
at Example.Input(Example.java:12)
at Example.main(Example.java:6)
riadh@giscard:Lookahead$

Can you help me to understand why "abc" is not accepted?

Best regards,

Riadh.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...