How to define the tokens dynamically?

[imp]

Hi all,
the following is a snippet of TOKEN section of a .jj file;

TOKEN :.
{
< article: "a"|"an"|"the" >
}

Rather than hardcoded I want the tokens to be populated from a table/database/xml/csv .
.
Please inform whether i can specify this in the .jj file??
Please inform if you know any other way(by editing the generated java class or any thing else)
thanks;
imp

[Nathan D. Ryan]

A parser generator (conceptually) produces two pieces, a lexer and a  
parser. The lexer is responsible for chopping up a stream of  
characters into a stream of tokens. The parser is instead responsible  
for recognizing patterns within the stream of tokens.

JavaCC does not provide a mechanism by which you can define tokens in  
some other specification and then have JavaCC generate the lexer from  
that specification. However, Javacc does allow you to specify that a  
generated parser should use a custom lexer of your own creation  
(provided it conforms to a particular interface). You can either  
redefine the tokens as defined in the alternative specification, or  
you can build a custom lexer around that specification.

Nathan


On Sep 9, 2007, at 12.08, imp wrote:

> Hi all, the following is a snippet of TOKEN section of a .jj file;  
> TOKEN : { } Rather than hardcoded I want the tokens to be populated  
> from a table/database/xml/csv . Please inform whether i can specify  
> in the .jj file?? Please inform if you know any other way(by  
> editing the generated javaclass or any thing else) thanks imp
> View this message in context: How to define the tokens dynamically?
> Sent from the java.net - javacc users mailing list archive at  
> Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[J.Chris Findlay]

..or write a javacc parser for the custom xml/whatever format that
outputs standard token definitions, ready for insertion into the .jj
file (c:

On 10/09/2007, Nathan D. Ryan <scudobuio@...> wrote:

> A parser generator (conceptually) produces two pieces, a lexer and a
> parser. The lexer is responsible for chopping up a stream of
> characters into a stream of tokens. The parser is instead responsible
> for recognizing patterns within the stream of tokens.
>
> JavaCC does not provide a mechanism by which you can define tokens in
> some other specification and then have JavaCC generate the lexer from
> that specification. However, Javacc does allow you to specify that a
> generated parser should use a custom lexer of your own creation
> (provided it conforms to a particular interface). You can either
> redefine the tokens as defined in the alternative specification, or
> you can build a custom lexer around that specification.
>
> Nathan
>
>
> On Sep 9, 2007, at 12.08, imp wrote:
>
> > Hi all, the following is a snippet of TOKEN section of a .jj file;
> > TOKEN : { } Rather than hardcoded I want the tokens to be populated
> > from a table/database/xml/csv . Please inform whether i can specify
> > in the .jj file?? Please inform if you know any other way(by
> > editing the generated javaclass or any thing else) thanks imp
> > View this message in context: How to define the tokens dynamically?
> > Sent from the java.net - javacc users mailing list archive at
> > Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@...
> For additional commands, e-mail: users-help@...
>
>

--
 - J.Chris Findlay
   (c:

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...

[Theodore S. Norvell]

You might use javacc's generated tokenizer to find words, but then use a
token
action to look up the word in a table and change the kind field.

--
Dr. Theodore Norvell                    http://www.engr.mun.ca/~theo
Electrical and Computer Engineering,     theo at engr dot mun dot ca
Faculty of Engineering and Applied Science        Ph [phone]
Memorial University of Newfoundland              Fax [phone]
St. John's, NL, Canada, A1B 3X5

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...