« Return to Thread: Antlr v3 and Netbeans Lexer Integration

Re: Antlr v3 and Netbeans Lexer Integration

by Andreas Stefik :: Rate this Message:

Reply to Author | View in Thread


Great work!

I saw that you implemented TokenId like the orginal guide did.
I think its better to define SqlEditorTokenId as enum, and get rid of
the "int id".

 
If I recall, in ANTLR, the reason why that int ID is there is because ANTLR generates .token files, which have the ID's embedded. ANTLR uses these id's internally to store the type of each token. For example, mine looks something like this:

FUNCTION=12
EQUALITY=19
MODULO=33
CLASS=9
DOUBLE_QUOTE=39
PERIOD=17

The nice thing, then, about having an ID is that you can then easily write a program that will regenerate your architecture from the .token files, generating stuff that is conceptually like this:

new CompilerTokenId("FUNCTION", "keyword", 12),
new CompilerTokenId("EQUALITY", "operator", 19),
new CompilerTokenId("MODULO", "operator", 33),
new CompilerTokenId("CLASS", "keyword", 9),
new CompilerTokenId("DOUBLE_QUOTE", "whitespace", 39),
new CompilerTokenId("PERIOD", "operator", 17),

I can't speak for everyone, but in my case, having the ID like this makes the architecture map nicely to ANTLR, which had made it easy to debug (I can check ANTLR internals and compare them with mine).

Anyway, just my two cents.

Andy

 « Return to Thread: Antlr v3 and Netbeans Lexer Integration