Switching lexer states / Update FAQ ?

View: New views
1 Messages — Rating Filter:   Alert me  

Switching lexer states / Update FAQ ?

by Guenter Bartsch-4 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

hi,

i am currently working on a grammar for a language that has a
different language embedded in it (VHDL 2008 / PSL support) so I ended
up having to switch between lexer states from my parser.

I followed the method descriped in the FAQ

http://www.engr.mun.ca/~theo/JavaCC-FAQ/javacc-faq-moz.htm#tth_sEc3.13

but unfortunately the SetState() code seems to be a bit out of date so
I ended up coding my own function for that purpose:

        private void SetState(int state) {
       if (state != token_source.curLexState) {
       
        int backupCol = jj_nt.beginColumn;
        int backupLine = jj_nt.beginLine;
       
                        int curCol = stream.getEndColumn();
                        int curLine = stream.getEndLine();
       
                        while (curCol > backupCol || curLine > backupLine) {
       
        stream.backup(1);
            curCol = stream.getEndColumn();
            curLine = stream.getEndLine();
        }
       
        if (curCol == backupCol && curLine == backupLine) {
        stream.backup(1);
        }

            token_source.SwitchTo(state);

            jj_nt = token_source.getNextToken();

       }
}

basically i am backup up char by char until i reach the start location
of the current token an re-start tokenizing from there. this might not
be the most performant solution (comments/suggestions very welcome!)
but so far it seems to work for me and treats line/col numbers
correctly.

maybe it is time to update the FAQ?

anyway, thanks for this great tools and keep up the good work,

   guenter

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@...
For additional commands, e-mail: users-help@...