Handling escaped characters in strings

[roger_p]

I have a language which allows for strings of the form 'this is a string', and it also allows for backslash as the escape character for things like \t (tab) and \n (newline).  This is defined in Sable similar to lots of other examples I've seen:
  escape_sequence = '\b' | '\t' | '\n' | '\f' | '\r' | '\"' | '\' ''' | '\\' ;
  single_character = [input_char - [''' + '\' ]];
   string_literal = (''' ( single_character | escape_sequence )* ''');

and that all works just fine, in that it will correctly recognize and return a token, for example that contains 'this is \t a string'  or whatever.  
My question is whether there is a simple way to actually substitute  for the escaped characters the actual characters in the token itself.  In other words, I would like when I get the token to not have it contain literally a "\t" but rather contain the ascii TAB character.  Is there a convienent way for me to modify what is stored in the tokens to do that replacement?

Thanks!

Roger

--
An Excellent Credit Score is 750
See Yours in Just 2 Easy Steps!


_______________________________________________
SableCC-Discussion mailing list
SableCC-Discussion@...
http://lists.sablecc.org/listinfo/sablecc-discussion

[Christopher Van Kirk]

How about a pre-parsing adapter that transforms the strings in the token
objects?


-----Original Message-----
From:
sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@...
[mailto:sablecc-discussion-bounces+chris.vankirk=fdcjapan.com@....
org] On Behalf Of Roger Pomeroy
Sent: Thursday, September 10, 2009 4:07 AM
To: sablecc-discussion@...
Subject: Handling escaped characters in strings

I have a language which allows for strings of the form 'this is a string',
and it also allows for backslash as the escape character for things like \t
(tab) and \n (newline).  This is defined in Sable similar to lots of other
examples I've seen:
  escape_sequence = '\b' | '\t' | '\n' | '\f' | '\r' | '\"' | '\' ''' | '\\'
;
  single_character = [input_char - [''' + '\' ]];
   string_literal = (''' ( single_character | escape_sequence )* ''');

and that all works just fine, in that it will correctly recognize and return
a token, for example that contains 'this is \t a string'  or whatever.  
My question is whether there is a simple way to actually substitute  for the
escaped characters the actual characters in the token itself.  In other
words, I would like when I get the token to not have it contain literally a
"\t" but rather contain the ascii TAB character.  Is there a convienent way
for me to modify what is stored in the tokens to do that replacement?

Thanks!

Roger

--
An Excellent Credit Score is 750
See Yours in Just 2 Easy Steps!


_______________________________________________
SableCC-Discussion mailing list
SableCC-Discussion@...
http://lists.sablecc.org/listinfo/sablecc-discussion
No virus found in this incoming message.
Checked by AVG - www.avg.com
Version: 8.5.409 / Virus Database: 270.13.88/2357 - Release Date: 09/09/09
17:50:00


_______________________________________________
SableCC-Discussion mailing list
SableCC-Discussion@...
http://lists.sablecc.org/listinfo/sablecc-discussion