tag:old.nabble.com,2006:forum-13492Nabble - java.net - JavaCC2009-12-17T08:07:44ZJavaCC is a parser/scanner generator for java java.net - JavaCC home is <a href="http://javacc.dev.java.net/" target="_top" rel="nofollow">here</a>.tag:old.nabble.com,2006:post-26830429Re: Example for .properties parser2009-12-17T08:07:44Z2009-12-17T08:07:44ZaghartaHi,
<br>The problem is that the extra iso8859-1 chars are written in unicode format (see java.util.properties load & store).
<br>Because the file that i will edit is similar "but not identical" to .properties files, i need to save the file (from ide) without converting extra utf chars.
<br>From that, the needs to do not edit the file in .properties format.
<br><br>Thanks a lot!
<br><br>Agharta
<br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">darose wrote:</div>
<div class="quote-message shrinkable-quote">agharta wrote:
<br>> Hi all,
<br>> I'm new in javacc, but i don't have found any example about a simple
<br>> .properties parser.
<br>>
<br>> The parser should be simple: Like .properties syntax, it should catch
<br>> "key=value" expression (respectively: key, equal sign and value). See
<br>> Netbeans or eclipse .properties editor for example.
<br>>
<br>> Any idea? I've tired to implement it by myself but wihout success....sorry.
<br>>
<br>> Thanks in advice,
<br>>
<br>> Agharta
<br><br>Just curious: any reason why you can just use
<br>java.util.Properties.load() for this?
<br><br>DR
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: users-unsubscribe@javacc.dev.java.net
<br>For additional commands, e-mail: users-help@javacc.dev.java.net
<br></div>
</div></blockquote>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26830338Re: Example for .properties parser2009-12-17T08:03:27Z2009-12-17T08:03:27ZaghartaNice!
<br>Thanks a lot!!!!!!!!!!
<br><br>Regards,
<br><br>agharta
<br><br><br><blockquote class="quote light-black dark-border-color"><div class="quote light-border-color">
<div class="quote-author" style="font-weight: bold;">Dale Anson wrote:</div>
<div class="quote-message shrinkable-quote"><a href="http://jedit.svn.sourceforge.net/viewvc/jedit/plugins/JavaSideKick/trunk/src/sidekick/property/parser/property/PropertyParser.jj" target="_top" rel="nofollow">http://jedit.svn.sourceforge.net/viewvc/jedit/plugins/JavaSideKick/trunk/src/sidekick/property/parser/property/PropertyParser.jj</a><br><br>On Thu, Dec 17, 2009 at 1:52 AM, agharta <agharta@interfree.it> wrote:
<br><br>>
<br>> Hi all,
<br>> I'm new in javacc, but i don't have found any example about a simple
<br>> .properties parser.
<br>>
<br>> The parser should be simple: Like .properties syntax, it should catch
<br>> "key=value" expression (respectively: key, equal sign and value). See
<br>> Netbeans or eclipse .properties editor for example.
<br>>
<br>> Any idea? I've tired to implement it by myself but wihout success....sorry.
<br>>
<br>> Thanks in advice,
<br>>
<br>> Agharta
<br>>
<br>>
<br>>
<br>>
<br>>
<br>>
<br>> --
<br>> View this message in context:
<br>> <a href="http://old.nabble.com/Example-for-.properties-parser-tp26824955p26824955.html" target="_top" rel="nofollow">http://old.nabble.com/Example-for-.properties-parser-tp26824955p26824955.html</a><br>> Sent from the java.net - javacc users mailing list archive at Nabble.com.
<br>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: users-unsubscribe@javacc.dev.java.net
<br>> For additional commands, e-mail: users-help@javacc.dev.java.net
<br>>
<br>>
</div>
</div></blockquote>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26829908Re: Example for .properties parser2009-12-17T07:38:00Z2009-12-17T07:38:00ZDale Anson<a href="http://jedit.svn.sourceforge.net/viewvc/jedit/plugins/JavaSideKick/trunk/src/sidekick/property/parser/property/PropertyParser.jj" target="_top" rel="nofollow">http://jedit.svn.sourceforge.net/viewvc/jedit/plugins/JavaSideKick/trunk/src/sidekick/property/parser/property/PropertyParser.jj</a><br>
<br><div class="gmail_quote">On Thu, Dec 17, 2009 at 1:52 AM, agharta <span dir="ltr"><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26829908&i=0" target="_top" rel="nofollow">agharta@...</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
Hi all,<br>
I'm new in javacc, but i don't have found any example about a simple<br>
.properties parser.<br>
<br>
The parser should be simple: Like .properties syntax, it should catch<br>
"key=value" expression (respectively: key, equal sign and value). See<br>
Netbeans or eclipse .properties editor for example.<br>
<br>
Any idea? I've tired to implement it by myself but wihout success....sorry.<br>
<br>
Thanks in advice,<br>
<br>
Agharta<br>
<font color="#888888"><br>
<br>
<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href="http://old.nabble.com/Example-for-.properties-parser-tp26824955p26824955.html" target="_blank" rel="nofollow">http://old.nabble.com/Example-for-.properties-parser-tp26824955p26824955.html</a><br>
Sent from the <a href="http://java.net" target="_blank" rel="nofollow">java.net</a> - javacc users mailing list archive at Nabble.com.<br>
<br>
<br>
---------------------------------------------------------------------<br>
To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26829908&i=1" target="_top" rel="nofollow">users-unsubscribe@...</a><br>
For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26829908&i=2" target="_top" rel="nofollow">users-help@...</a><br>
<br>
</font></blockquote></div><br><div style="visibility: hidden;" title="1261064224151" id="_booktextmark_tab_id_"></div><div style="visibility: hidden;" title="1261064224153" id="_booktextmark_tab_id_"></div><div style="visibility: hidden;" title="1261064224155" id="_booktextmark_tab_id_">
</div><div style="visibility: hidden;" title="1261064224157" id="_booktextmark_tab_id_"></div><div style="visibility: hidden;" title="1261064224393" id="_booktextmark_tab_id_"></div><div style="visibility: hidden;" title="1261064261030" id="_booktextmark_tab_id_">
</div>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26829398Re: Example for .properties parser2009-12-17T07:07:59Z2009-12-17T07:07:59Zdaroseagharta wrote:
<div class='shrinkable-quote'><br>> Hi all,
<br>> I'm new in javacc, but i don't have found any example about a simple
<br>> .properties parser.
<br>>
<br>> The parser should be simple: Like .properties syntax, it should catch
<br>> "key=value" expression (respectively: key, equal sign and value). See
<br>> Netbeans or eclipse .properties editor for example.
<br>>
<br>> Any idea? I've tired to implement it by myself but wihout success....sorry.
<br>>
<br>> Thanks in advice,
<br>>
<br>> Agharta
</div><br>Just curious: any reason why you can just use
<br>java.util.Properties.load() for this?
<br><br>DR
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26829398&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26829398&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26824955Example for .properties parser2009-12-17T00:52:19Z2009-12-17T00:52:19ZaghartaHi all,
<br>I'm new in javacc, but i don't have found any example about a simple .properties parser.
<br><br>The parser should be simple: Like .properties syntax, it should catch "key=value" expression (respectively: key, equal sign and value). See Netbeans or eclipse .properties editor for example.
<br><br>Any idea? I've tired to implement it by myself but wihout success....sorry.
<br><br>Thanks in advice,
<br><br>Agharta
<br><br><br><br><br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26765835Re: Using a dictionary for tokens2009-12-13T04:52:47Z2009-12-13T04:52:47ZXue-Feng YangThanks, Marc. Your suggestion works. However, in my system, there are many internationalized grammars so that too many Java files will be generated from different templates and languages. I still think to extend JavaCC to fit this may be a better solution.
<br><br><br><br>----- Original Message ----
<br>From: Mazas Marc <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=0" target="_top" rel="nofollow">mmazas@...</a>>
<br>To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=1" target="_top" rel="nofollow">users@...</a>
<br>Sent: Sun, December 13, 2009 5:14:44 PM
<br>Subject: RE: [JavaCC] Using a dictionary for tokens
<br><br>Hi
<br>For this I would think about using a template engine that would generate one grammar for each language from the grammar skeleton and the property files.
<br>Marc
<br><br>-----Message d'origine-----
<br>De : Xue-Feng Yang [mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=2" target="_top" rel="nofollow">just4look@...</a>]
<br>Envoyé : dimanche 13 décembre 2009 07:35
<br>À : <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=3" target="_top" rel="nofollow">users@...</a>
<br>Objet : Re: [JavaCC] Using a dictionary for tokens
<br><br>Hi,
<br><br>Is it easy to extend JavaCC so that all tokens could be imported with Properties? I need to define internationalized grammars?
<br><br><br>Xue-Feng
<br><br><br>----- Original Message ----
<br>From: "<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=4" target="_top" rel="nofollow">sreeni@...</a>" <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=5" target="_top" rel="nofollow">sreeni@...</a>>
<br>To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=6" target="_top" rel="nofollow">users@...</a>
<br>Cc: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=7" target="_top" rel="nofollow">users@...</a>
<br>Sent: Sun, December 13, 2009 1:08:24 AM
<br>Subject: Re: [JavaCC] Using a dictionary for tokens
<br><br><br>No, you cannot do that. Anyway, why do you need keywords? This is not a formal language. So I suggest you use USER_TOKEN_MANAGER and generate your tokens there. For example, you might want tokens like NOUN, PRONOUN, VERB etc. that you can do more efficiently looking up a dictionary. And then you can probably use the BNF part to parse them.
<br><br>BTW, a technical issue with your specific example - never use a | or string literals. That generates horrible lexers. Use separate tokens for each of the strings e.g., CATS: "cats" | DOGS: "dogs" etc. and have a production for WORD that is just a choice between these tokens.
<br><br>Hope that helps.
<br><br>Sreeni.
<br><div class='shrinkable-quote'><br>> Hello,
<br>> I am using javaCC for morphological parsing of Marathi language.
<br>> Some of the regular expressions are of the form:
<br>>
<br>> WORD: "dogs" | "cats" | "mice" | ...
<br>>
<br>> If I hard-code these words, my parser works fine.
<br>> But, this is not what I desire,
<br>> There is one module which will generate canonical forms given the
<br>> root, and these canonical forms will be there within the above
<br>> specified regular expression.
<br>> E.g. Root Canonical Form
<br>> भक्त भक्ता
<br>> माळा माळ्या
<br>>
<br>> Canonical forms are genrated run-time. And then my regular expression
<br>> should look like:
<br>>
<br>> WORD: <"भक्त" | " भक्ता" | "माळा" | "माळ्या">
<br>>
<br>> Is there anyway to include variables within the regular expression in
<br>> javacc?
<br>>
<br>> -Mugdha
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=8" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=9" target="_top" rel="nofollow">users-help@...</a>
<br>>
<br>>
</div><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=10" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=11" target="_top" rel="nofollow">users-help@...</a>
<br><br><br> __________________________________________________________________
<br>Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now <a href="http://ca.toolbar.yahoo.com" target="_top" rel="nofollow">http://ca.toolbar.yahoo.com</a>.
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=12" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=13" target="_top" rel="nofollow">users-help@...</a>
<br><br><br> __________________________________________________________________
<br>Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now
<br><a href="http://ca.toolbar.yahoo.com" target="_top" rel="nofollow">http://ca.toolbar.yahoo.com</a>.
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=14" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26765835&i=15" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26764543RE: Using a dictionary for tokens2009-12-13T01:14:44Z2009-12-13T01:14:44ZMazas MarcHi
<br>For this I would think about using a template engine that would generate one grammar for each language from the grammar skeleton and the property files.
<br>Marc
<br><br>-----Message d'origine-----
<br>De : Xue-Feng Yang [mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=0" target="_top" rel="nofollow">just4look@...</a>]
<br>Envoyé : dimanche 13 décembre 2009 07:35
<br>À : <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=1" target="_top" rel="nofollow">users@...</a>
<br>Objet : Re: [JavaCC] Using a dictionary for tokens
<br><br>Hi,
<br><br>Is it easy to extend JavaCC so that all tokens could be imported with Properties? I need to define internationalized grammars?
<br><br><br>Xue-Feng
<br><br><br>----- Original Message ----
<br>From: "<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=2" target="_top" rel="nofollow">sreeni@...</a>" <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=3" target="_top" rel="nofollow">sreeni@...</a>>
<br>To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=4" target="_top" rel="nofollow">users@...</a>
<br>Cc: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=5" target="_top" rel="nofollow">users@...</a>
<br>Sent: Sun, December 13, 2009 1:08:24 AM
<br>Subject: Re: [JavaCC] Using a dictionary for tokens
<br><br><br>No, you cannot do that. Anyway, why do you need keywords? This is not a formal language. So I suggest you use USER_TOKEN_MANAGER and generate your tokens there. For example, you might want tokens like NOUN, PRONOUN, VERB etc. that you can do more efficiently looking up a dictionary. And then you can probably use the BNF part to parse them.
<br><br>BTW, a technical issue with your specific example - never use a | or string literals. That generates horrible lexers. Use separate tokens for each of the strings e.g., CATS: "cats" | DOGS: "dogs" etc. and have a production for WORD that is just a choice between these tokens.
<br><br>Hope that helps.
<br><br>Sreeni.
<br><div class='shrinkable-quote'><br>> Hello,
<br>> I am using javaCC for morphological parsing of Marathi language.
<br>> Some of the regular expressions are of the form:
<br>>
<br>> WORD: "dogs" | "cats" | "mice" | ...
<br>>
<br>> If I hard-code these words, my parser works fine.
<br>> But, this is not what I desire,
<br>> There is one module which will generate canonical forms given the
<br>> root, and these canonical forms will be there within the above
<br>> specified regular expression.
<br>> E.g. Root Canonical Form
<br>> भक्त भक्ता
<br>> माळा माळ्या
<br>>
<br>> Canonical forms are genrated run-time. And then my regular expression
<br>> should look like:
<br>>
<br>> WORD: <"भक्त" | " भक्ता" | "माळा" | "माळ्या">
<br>>
<br>> Is there anyway to include variables within the regular expression in
<br>> javacc?
<br>>
<br>> -Mugdha
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=6" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=7" target="_top" rel="nofollow">users-help@...</a>
<br>>
<br>>
</div><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=8" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=9" target="_top" rel="nofollow">users-help@...</a>
<br><br><br> __________________________________________________________________
<br>Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now <a href="http://ca.toolbar.yahoo.com" target="_top" rel="nofollow">http://ca.toolbar.yahoo.com</a>.
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=10" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26764543&i=11" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26763999Re: Using a dictionary for tokens2009-12-12T22:34:46Z2009-12-12T22:34:46ZXue-Feng YangHi,
<br><br>Is it easy to extend JavaCC so that all tokens could be imported with Properties? I need to define internationalized grammars?
<br><br><br>Xue-Feng
<br><br><br>----- Original Message ----
<br>From: "<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=0" target="_top" rel="nofollow">sreeni@...</a>" <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=1" target="_top" rel="nofollow">sreeni@...</a>>
<br>To: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=2" target="_top" rel="nofollow">users@...</a>
<br>Cc: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=3" target="_top" rel="nofollow">users@...</a>
<br>Sent: Sun, December 13, 2009 1:08:24 AM
<br>Subject: Re: [JavaCC] Using a dictionary for tokens
<br><br><br>No, you cannot do that. Anyway, why do you need keywords? This is not a
<br>formal language. So I suggest you use USER_TOKEN_MANAGER and generate your
<br>tokens there. For example, you might want tokens like NOUN, PRONOUN, VERB
<br>etc. that you can do more efficiently looking up a dictionary. And then
<br>you can probably use the BNF part to parse them.
<br><br>BTW, a technical issue with your specific example - never use a | or
<br>string literals. That generates horrible lexers. Use separate tokens for
<br>each of the strings e.g., CATS: "cats" | DOGS: "dogs" etc. and have a
<br>production for WORD that is just a choice between these tokens.
<br><br>Hope that helps.
<br><br>Sreeni.
<br><div class='shrinkable-quote'><br>> Hello,
<br>> I am using javaCC for morphological parsing of Marathi language.
<br>> Some of the regular expressions are of the form:
<br>>
<br>> WORD: "dogs" | "cats" | "mice" | ...
<br>>
<br>> If I hard-code these words, my parser works fine.
<br>> But, this is not what I desire,
<br>> There is one module which will generate canonical forms given the root,
<br>> and these canonical forms will be there within the above specified
<br>> regular expression.
<br>> E.g. Root Canonical Form
<br>> भक्त भक्ता
<br>> माळा माळ्या
<br>>
<br>> Canonical forms are genrated run-time. And then my regular expression
<br>> should look like:
<br>>
<br>> WORD: <"भक्त" | " भक्ता" | "माळा" | "माळ्या">
<br>>
<br>> Is there anyway to include variables within the regular expression in
<br>> javacc?
<br>>
<br>> -Mugdha
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=4" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=5" target="_top" rel="nofollow">users-help@...</a>
<br>>
<br>>
</div><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=6" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=7" target="_top" rel="nofollow">users-help@...</a>
<br><br><br> __________________________________________________________________
<br>Yahoo! Canada Toolbar: Search from anywhere on the web, and bookmark your favourite sites. Download it now
<br><a href="http://ca.toolbar.yahoo.com" target="_top" rel="nofollow">http://ca.toolbar.yahoo.com</a>.
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=8" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26763999&i=9" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26759356Re: Using a dictionary for tokens2009-12-12T09:08:24Z2009-12-12T09:08:24ZSreenivasa Viswanadha<br>No, you cannot do that. Anyway, why do you need keywords? This is not a
<br>formal language. So I suggest you use USER_TOKEN_MANAGER and generate your
<br>tokens there. For example, you might want tokens like NOUN, PRONOUN, VERB
<br>etc. that you can do more efficiently looking up a dictionary. And then
<br>you can probably use the BNF part to parse them.
<br><br>BTW, a technical issue with your specific example - never use a | or
<br>string literals. That generates horrible lexers. Use separate tokens for
<br>each of the strings e.g., CATS: "cats" | DOGS: "dogs" etc. and have a
<br>production for WORD that is just a choice between these tokens.
<br><br>Hope that helps.
<br><br>Sreeni.
<br><div class='shrinkable-quote'><br>> Hello,
<br>> I am using javaCC for morphological parsing of Marathi language.
<br>> Some of the regular expressions are of the form:
<br>>
<br>> WORD: "dogs" | "cats" | "mice" | ...
<br>>
<br>> If I hard-code these words, my parser works fine.
<br>> But, this is not what I desire,
<br>> There is one module which will generate canonical forms given the root,
<br>> and these canonical forms will be there within the above specified
<br>> regular expression.
<br>> E.g. Root Canonical Form
<br>> भक्त भक्ता
<br>> माळा माळ्या
<br>>
<br>> Canonical forms are genrated run-time. And then my regular expression
<br>> should look like:
<br>>
<br>> WORD: <"भक्त" | " भक्ता" | "माळा" | "माळ्या">
<br>>
<br>> Is there anyway to include variables within the regular expression in
<br>> javacc?
<br>>
<br>> -Mugdha
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26759356&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26759356&i=1" target="_top" rel="nofollow">users-help@...</a>
<br>>
<br>>
</div><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26759356&i=2" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26759356&i=3" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26757581Using a dictionary for tokens2009-12-11T22:34:15Z2009-12-11T22:34:15ZMugdha BApatHello,
<br>I am using javaCC for morphological parsing of Marathi language.
<br>Some of the regular expressions are of the form:
<br><br>WORD: "dogs" | "cats" | "mice" | ...
<br><br>If I hard-code these words, my parser works fine.
<br>But, this is not what I desire,
<br>There is one module which will generate canonical forms given the root,
<br>and these canonical forms will be there within the above specified
<br>regular expression.
<br>E.g. Root Canonical Form
<br>भक्त भक्ता
<br>माळा माळ्या
<br><br>Canonical forms are genrated run-time. And then my regular expression
<br>should look like:
<br><br>WORD: <"भक्त" | " भक्ता" | "माळा" | "माळ्या">
<br><br>Is there anyway to include variables within the regular expression in
<br>javacc?
<br><br>-Mugdha
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26757581&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26757581&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26724872RE: Left Recursion Newbie Problem2009-12-10T02:15:29Z2009-12-10T02:15:29ZMazas MarcYou can look at the Java or the JavaCC grammars themselves to see how these kind of constructs are handled :
<br>Expression / ConditionalExpression / ConditionalOrExpression / ConditionalAndExpression / InclusiveOrExpression / ...
<br><br>
<br><br>-----Message d'origine-----
<br>De : happYboi [mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26724872&i=0" target="_top" rel="nofollow">ber_lim25@...</a>]
<br>Envoyé : jeudi 10 décembre 2009 11:03
<br>À : <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26724872&i=1" target="_top" rel="nofollow">users@...</a>
<br>Objet : [JavaCC] Left Recursion Newbie Problem
<br><br><br>Hey guys,
<br><br>I have a lack of practice with left-recursion so here's the problem:
<br><br>Here is a snippet of my specification:
<br><br>void Exp():
<br>{}
<br>{
<br> Plus()
<br>}
<br><br>void Plus():
<br>{}
<br>{
<br> Exp() "+" Exp()
<br>}
<br><br>Of course the above would result in a left recursion problem. Any ideas how should I solve such an issue?
<br>--
<br>View this message in context: <a href="http://old.nabble.com/Left-Recursion-Newbie-Problem-tp26724684p26724684.html" target="_top" rel="nofollow">http://old.nabble.com/Left-Recursion-Newbie-Problem-tp26724684p26724684.html</a><br>Sent from the java.net - javacc users mailing list archive at Nabble.com.
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26724872&i=2" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26724872&i=3" target="_top" rel="nofollow">users-help@...</a>
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26724872&i=4" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26724872&i=5" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26724684Left Recursion Newbie Problem2009-12-10T02:02:44Z2009-12-10T02:02:44ZhappYboiHey guys,
<br><br>I have a lack of practice with left-recursion so here's the problem:
<br><br>Here is a snippet of my specification:
<br><br>void Exp():
<br>{}
<br>{
<br> Plus()
<br>}
<br><br>void Plus():
<br>{}
<br>{
<br> Exp() "+" Exp()
<br>}
<br><br>Of course the above would result in a left recursion problem. Any ideas how should I solve such an issue?<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26623604convert file into another2009-12-03T02:06:36Z2009-12-03T02:06:36ZMichel Erard-6<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=DE-CH link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>Hello,
<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>thanks
for the code snipplet. This does more or less exactly what I was looking for.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>In
real world my records are a bit more complex. I’ve nearly 10 record types
with up to 30 fields each </span><span lang=FR-CH style='font-size:10.0pt;
font-family:Wingdings'>L</span><span lang=FR-CH style='font-size:10.0pt;
font-family:"Courier New"'><o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>To
define them field by field as Token is not possible when they cannot be
described distinct ? Because I know the exact order they arrive in a
record.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>This
would make the jj file a bit more readable.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>Thanks,
<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>Mike<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>PS :
Sorry, if it arrives two times. </span><span style='font-size:10.0pt;
font-family:"Courier New"'>I’m not sure if I had some problems the first
time.;-)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
Note that defining TEST_A and TEST_B as the same thing without using lexical<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
states will trip you up later. When javacc sees a string of digits, it
will<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
assign it the tokenID of TEST_A every time as that is the first longest<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
match found. Better to define general things like numbers using a single<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
NUMBER token to prevent such issues, and to make the productions later<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
easier to read anyway.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
On Thu, Dec 3, 2009 at 12:02 AM, Michel Erard <<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26623604&i=0" target="_top" rel="nofollow">michel.erard@...</a>>wrote:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
hello,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
I'm an absolutely beginner with javaCC, what you'll see in my question;-)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
But it would be nice, if someone could give me some hints.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
I've to read in a file with different types of records and these records<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
has to be converted and written out to a second file. Every field in another<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
way. The file can contain different types of records, so line for line<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
shoult be read in, decided which type of record it is and then be converted<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
in an specific way.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
Im testing with these dummy data in my file:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
123;TYPE_A;123<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
456;TYPE_B;456<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
the jj file, I've made linke this:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
options {<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
STATIC = false;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
LOOKAHEAD = 1;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
IGNORE_CASE = true;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
PARSER_BEGIN(PersoConverter)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
import java.io.File;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
import java.io.FileInputStream;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
import java.io.FileOutputStream;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
import java.io.FileNotFoundException;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
public class PersoConverter {<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
private File outputFile;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
public PersoConverter(File input, File output) throws<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
FileNotFoundException {<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
this(new FileInputStream(input), null);<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
this.outputFile = output;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
PARSER_END(PersoConverter)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
// lexical analyser<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
SKIP : {<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
"\t" | " " | "\n" | "\r"<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
SPECIAL_TOKEN :<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
{<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< FILLER : ";" > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< #EOL : ( "\r\n" | "\n" | "\r" ) ><o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
TOKEN :<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
{<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< RECORD_A : < SEQUENCE_NUMBER > < FILLER > < TYPE_A >
< FILLER > <<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
TEST_A > < EOL > > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< RECORD_B : < SEQUENCE_NUMBER > < FILLER > < TYPE_B >
< FILLER > <<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
TEST_B > < EOL > > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< SEQUENCE_NUMBER : ([ "0"-"9" ])+ > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< TYPE_A : "TYPE_A" > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< TYPE_B : "TYPE_B" > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< TEST_A : ([ "0"-"9" ])+ > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< TEST_B : ([ "0"-"9" ])+ ><o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
public void convert() :<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
{}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
{<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
( <RECORD_A> | < RECORD_B > )* <EOF><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
I'm happy, this does work. </span><span lang=FR-CH style='font-size:10.0pt;
font-family:"Courier New"'>The file can be parsed. But what now? What do I<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
have to do, to write each file of a Record into the output file calling a<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
conversion-method before?<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
Can someone give me a hint?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
Thanks,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
Mike<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>-- <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>-
J.Chris Findlay<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>
(c:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><o:p> </o:p></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26626144convert file into another2009-12-02T23:42:54Z2009-12-02T23:42:54ZMichel Erard-5<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang=DE-CH link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>Hello,
<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>thanks
for the code snipplet. This does more or less exactly what I was looking for.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>In
real world my records are a bit more complex. I’ve nearly 10 record types
with up to 30 fields each </span><span lang=FR-CH style='font-size:10.0pt;
font-family:Wingdings'>L</span><span lang=FR-CH style='font-size:10.0pt;
font-family:"Courier New"'><o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>To
define them field by field as Token is not possible when they cannot be
described distinct ? Because I know the exact order they arrive in a
record.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>This
would make the jj file a bit more readable.<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>Thanks,
<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>Mike<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
Note that defining TEST_A and TEST_B as the same thing without using lexical<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
states will trip you up later. When javacc sees a string of digits, it
will<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
assign it the tokenID of TEST_A every time as that is the first longest<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
match found. Better to define general things like numbers using a single<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
NUMBER token to prevent such issues, and to make the productions later<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
easier to read anyway.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
On Thu, Dec 3, 2009 at 12:02 AM, Michel Erard
<<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26626144&i=0" target="_top" rel="nofollow">michel.erard@...</a>>wrote:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
hello,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
I'm an absolutely beginner with javaCC, what you'll see in my question;-)<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
But it would be nice, if someone could give me some hints.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
I've to read in a file with different types of records and these records<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
has to be converted and written out to a second file. Every field in another<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
way. The file can contain different types of records, so line for line<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
shoult be read in, decided which type of record it is and then be converted<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
in an specific way.<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
Im testing with these dummy data in my file:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
123;TYPE_A;123<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
456;TYPE_B;456<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
the jj file, I've made linke this:<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
options {<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
STATIC = false;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
LOOKAHEAD = 1;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
IGNORE_CASE = true;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
PARSER_BEGIN(PersoConverter)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
import java.io.File;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
import java.io.FileInputStream;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
import java.io.FileOutputStream;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
import java.io.FileNotFoundException;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
public class PersoConverter {<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
private File outputFile;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
public PersoConverter(File input, File output) throws<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
FileNotFoundException {<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
this(new FileInputStream(input), null);<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
this.outputFile = output;<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
PARSER_END(PersoConverter)<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
// lexical analyser<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
SKIP : {<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
"\t" | " " | "\n" | "\r"<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
SPECIAL_TOKEN :<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
{<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< FILLER : ";" > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< #EOL : ( "\r\n" | "\n" | "\r" ) ><o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
TOKEN :<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
{<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< RECORD_A : < SEQUENCE_NUMBER > < FILLER > < TYPE_A >
< FILLER > <<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
TEST_A > < EOL > > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< RECORD_B : < SEQUENCE_NUMBER > < FILLER > < TYPE_B >
< FILLER > <<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
TEST_B > < EOL > > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< SEQUENCE_NUMBER : ([ "0"-"9" ])+ > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< TYPE_A : "TYPE_A" > |<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
< TYPE_B : "TYPE_B" > |<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
< TEST_A : ([ "0"-"9" ])+ > |<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
< TEST_B : ([ "0"-"9" ])+ ><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
public void convert() :<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
{}<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
{<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
( <RECORD_A> | < RECORD_B > )* <EOF><o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
}<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
I'm happy, this does work. </span><span lang=FR-CH style='font-size:10.0pt;
font-family:"Courier New"'>The file can be parsed. But what now? What do I<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
have to do, to write each file of a Record into the output file calling a<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
conversion-method before?<o:p></o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span lang=FR-CH style='font-size:10.0pt;font-family:"Courier New"'>>
Can someone give me a hint?<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
Thanks,<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>>
Mike<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>-- <o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>-
J.Chris Findlay<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'>
(c:<o:p></o:p></span></p>
<p class=MsoNormal><span style='font-size:10.0pt;font-family:"Courier New"'><o:p> </o:p></span></p>
<p class=MsoNormal><o:p> </o:p></p>
</div>
</body>
</html>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26615001Re: convert file into another2009-12-02T11:36:31Z2009-12-02T11:36:31ZJ.Chris FindlayNote that defining TEST_A and TEST_B as the same thing without using lexical states will trip you up later. When javacc sees a string of digits, it will assign it the tokenID of TEST_A every time as that is the first longest match found. Better to define general things like numbers using a single NUMBER token to prevent such issues, and to make the productions later easier to read anyway.<br>
<br><div class="gmail_quote">On Thu, Dec 3, 2009 at 12:02 AM, Michel Erard <span dir="ltr"><<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26615001&i=0" target="_top" rel="nofollow">michel.erard@...</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
hello, <br><br>I'm an absolutely beginner with javaCC, what you'll see in my question;-) But it would be nice, if someone could give me some hints.<br><br>I've to read in a file with different types of records and these records has to be converted and written out to a second file. Every field in another way. The file can contain different types of records, so line for line shoult be read in, decided which type of record it is and then be converted in an specific way.<br>
<br>Im testing with these dummy data in my file:<br><br>123;TYPE_A;123<br>456;TYPE_B;456<br><br>the jj file, I've made linke this:<br><br>options {<br> STATIC = false;<br> LOOKAHEAD = 1;<br> IGNORE_CASE = true;<br>
}<br><br>PARSER_BEGIN(PersoConverter)<br><br>import java.io.File;<br>import java.io.FileInputStream;<br>import java.io.FileOutputStream;<br>import java.io.FileNotFoundException;<br><br>public class PersoConverter {<br><br>
private File outputFile;<br><br> public PersoConverter(File input, File output) throws FileNotFoundException {<br> this(new FileInputStream(input), null);<br> this.outputFile = output;<br> }<br><br>}<br><br>PARSER_END(PersoConverter)<br>
<br>// lexical analyser<br>SKIP : {<br> "\t" | " " | "\n" | "\r"<br>}<br> <br>SPECIAL_TOKEN :<br>{<br> < FILLER : ";" > |<br> < #EOL : ( "\r\n" | "\n" | "\r" ) ><br>
}<br><br>TOKEN :<br>{<br> < RECORD_A : < SEQUENCE_NUMBER > < FILLER > < TYPE_A > < FILLER > < TEST_A > < EOL > > |<br> < RECORD_B : < SEQUENCE_NUMBER > < FILLER > < TYPE_B > < FILLER > < TEST_B > < EOL > > |<br>
< SEQUENCE_NUMBER : ([ "0"-"9" ])+ > |<br> < TYPE_A : "TYPE_A" > |<br> < TYPE_B : "TYPE_B" > |<br> < TEST_A : ([ "0"-"9" ])+ > |<br>
< TEST_B : ([ "0"-"9" ])+ > <br>}<br><br>public void convert() :<br>{}<br>{ <br> ( <RECORD_A> | < RECORD_B > )* <EOF><br>}<br><br>I'm happy, this does work. The file can be parsed. But what now? What do I have to do, to write each file of a Record into the output file calling a conversion-method before?<br>
<br>Can someone give me a hint?<br><br>Thanks, <br><br>Mike<br>
</blockquote></div><br><br clear="all"><br>-- <br> - J.Chris Findlay<br> (c:<br>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26611750RE: convert file into another2009-12-02T08:24:13Z2009-12-02T08:24:13ZMazas Marc<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<META content="MSHTML 6.00.6000.16825" name=GENERATOR></HEAD>
<BODY>
<DIV dir=ltr align=left><SPAN class=115421816-02122009><FONT face=Arial color=#0000ff size=2>Hi Mike</FONT></SPAN></DIV>
<DIV><FONT face=Arial color=#0000ff size=2></FONT> </DIV>
<DIV><SPAN class=115421816-02122009><FONT face=Arial color=#0000ff size=2>
<DIV><SPAN class=115421816-02122009><FONT face=Arial color=#0000ff size=2>You
can use use syntactic actions (within bnf productions) working on the
token image.</FONT></SPAN></DIV>Here below is an example of what you need
(without any file management harness, and with a simplified grammar to make
it work on the example)</FONT></SPAN></DIV>
<DIV><SPAN class=115421816-02122009><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV><SPAN class=115421816-02122009><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV><SPAN class=115421816-02122009><FONT color=#008000 size=2><FONT face=Arial color=#0000ff size=2>
<DIV align=left>options</DIV></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>STATIC</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> = </FONT><B><FONT size=2><FONT size=2>false</B></FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>;</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>DEBUG_TOKEN_MANAGER</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> = </FONT><B><FONT size=2><FONT size=2>false</B></FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>;</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>LOOKAHEAD</FONT></B></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>= 3;</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>IGNORE_CASE</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> = </FONT><B><FONT size=2><FONT size=2>true</B></FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>;</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>}</FONT></DIV>
<DIV align=left> </DIV></FONT><FONT color=#008000 size=2><FONT color=#008000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>PARSER_BEGIN</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>(Michel)</FONT></DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>package</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> foo.erard;</FONT></DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>import</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>
java.io.BufferedReader;</FONT></DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>import</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> java.io.File;</FONT></DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>import</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>
java.io.FileInputStream;</FONT></DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>import</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>
java.io.FileOutputStream;</FONT></DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>import</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>
java.io.FileNotFoundException;</FONT></DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>import</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> java.io.Reader;</FONT></DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>import</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> java.io.StringReader;</FONT></DIV>
<DIV align=left> </DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>public</FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>class</FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> Michel</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left></FONT><FONT color=#000080 size=2><FONT color=#000080 size=2><FONT face=Arial color=#0000ff>/** ouput buffer
*/</FONT></DIV></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>StringBuilder sb =
</FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>new</FONT></B></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> StringBuilder(256);</FONT></DIV>
<DIV align=left> </DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>public</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><B><FONT size=2><FONT size=2>static</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><B><FONT size=2><FONT size=2>void</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> main(String args []) </FONT><B><FONT size=2><FONT size=2>throws</B></FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> ParseException</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left></FONT><FONT color=#008000 size=2><FONT color=#008000 size=2><FONT face=Arial color=#0000ff>// input buffer (2
lines)</FONT></DIV></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>String someInput =
</FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2><FONT size=2>"123;TYPE_A;123\n"</FONT></FONT><FONT size=2> + </FONT><FONT size=2><FONT size=2>"456;TYPE_B;456\n"</FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>;</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>StringReader sr =
</FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>new</FONT></B></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> StringReader(someInput);</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>Reader r = </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>new</FONT></B></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> BufferedReader(sr);</FONT></DIV>
<DIV align=left></FONT><FONT color=#008000 size=2><FONT color=#008000 size=2><FONT face=Arial color=#0000ff>// create
parser</FONT></DIV></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>Michel parser =
</FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>new</FONT></B></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> Michel(r);</FONT></DIV>
<DIV align=left></FONT><FONT color=#008000 size=2><FONT color=#008000 size=2><FONT face=Arial color=#0000ff>// parse
input</FONT></DIV></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>parser.input();</FONT></DIV>
<DIV align=left></FONT><FONT color=#008000 size=2><FONT color=#008000 size=2><FONT face=Arial color=#0000ff>// display
output</FONT></DIV></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>System.out.println(</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff size=2>"input read :\n"</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> + someInput);</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>System.out.println(</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff size=2>"output produced :\n"</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> + parser.sb.toString());</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>}</FONT></DIV>
<DIV align=left> </DIV>
<DIV align=left></FONT><FONT color=#000080 size=2><FONT color=#000080 size=2><FONT face=Arial color=#0000ff>/** converts a record test field
*/</FONT></DIV></FONT></FONT><FONT size=2>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>static</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> String convert_a(</FONT><B><FONT size=2><FONT size=2>final</B></FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> String aStr)</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>return</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>"'"</FONT></FONT><FONT size=2> + aStr + </FONT><FONT size=2><FONT size=2>"'"</FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>;</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>}</FONT></DIV>
<DIV align=left> </DIV>
<DIV align=left></FONT><FONT color=#000080 size=2><FONT color=#000080 size=2><FONT face=Arial color=#0000ff>/** converts b record test field
*/</FONT></DIV></FONT></FONT><FONT size=2>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>static</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> String convert_b(</FONT><B><FONT size=2><FONT size=2>final</B></FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> String aStr)</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>return</FONT></B></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> String.valueOf(aStr.length());</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>}</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>}</FONT></DIV>
<DIV align=left> </DIV></FONT><FONT color=#008000 size=2><FONT color=#008000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>PARSER_END</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>(Michel)</FONT></DIV>
<DIV align=left> </DIV></FONT><FONT color=#008000 size=2><FONT color=#008000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>SKIP</FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> : { </FONT><FONT size=2><FONT size=2>"\t"</FONT></FONT><FONT size=2> | </FONT><FONT size=2><FONT size=2>"
"</FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>
}</FONT></DIV>
<DIV align=left> </DIV></FONT><FONT color=#008000 size=2><FONT color=#008000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>TOKEN</FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> :</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>FILLER</FONT></FONT><FONT size=2> </FONT><B><FONT size=2><FONT size=2>:</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>";"</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>| </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>EOL</FONT></FONT><FONT size=2> </FONT><B><FONT size=2><FONT size=2>:</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><B><FONT size=2><FONT size=2>(</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>"\r\n"</FONT></FONT><FONT size=2> | </FONT><FONT size=2><FONT size=2>"\n"</FONT></FONT><FONT size=2> | </FONT><FONT size=2><FONT size=2>"\r"</FONT></FONT><FONT size=2> </FONT><B><FONT size=2><FONT size=2>)</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>| </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>NUMBER</FONT></FONT><FONT size=2> </FONT><B><FONT size=2><FONT size=2>:</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><B><FONT size=2><FONT size=2>(</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2>[ </FONT><FONT size=2><FONT size=2>"0"</FONT></FONT><FONT size=2>-</FONT><FONT size=2><FONT size=2>"9"</FONT></FONT><FONT size=2> ]</FONT><B><FONT size=2><FONT size=2>)+</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>| </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>TYPE_A</FONT></FONT><FONT size=2> </FONT><B><FONT size=2><FONT size=2>:</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>"TYPE_A"</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>| </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>TYPE_B</FONT></FONT><FONT size=2> </FONT><B><FONT size=2><FONT size=2>:</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>"TYPE_B"</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>}</FONT></DIV>
<DIV align=left> </DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>public</FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>void</FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> record_a() :</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{ Token t;}</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>t = </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>NUMBER</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(t.image);
}</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>FILLER</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff size=2>"\t"</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>); }</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>t = </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>TYPE_A</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{
sb.append(t.image.toLowerCase()); }</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>FILLER</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff size=2>"\t"</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>); }</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>t = </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>NUMBER</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(convert_a(t.image));
}</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>EOL</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff size=2>"\n"</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>); }</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>}</FONT></DIV>
<DIV align=left> </DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>public</FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>void</FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> record_b() :</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{ Token t;}</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>t = </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>NUMBER</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(t.image);
}</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>FILLER</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff size=2>","</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>); }</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>t = </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>TYPE_B</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(t.image);
}</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>FILLER</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff size=2>","</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>); }</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>t = </FONT></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>NUMBER</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(convert_b(t.image));
}</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff><</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>EOL</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT size=2>
<DIV align=left><FONT face=Arial color=#0000ff>{ sb.append(</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff size=2>"\n"</FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff>); }</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>}</FONT></DIV>
<DIV align=left> </DIV></FONT><FONT color=#800000 size=2><FONT color=#800000 size=2>
<DIV align=left><FONT face=Arial color=#0000ff>public</FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>void</FONT></FONT></FONT></FONT><FONT size=2><FONT face=Arial color=#0000ff> input() :</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{}</FONT></DIV>
<DIV align=left><FONT face=Arial color=#0000ff>{</FONT></DIV>
<DIV align=left></FONT><B><FONT size=2><FONT size=2><FONT face=Arial color=#0000ff>(</FONT></B></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> record_a() | record_b() </FONT><B><FONT size=2><FONT size=2>)*</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><B><FONT size=2><FONT size=2><</B></FONT></FONT></FONT></FONT><FONT face=Arial><FONT color=#0000ff><FONT size=2> </FONT><FONT size=2><FONT size=2>EOF</FONT></FONT><FONT size=2> </FONT><B><FONT color=#ff00ff size=2><FONT color=#ff00ff size=2>></DIV></B></FONT></FONT></FONT></FONT><FONT face=Arial color=#0000ff size=2>
<DIV align=left>}</DIV></FONT></SPAN>
<DIV class=MsoNormal align=left><?xml:namespace prefix = o ns =
"urn:schemas-microsoft-com:office:office" /><o:p><FONT face=Arial color=#0000ff size=2> </FONT></o:p></DIV>
<DIV> </DIV><BR>
<BLOCKQUOTE style="MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=fr dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>De :</B> Michel Erard
[mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26611750&i=0" target="_top" rel="nofollow">michel.erard@...</a>] <BR><B>Envoyé :</B> mercredi 2 décembre
2009 12:02<BR><B>À :</B> <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26611750&i=1" target="_top" rel="nofollow">users@...</a><BR><B>Objet :</B>
[JavaCC] convert file into another<BR></FONT><BR></DIV>
<DIV></DIV>hello, <BR><BR>I'm an absolutely beginner with javaCC, what you'll
see in my question;-) But it would be nice, if someone could give me some
hints.<BR><BR>I've to read in a file with different types of records and these
records has to be converted and written out to a second file. Every field in
another way. The file can contain different types of records, so line for line
shoult be read in, decided which type of record it is and then be converted in
an specific way.<BR><BR>Im testing with these dummy data in my
file:<BR><BR>123;TYPE_A;123<BR>456;TYPE_B;456<BR><BR>the jj file, I've made
linke this:<BR><BR>options {<BR> STATIC =
false;<BR> LOOKAHEAD = 1;<BR> IGNORE_CASE
= true;<BR>}<BR><BR>PARSER_BEGIN(PersoConverter)<BR><BR>import
java.io.File;<BR>import java.io.FileInputStream;<BR>import
java.io.FileOutputStream;<BR>import
java.io.FileNotFoundException;<BR><BR>public class PersoConverter
{<BR><BR> private File outputFile;<BR><BR> public
PersoConverter(File input, File output) throws FileNotFoundException
{<BR> this(new FileInputStream(input),
null);<BR> this.outputFile = output;<BR>
}<BR><BR>}<BR><BR>PARSER_END(PersoConverter)<BR><BR>// lexical
analyser<BR>SKIP : {<BR> "\t" | " " | "\n" |
"\r"<BR>}<BR> <BR>SPECIAL_TOKEN :<BR>{<BR> < FILLER : ";" >
|<BR> < #EOL : ( "\r\n" | "\n" | "\r" ) ><BR>}<BR><BR>TOKEN
:<BR>{<BR> < RECORD_A : < SEQUENCE_NUMBER > <
FILLER > < TYPE_A > < FILLER > < TEST_A > < EOL >
> |<BR> < RECORD_B : < SEQUENCE_NUMBER > <
FILLER > < TYPE_B > < FILLER > < TEST_B > < EOL >
> |<BR> < SEQUENCE_NUMBER : ([ "0"-"9" ])+ >
|<BR> < TYPE_A : "TYPE_A" > |<BR>
< TYPE_B : "TYPE_B" > |<BR> < TEST_A : ([ "0"-"9"
])+ > |<BR> < TEST_B : ([ "0"-"9" ])+ >
<BR>}<BR><BR>public void convert() :<BR>{}<BR>{ <BR>
( <RECORD_A> | < RECORD_B > )* <EOF><BR>}<BR><BR>I'm happy,
this does work. The file can be parsed. But what now? What do I have to do, to
write each file of a Record into the output file calling a conversion-method
before?<BR><BR>Can someone give me a hint?<BR><BR>Thanks,
<BR><BR>Mike<BR></BLOCKQUOTE></BODY></HTML>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26610080Re: convert file into another2009-12-02T06:46:57Z2009-12-02T06:46:57ZFarrukh NajmiTom Copeland wrote:
<div class='shrinkable-quote'><br>> On Dec 2, 2009, at 6:02 AM, Michel Erard wrote:
<br>>
<br>>
<br>>> hello,
<br>>>
<br>>> I'm an absolutely beginner with javaCC, what you'll see in my question;-) But it would be nice, if someone could give me some hints.
<br>>>
<br>>> I've to read in a file with different types of records and these records has to be converted and written out to a second file. Every field in another way. The file can contain different types of records, so line for line shoult be read in, decided which type of record it is and then be converted in an specific way.
<br>>>
<br>>
<br>> Hi Michael -
<br>>
<br>> One solution would be to collect the data into a List as you parse it, then iterate over the List and convert each record as appropriate. You can collect the data using a syntactic action. Does that help?
<br>>
<br>>
</div><br>There are two basic approaches I have used. One is to translate/convert
<br>as I process the input. This is simpler but does not work in more comlex
<br>cases where multiple passes over the input is required. In the more
<br>complex case you can build a data structure in memory called an Abstract
<br>Syntax Tree (AST) using JavaCC's JJTree feature. The token of the tree
<br>represent your input tokens. You can traverse the tree as many times as
<br>needed for your translation
<br><br>Tom's suggestion above is somewhere in the middle where you are creating
<br>some in-memory representation but without using JJTree and that may be a
<br>fine option for you as I belive you your situation is relatively simpler.
<br><br>ps: And just in case on the off chance that the input format is XML then
<br>you consider XSLT instead of javacc.
<br><br>--
<br>Regards,
<br>Farrukh
<br><br>Web: <a href="http://www.wellfleetsoftware.com" target="_top" rel="nofollow">http://www.wellfleetsoftware.com</a><br><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26610080&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26610080&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26609831Re: convert file into another2009-12-02T06:30:44Z2009-12-02T06:30:44ZTom Copeland<br>On Dec 2, 2009, at 6:02 AM, Michel Erard wrote:
<br><br>> hello,
<br>>
<br>> I'm an absolutely beginner with javaCC, what you'll see in my question;-) But it would be nice, if someone could give me some hints.
<br>>
<br>> I've to read in a file with different types of records and these records has to be converted and written out to a second file. Every field in another way. The file can contain different types of records, so line for line shoult be read in, decided which type of record it is and then be converted in an specific way.
<br><br>Hi Michael -
<br><br>One solution would be to collect the data into a List as you parse it, then iterate over the List and convert each record as appropriate. You can collect the data using a syntactic action. Does that help?
<br><br>Yours,
<br><br>Tom
<br><a href="http://generatingparserswithjavacc.com/" target="_top" rel="nofollow">http://generatingparserswithjavacc.com/</a><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26609831&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26609831&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26608585convert file into another2009-12-02T03:02:01Z2009-12-02T03:02:01ZMichel Erard-6hello, <br><br>I'm an absolutely beginner with javaCC, what you'll see in my question;-) But it would be nice, if someone could give me some hints.<br><br>I've to read in a file with different types of records and these records has to be converted and written out to a second file. Every field in another way. The file can contain different types of records, so line for line shoult be read in, decided which type of record it is and then be converted in an specific way.<br>
<br>Im testing with these dummy data in my file:<br><br>123;TYPE_A;123<br>456;TYPE_B;456<br><br>the jj file, I've made linke this:<br><br>options {<br> STATIC = false;<br> LOOKAHEAD = 1;<br> IGNORE_CASE = true;<br>
}<br><br>PARSER_BEGIN(PersoConverter)<br><br>import java.io.File;<br>import java.io.FileInputStream;<br>import java.io.FileOutputStream;<br>import java.io.FileNotFoundException;<br><br>public class PersoConverter {<br><br>
private File outputFile;<br><br> public PersoConverter(File input, File output) throws FileNotFoundException {<br> this(new FileInputStream(input), null);<br> this.outputFile = output;<br> }<br><br>}<br><br>PARSER_END(PersoConverter)<br>
<br>// lexical analyser<br>SKIP : {<br> "\t" | " " | "\n" | "\r"<br>}<br> <br>SPECIAL_TOKEN :<br>{<br> < FILLER : ";" > |<br> < #EOL : ( "\r\n" | "\n" | "\r" ) ><br>
}<br><br>TOKEN :<br>{<br> < RECORD_A : < SEQUENCE_NUMBER > < FILLER > < TYPE_A > < FILLER > < TEST_A > < EOL > > |<br> < RECORD_B : < SEQUENCE_NUMBER > < FILLER > < TYPE_B > < FILLER > < TEST_B > < EOL > > |<br>
< SEQUENCE_NUMBER : ([ "0"-"9" ])+ > |<br> < TYPE_A : "TYPE_A" > |<br> < TYPE_B : "TYPE_B" > |<br> < TEST_A : ([ "0"-"9" ])+ > |<br>
< TEST_B : ([ "0"-"9" ])+ > <br>}<br><br>public void convert() :<br>{}<br>{ <br> ( <RECORD_A> | < RECORD_B > )* <EOF><br>}<br><br>I'm happy, this does work. The file can be parsed. But what now? What do I have to do, to write each file of a Record into the output file calling a conversion-method before?<br>
<br>Can someone give me a hint?<br><br>Thanks, <br><br>Mike<br>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26592424Re: JavaCC questions :2009-12-01T06:12:31Z2009-12-01T06:12:31ZTom Copeland<br>On Nov 18, 2009, at 4:55 AM, Andrew A wrote:
<br><br>> Hey everyone, I am new to javaCC and have looked into it and have some questions if anyone would be so kind to help answer. I couldn't locate the information in question myself.
<br>>
<br>> 1. Can the parser that javaCC creates be incorporated into a project as a normal class. That way we can invoke the class without the main method when necessary. I am abit confused as to why javaCC requires a main, unless to demonstrate it working by itself.
<br>>
<br>> 2. I would like to not only parse text, but either build an object or add additional notes to the parsed text. JavaCC appears to be able to do this, is there any good examples or tutorials that someone could suggest related to this area.
<br><br>Hi Andrew -
<br><br>As Farrukh said, JJTree is a good way to do this. This tutorial has some good information:
<br><br><a href="http://www.softwaresecretweapons.com/jspwiki/jjtree-tutorial-for-advanced-java-parsing" target="_top" rel="nofollow">http://www.softwaresecretweapons.com/jspwiki/jjtree-tutorial-for-advanced-java-parsing</a><br><br>Yours,
<br><br>Tom
<br><a href="http://generatingparserswithjavacc.com/" target="_top" rel="nofollow">http://generatingparserswithjavacc.com/</a><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26592424&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26592424&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26408213Re: JavaCC questions :2009-11-18T05:56:38Z2009-11-18T05:56:38ZFarrukh Najmi<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Andrew A wrote:
<blockquote cite="mid:BLU143-W21A58FC6C76C798B77C32CC7A30@phx.gbl" type="cite">
Hey
everyone, I am new to javaCC and have looked into it and have some
questions if anyone would be so kind to help answer. I couldn't locate
the information in question myself.<br>
<br>
1. Can the parser that javaCC creates be incorporated into a project as
a normal class. That way we can invoke the class without the main
method when necessary. I am abit confused as to why javaCC requires a
main, unless to demonstrate it working by itself.<br>
</blockquote>
<br>
Yes it can. I use it without main method like a normal class routinely:<br>
<br>
See methods like the following in generated parser main class
MyParser.java (assuming parser is called MyParser):<br>
<br>
/** Constructor with InputStream. */<br>
public MyParser(java.io.InputStream stream) {<br>
this(stream, null);<br>
}<br>
/** Constructor with InputStream and supplied encoding */<br>
public MyParser(java.io.InputStream stream, String encoding) {<br>
...<br>
}<br>
<br>
Create parser using above methods and then call the method
corresponding to your start production to do the parsing of the stream.<br>
<br>
My only issue is that needed a non-arg constructor and a set method to
set the stream. So I had to use some ugly kludges to do that.<br>
If any one know a decent way to address my need please share. Otherwise
I hope the dev team would consider this as as RFE.<br>
<br>
Thanks.<br>
<blockquote cite="mid:BLU143-W21A58FC6C76C798B77C32CC7A30@phx.gbl" type="cite"> <br>
2. I would like to not only parse text, but either build an object or
add additional notes to the parsed text. JavaCC appears to be able to
do this, is there any good examples or tutorials that someone could
suggest related to this area.<br>
</blockquote>
<br>
The jjtree program generates an Abstract SYntax Tree (AST) that can be
decorated with arbitrary info. I have not used it in a while but it was
not to hard to get that working based on available docs as I recall.
Perhaps someone more experienced can guide better on this.<br>
<br>
Best of luck.<br>
<br>
<br>
<pre class="moz-signature" cols="72">--
Regards,
Farrukh
Web: <a class="moz-txt-link-freetext" href="http://www.wellfleetsoftware.com" target="_top" rel="nofollow">http://www.wellfleetsoftware.com</a>
</pre>
</body>
</html>
---------------------------------------------------------------------
To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26408213&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26408213&i=1" target="_top" rel="nofollow">users-help@...</a>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26407680JavaCC questions :2009-11-18T01:55:02Z2009-11-18T01:55:02ZMelanie_Green<html>
<head>
</head>
<body class='hmmessage'>
Hey everyone, I am new to javaCC and have looked into it and have some questions if anyone would be so kind to help answer. I couldn't locate the information in question myself.<BR>
<BR>
1. Can the parser that javaCC creates be incorporated into a project as a normal class. That way we can invoke the class without the main method when necessary. I am abit confused as to why javaCC requires a main, unless to demonstrate it working by itself.<BR>
<BR>
2. I would like to not only parse text, but either build an object or add additional notes to the parsed text. JavaCC appears to be able to do this, is there any good examples or tutorials that someone could suggest related to this area.<BR>
<BR>
Thank You<BR> <br /><hr />New Windows 7: Find the right PC for you. <a href='http://windows.microsoft.com/shop' target='_new' rel="nofollow">Learn more.</a></body>
</html><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26361104Re: We are trying to create an off-side rule parser, but have problems with multiple UNINDENT.2009-11-15T09:13:15Z2009-11-15T09:13:15ZSreenivasa Viswanadha<br>I don't remember now why I did it, but I had to abandon this. So instead,
<br>we did it using semantic lookahead. Basically, just ignore all white space
<br>as you normally do, setTabSize(1) and then do something like the
<br>following.
<br><br>Hope that helps,
<br>Sreeni.
<br><br><br> try
<br> {
<br> LOOKAHEAD({getToken(0).beginLine == getToken(1).beginLine})
<br> stmt_list() /*<NEWLINE>*/ // multiple statements on the same line
<br> |
<br><br> (
<br> { indent = getToken(1).beginColumn; }
<br> (
<br> LOOKAHEAD({ getToken(1).kind != EOF &&
<br> indent == getToken(1).beginColumn })
<br> statement()
<br> ) +
<br> )
<br> }
<br> finally
<br> {
<br> indent = oldIndent;
<br> }
<br><br><br><div class='shrinkable-quote'><br>> We are trying to create an off-side rule parser that uses indentation
<br>> (whitespace) rather than braces to delimit blocks in the abstract syntax.
<br>> Ultimately, we'll simplify the concrete syntax to be more like
<br>> python/haskell, but right now we are using a concrete syntax similar to
<br>> that
<br>> of C, with the exception of INDENT and UNINDENT tokens to delimit blocks
<br>> rather than brace tokens.
<br>>
<br>> In the token manager declarations, we setup a stack to keep track of
<br>> indentation. The stack stores the length of the whitespace comprising
<br>> indentation as ints, and corresponding peek, pop and push methods are
<br>> defined.
<br>>
<br>> We define a DEFAULT lexical state in which the lexar begins, and we also
<br>> switch back to the DEFAULT state from the IN_LINE state whenever we
<br>> encounter an end of line in the latter state. In the DEFAULT state, we
<br>> skip
<br>> empty lines, and we use MORE to capture all the whitespace comprising the
<br>> indentation.
<br>>
<br>> After matching all the spaces or tabs in the current indentation, we have
<br>> an
<br>> action associated with an empty string match that determines if the
<br>> current
<br>> line is indented, unindented or the same indentation as the previous line.
<br>> We compare the length of the whitespace of the current line to the value
<br>> at
<br>> the top of the stack.
<br>>
<br>> If the length of the whitespace of the current line is greater than the
<br>> value on the top of the stack, then an indent occurred, and we push the
<br>> length on the stack, and we switch to the INDENTING lexar state. In this
<br>> state, an empty string match causes the lexar to send an INDENT token to
<br>> the
<br>> parser, and this works in our tests.
<br>>
<br>> If the length of the whitespace of the current line is less than the value
<br>> on the top of the stack, then an unindent occurred, and we pop the stack
<br>> until the value on the top of the stack is the same as the length of the
<br>> current whitespace. Each time we pop, our goal is to send an UNINDENT
<br>> token
<br>> to the parser, but we are presently unable to send more than one UNINDENT
<br>> token. Strangely, an UNINDENT token is sent only after the last pop,
<br>> whereas it would seem that this would happen after the first pop.
<br>>
<br>> The second problem is that the parser becomes stuck in an infinite loop of
<br>> empty string matches in this state. This is also strange because the
<br>> UNINDENT token is only matched after the last pop, but also because after
<br>> the last pop, the length of the whitespace of the current line is equal to
<br>> the value on the top of the stack, so the lexar should break out of the
<br>> while loop in the action, and the last else clause in the action should
<br>> switch back to the IN_LINE lexar state.
<br>>
<br>> -------------------------------------------------------------------------------
<br>>
<br>> Excerpt from Parser.jj:
<br>>
<br>> TOKEN_MGR_DECLS:
<br>> {
<br>> Integer[] init_array = new Integer[] {new Integer(0)};
<br>> java.util.ArrayList<Integer> stack =
<br>> new
<br>> java.util.ArrayList<Integer>(java.util.Arrays.asList(init_array));
<br>> int peek() { return stack.get(stack.size() - 1); }
<br>> int pop() { return stack.remove(stack.size() - 1); }
<br>> void push(int i) { stack.add(i);}
<br>> }
<br>>
<br>> <DEFAULT>
<br>> SKIP:
<br>> {
<br>> "\r" // { System.out.print("(DEFAULT STATE - SKIP: CR)"); }
<br>> | "\n" // { System.out.print("(DEFAULT STATE - SKIP: NL)"); }
<br>> }
<br>>
<br>> <DEFAULT>
<br>> MORE:
<br>> {
<br>> " " // { System.out.print("(DEFAULT STATE - MORE: S)"); }
<br>> | "\t" // { System.out.print("(DEFAULT STATE - MORE: T)"); }
<br>> | ""
<br>> {
<br>> if (image.length() > peek())
<br>> {
<br>> push(image.length());
<br>> System.out.print("(push " + stack + ")");
<br>> SwitchTo(INDENTING);
<br>> }
<br>> else if (image.length() < peek())
<br>> {
<br>> while (image.length() < peek())
<br>> {
<br>> pop();
<br>> System.out.print("(pop " + stack +")");
<br>> SwitchTo(UNINDENTING);
<br>> }
<br>> if (image.length() != peek())
<br>> {
<br>> //throw new TokenMgrError("Improper unindent.",
<br>> //TokenMgrError.LEXICAL_ERROR);
<br>> System.out.print("Error: Improper unindent.");
<br>> }
<br>> }
<br>> else
<br>> {
<br>> SwitchTo(IN_LINE);
<br>> }
<br>> }
<br>> }
<br>>
<br>> <INDENTING>
<br>> TOKEN:
<br>> {
<br>> <INDENT: ""> { System.out.print("<INDENT>"); } : IN_LINE
<br>> }
<br>>
<br>> <UNINDENTING>
<br>> TOKEN:
<br>> {
<br>> <UNINDENT: ""> { System.out.print("<UNINDENT>"); } : DEFAULT
<br>> }
<br>>
<br>> <IN_LINE>
<br>> SKIP:
<br>> {
<br>> | " "
<br>> | "\t"
<br>> | "\r"
<br>> }
<br>>
<br>> <IN_LINE>
<br>> TOKEN:
<br>> {
<br>>
<br>> /* ALL THE REGULAR TOKENS */
<br>>
<br>> | <EOL: "\n"> { System.out.print("<EOL>\n"); } : DEFAULT
<br>> }
<br>>
<br>> -------------------------------------------------------------------------------
<br>>
<br>> Sample input:
<br>>
<br>> int main ()
<br>> if (true)
<br>> return 1;
<br>> int i;
<br>>
<br>> -------------------------------------------------------------------------------
<br>>
<br>> Sample output:
<br>>
<br>> bash-3.2$ ant && java -cp bin proj.Hmm test1.py
<br>> Buildfile: build.xml
<br>>
<br>> printJavaVersion:
<br>> [echo] YOU ARE USING JAVA 1.6.0_11
<br>> [echo]
<br>> [echo] Java located at: /lusr/opt/jdk1.6.0_11/jre
<br>>
<br>> all:
<br>> [javacc] Java Compiler Compiler Version 4.1 (Parser Generator)
<br>> [javacc] (type "javacc" with no arguments for help)
<br>> [javacc] Reading from file
<br>> /v/filer4b/v38q001/cjh/cs345/project/proj/src/proj/parser/Parser.jj . . .
<br>> [javacc] Warning: Line 43, Column 5: Regular expression can be matched
<br>> by
<br>> the empty string ("") in lexical state DEFAULT. This can result in an
<br>> endless loop of empty string matches.
<br>> [javacc] Warning: Line 86, Column 5: Regular expression for UNINDENT
<br>> can
<br>> be matched by the empty string ("") in lexical state UNINDENTING. This
<br>> regular expression along with the regular expressions at line 43, column 5
<br>> forms the cycle
<br>> [javacc] UNINDENTING-->DEFAULT-->DEFAULT
<br>> [javacc] containing regular expressions with empty matches. This can
<br>> result in an endless loop of empty string matches.
<br>> [javacc] File "TokenMgrError.java" is being rebuilt.
<br>> [javacc] File "ParseException.java" is being rebuilt.
<br>> [javacc] File "Token.java" is being rebuilt.
<br>> [javacc] File "SimpleCharStream.java" is being rebuilt.
<br>> [javacc] Parser generated with 0 errors and 2 warnings.
<br>> [javac] Compiling 7 source files to
<br>> /v/filer4b/v38q001/cjh/cs345/project/proj/bin
<br>> [javac] Note:
<br>> /v/filer4b/v38q001/cjh/cs345/project/proj/src/proj/parser/Parser.java uses
<br>> unchecked or unsafe operations.
<br>> [javac] Note: Recompile with -Xlint:unchecked for details.
<br>>
<br>> BUILD SUCCESSFUL
<br>> Total time: 3 seconds
<br>> (program - start)<INT><IDENTIFIER><LPAREN><RPAREN><EOL>
<br>> (block - start)(push [0, 4])<INDENT><IF><LPAREN><RPAREN><EOL>
<br>> (block - start)(push [0, 4, 8])<INDENT><RETURN><INTEGER><SEMI>(return -
<br>> end)<EOL>
<br>> (pop [0, 4])(pop [0])<UNINDENT>(block - end)Exception in thread "main"
<br>> proj.parser.TokenMgrError: Error: Bailing out of infinite loop caused by
<br>> repeated empty string matches at line 4, column 1.
<br>> at proj.parser.ParserTokenManager.MoreLexicalActions(Unknown
<br>> Source)
<br>> at proj.parser.ParserTokenManager.getNextToken(Unknown Source)
<br>> at proj.parser.Parser.jj_ntk(Unknown Source)
<br>> at proj.parser.Parser.ifStatement(Unknown Source)
<br>> at proj.parser.Parser.statement(Unknown Source)
<br>> at proj.parser.Parser.statements(Unknown Source)
<br>> at proj.parser.Parser.block(Unknown Source)
<br>> at proj.parser.Parser.restOfFunction(Unknown Source)
<br>> at proj.parser.Parser.Program(Unknown Source)
<br>> at proj.Hmm.main(Unknown Source)
<br>>
</div><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26361104&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26361104&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26355155We are trying to create an off-side rule parser, but have problems with multiple UNINDENT.2009-11-14T15:30:22Z2009-11-14T15:30:22Zchris heathcottWe are trying to create an off-side rule parser that uses indentation
(whitespace) rather than braces to delimit blocks in the abstract
syntax. Ultimately, we'll simplify the concrete syntax to be more like
python/haskell, but right now we are using a concrete syntax similar to that of C, with the
exception of INDENT and UNINDENT tokens to delimit blocks rather than
brace tokens. <br>
<br>In the token manager declarations, we setup a stack to keep track
of indentation. The stack stores the length of the whitespace
comprising indentation as ints, and corresponding peek, pop and push
methods are defined.<br>
<br>We define a DEFAULT lexical state in which the lexar begins, and we
also switch back to the DEFAULT state from the IN_LINE state whenever
we encounter an end of line in the latter state. In the DEFAULT state,
we skip empty lines, and we use MORE to capture all the whitespace
comprising the indentation.<br>
<br>After matching all the spaces or tabs in the current indentation,
we have an action associated with an empty string match that determines
if the current line is indented, unindented or the same indentation as
the previous line. We compare the length of the whitespace of the
current line to the value at the top of the stack.<br>
<br>If the
length of the whitespace of the current line is greater than the value
on the top of the stack, then an indent occurred, and we push the
length on the stack, and we switch to the INDENTING lexar state. In
this state, an empty string match causes the lexar to send an INDENT
token to the parser, and this works in our tests.<br>
<br>If the length of the whitespace of the current line is less than
the value on the top of the stack, then an unindent occurred, and we
pop the stack until the value on the top of the stack is the same as
the length of the current whitespace. Each time we pop, our goal is to
send an UNINDENT token to the parser, but we are presently unable to
send more than one UNINDENT token. Strangely, an UNINDENT token is
sent only after the last pop, whereas it would seem that this would
happen after the first pop.<br>
<br>The second problem is that the parser becomes stuck in an infinite
loop of empty string matches in this state. This is also strange
because the UNINDENT token is only matched after the last pop, but also
because after the last pop, the length of the whitespace of the current
line is equal to the value on the top of the stack, so the lexar should
break out of the while loop in the action, and the last else clause in
the action should switch back to the IN_LINE lexar state.<br>
<br>-------------------------------------------------------------------------------<br><div id=":35" class="ii gt"><br>Excerpt from Parser.jj:<br><br>TOKEN_MGR_DECLS:<br>{<br> Integer[] init_array = new Integer[] {new Integer(0)};<br>
java.util.ArrayList<Integer> stack =<br>
new java.util.ArrayList<Integer>(java.util.Arrays.asList(init_array));<br> int peek() { return stack.get(stack.size() - 1); }<br> int pop() { return stack.remove(stack.size() - 1); }<br> void push(int i) { stack.add(i);}<br>
}<br><br><DEFAULT><br>SKIP:<br>{<br> "\r" // { System.out.print("(DEFAULT STATE - SKIP: CR)"); }<br> | "\n" // { System.out.print("(DEFAULT STATE - SKIP: NL)"); }<br>}<br>
<br><DEFAULT><br>MORE:<br>{<br> " " // { System.out.print("(DEFAULT STATE - MORE: S)"); }<br> | "\t" // { System.out.print("(DEFAULT STATE - MORE: T)"); }<br> | ""<br>
{<br> if (image.length() > peek())<br> {<br> push(image.length());<br> System.out.print("(push " + stack + ")");<br> SwitchTo(INDENTING);<br> }<br>
else if (image.length() < peek())<br> {<br> while (image.length() < peek())<br> {<br> pop();<br> System.out.print("(pop " + stack +")");<br>
SwitchTo(UNINDENTING);<br> }<br> if (image.length() != peek())<br> {<br> //throw new TokenMgrError("Improper unindent.",<br> //TokenMgrError.LEXICAL_ERROR);<br>
System.out.print("Error: Improper unindent.");<br> }<br> }<br> else<br> {<br> SwitchTo(IN_LINE);<br> }<br> }<br>}<br><br><INDENTING><br>
TOKEN:<br>{<br> <INDENT: ""> { System.out.print("<INDENT>"); } : IN_LINE<br>}<br><br><UNINDENTING><br>TOKEN:<br>{<br> <UNINDENT: ""> { System.out.print("<UNINDENT>"); } : DEFAULT<br>
}<br><br><IN_LINE><br>SKIP:<br>{<br> | " "<br> | "\t"<br> | "\r"<br>}<br><br><IN_LINE><br>TOKEN:<br>{<br> <br> /* ALL THE REGULAR TOKENS */<br><br> | <EOL: "\n"> { System.out.print("<EOL>\n"); } : DEFAULT<br>
}<br><br>-------------------------------------------------------------------------------<br><br>Sample input:<br><br>int main ()<br> if (true)<br> return 1;<br>int i;<br><br>-------------------------------------------------------------------------------<br>
<br>Sample output:<br><br>bash-3.2$ ant && java -cp bin proj.Hmm test1.py<br>Buildfile: build.xml<br><br>printJavaVersion:<br> [echo] YOU ARE USING JAVA 1.6.0_11<br> [echo] <br> [echo] Java located at: /lusr/opt/jdk1.6.0_11/jre<br>
<br>all:<br> [javacc] Java Compiler Compiler Version 4.1 (Parser Generator)<br> [javacc] (type "javacc" with no arguments for help)<br> [javacc] Reading from file /v/filer4b/v38q001/cjh/cs345/project/proj/src/proj/parser/Parser.jj . . .<br>
[javacc] Warning: Line 43, Column 5: Regular expression can be
matched by the empty string ("") in lexical state DEFAULT. This can
result in an endless loop of empty string matches.<br>
[javacc] Warning: Line 86, Column 5: Regular expression for UNINDENT
can be matched by the empty string ("") in lexical state UNINDENTING.
This regular expression along with the regular expressions at line 43,
column 5 forms the cycle <br>
[javacc] UNINDENTING-->DEFAULT-->DEFAULT<br> [javacc] containing regular expressions with empty matches. This can result in an endless loop of empty string matches.<br> [javacc] File "TokenMgrError.java" is being rebuilt.<br>
[javacc] File "ParseException.java" is being rebuilt.<br> [javacc] File "Token.java" is being rebuilt.<br> [javacc] File "SimpleCharStream.java" is being rebuilt.<br> [javacc] Parser generated with 0 errors and 2 warnings.<br>
[javac] Compiling 7 source files to /v/filer4b/v38q001/cjh/cs345/project/proj/bin<br> [javac] Note: /v/filer4b/v38q001/cjh/cs345/project/proj/src/proj/parser/Parser.java uses unchecked or unsafe operations.<br> [javac] Note: Recompile with -Xlint:unchecked for details.<br>
<br>BUILD SUCCESSFUL<br>Total time: 3 seconds<br>(program - start)<INT><IDENTIFIER><LPAREN><RPAREN><EOL><br>(block - start)(push [0, 4])<INDENT><IF><LPAREN><RPAREN><EOL><br>
(block - start)(push [0, 4, 8])<INDENT><RETURN><INTEGER><SEMI>(return - end)<EOL><br>(pop
[0, 4])(pop [0])<UNINDENT>(block - end)Exception in thread "main"
proj.parser.TokenMgrError: Error: Bailing out of infinite loop caused
by repeated empty string matches at line 4, column 1.<br>
at proj.parser.ParserTokenManager.MoreLexicalActions(Unknown Source)<br> at proj.parser.ParserTokenManager.getNextToken(Unknown Source)<br> at proj.parser.Parser.jj_ntk(Unknown Source)<br> at proj.parser.Parser.ifStatement(Unknown Source)<br>
at proj.parser.Parser.statement(Unknown Source)<br> at proj.parser.Parser.statements(Unknown Source)<br> at proj.parser.Parser.block(Unknown Source)<br> at proj.parser.Parser.restOfFunction(Unknown Source)<br>
at proj.parser.Parser.Program(Unknown Source)<br> at proj.Hmm.main(Unknown Source)<br></div>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26295286Re: Use Longs for Line Numbers2009-11-10T18:57:07Z2009-11-10T18:57:07ZTom Copeland<br>On Nov 9, 2009, at 3:46 PM, <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26295286&i=0" target="_top" rel="nofollow">sreeni@...</a> wrote:
<br><br>>
<br>> Yes, we should fix it. But for now, you can simply edit the generated
<br>> files - Token.java and SimpleCharStream.java to make the line numbers
<br>> long. There might be some issues with error reporting, but it should work.
<br>>
<br>> Tom, I'm not sure I understand what you are saying. The line number stuff
<br>> is rather straightforward. It's used purely for reporting. It doesn't
<br>> affect the generated code at all.
<br><br>Hm, maybe I misunderstood... I was thinking that Kurt was saying that there were more than Integer.MAX_INT tokens... so that all those int[] arrays of tokens and such would need to be long[] instead. But maybe I'm misrepresenting his question...
<br><br>Yours,
<br><br>Tom
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26295286&i=1" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26295286&i=2" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26282907[Issue 233] New - Enable back up of one token/ensure next token at certain grammar points2009-11-10T04:55:22Z2009-11-10T04:55:22Zarataj-2<a href="https://javacc.dev.java.net/issues/show_bug.cgi?id=233" target="_top" rel="nofollow">https://javacc.dev.java.net/issues/show_bug.cgi?id=233</a><br> Issue #|233
<br> Summary|Enable back up of one token/ensure next token at certa
<br> |in grammar points
<br> Component|javacc
<br> Version|5.0
<br> Platform|All
<br> OS/Version|All
<br> URL|
<br> Status|NEW
<br> Status whiteboard|
<br> Keywords|
<br> Resolution|
<br> Issue type|ENHANCEMENT
<br> Priority|P3
<br> Subcomponent|general
<br> Assigned to|sreeni
<br> Reported by|arataj
<br><br><br><br><br><br><br>------- Additional comments from <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26282907&i=0" target="_top" rel="nofollow">arataj@...</a> Tue Nov 10 12:55:22 +0000 2009 -------
<br>What about a way to ensure that, at certain points of the grammar, backing up a
<br>single token is safe and, that a next token exists?
<br><br>I use the former for error recovery, but I do not know if it won't cause weird
<br>lexer/parser behavior in some cases.
<br><br>I can ensure the latter by adding artificial choices to the grammar, but I still
<br>do not know if it works in all cases.
<br><br>Such features might add a bit of flexibility that one has with hand--made parsers.
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26282907&i=1" target="_top" rel="nofollow">issues-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26282907&i=2" target="_top" rel="nofollow">issues-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-issues-f13494.html" embed="fixTarget[13494]" target="_top" >java.net - javacc issues</a></p>tag:old.nabble.com,2006:post-26273407Re: Use Longs for Line Numbers2009-11-09T12:46:22Z2009-11-09T12:46:22ZSreenivasa Viswanadha<br>Yes, we should fix it. But for now, you can simply edit the generated
<br>files - Token.java and SimpleCharStream.java to make the line numbers
<br>long. There might be some issues with error reporting, but it should work.
<br><br>Tom, I'm not sure I understand what you are saying. The line number stuff
<br>is rather straightforward. It's used purely for reporting. It doesn't
<br>affect the generated code at all.
<br><div class='shrinkable-quote'><br>>
<br>> On Oct 6, 2009, at 6:57 PM, Kurt Guenther wrote:
<br>>
<br>>>
<br>>> I'm currently parsing an 8 GB file with 241,000,000 lines which is
<br>>> about 12.5% of the 2^31 -1 max value of int. I'm supposed to
<br>>> support files up to 200 GB, so is there a way to generate the token
<br>>> manager with longs instead of ints?
<br>>
<br>> I keep thinking about this one. We can't do just a search and replace
<br>> on the generated code since other stuff uses ints as well - e.g., the
<br>> lexical states are contained in an array of integers. I think it'd
<br>> also affect the parser generating code, since we'd need to make jj_gen
<br>> and jj_ntk and others into longs as well.
<br>>
<br>> At the end of the day, it might be easier to preprocess your input
<br>> data to chunk it into smaller files.
<br>>
<br>> Yours,
<br>>
<br>> Tom
<br>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26273407&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26273407&i=1" target="_top" rel="nofollow">users-help@...</a>
<br>>
<br>>
</div><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26273407&i=2" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26273407&i=3" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26268776Re: Dynamic tokenizing2009-11-09T07:56:47Z2009-11-09T07:56:47ZBill Fenlason-2Can't you just use a symbol table? Macro processors often do this kind
<br>of thing. Another similar example is "typedef" in C.
<br><br>I haven't used COMMON_TOKEN_ACTION since I use USER_TOKEN_MANAGER, but I
<br>would think that you can update a symbol table as the "UserKeywords"
<br>statement is parsed. Then in the common action code you can check
<br>generic word tokens in the symbol table, and if found, change the "kind"
<br>field in the token to a different token type (e.g. USER_KW). The actual
<br>keyword text is in the "image" field. In the grammar you can specify
<br>the <USER_KW> "{" ... "}" sequence anywhere it may occur and process it
<br>as necessary as it is parsed.
<br><br>Depending on what you are doing with the content within the braces, you
<br>may have to do some additional processing in the common action code. If
<br>you were trying to do text substitution or macro processing (i.e.
<br>replacing the USER_KW {...} sequence with a different string of tokens
<br>to be parsed), that would be a whole different ballgame.
<br><br>Bill
<br><br>Bachelier, Georges wrote:
<div class='shrinkable-quote'><br>> Hi!
<br>>
<br>>
<br>>
<br>> I have been searching a solution for three days to the following problem:
<br>>
<br>>
<br>>
<br>> I am building a JavaCC parser for a language having an original feature: the language allows you to define user key words which may be found anywhere in the input file after their declaration.
<br>>
<br>>
<br>>
<br>> User key words are declared with the following statement : UserKeywords <my_user_kw1> ... <my_user_kwn> ;
<br>>
<br>>
<br>>
<br>> Once keywords have been declared, they may be used like this :
<br>>
<br>>
<br>>
<br>> <my_user_kw1> {
<br>>
<br>> <any_sequence_of_characters>
<br>>
<br>> }
<br>>
<br>>
<br>>
<br>> <my_user_kwn> {
<br>>
<br>> <any_sequence_of_characters>
<br>>
<br>> }
<br>>
<br>>
<br>>
<br>> The sequences of characters do not matter; we do not use them, but they are surrounded by braces.
<br>>
<br>>
<br>>
<br>> I have written a parser production in order to get the user key words and I have tried to play with the tokenizer CommonTokenAction function, but without success.
<br>>
<br>>
<br>>
<br>> Could someone help me on this issue, please?
<br>>
<br>>
<br>>
<br>> Georges
<br>>
</div><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26268776&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26268776&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26267178Re: Use Longs for Line Numbers2009-11-09T06:19:53Z2009-11-09T06:19:53ZTom Copeland<br>On Oct 6, 2009, at 6:57 PM, Kurt Guenther wrote:
<br><br>>
<br>> I'm currently parsing an 8 GB file with 241,000,000 lines which is
<br>> about 12.5% of the 2^31 -1 max value of int. I'm supposed to
<br>> support files up to 200 GB, so is there a way to generate the token
<br>> manager with longs instead of ints?
<br><br>I keep thinking about this one. We can't do just a search and replace
<br>on the generated code since other stuff uses ints as well - e.g., the
<br>lexical states are contained in an array of integers. I think it'd
<br>also affect the parser generating code, since we'd need to make jj_gen
<br>and jj_ntk and others into longs as well.
<br><br>At the end of the day, it might be easier to preprocess your input
<br>data to chunk it into smaller files.
<br><br>Yours,
<br><br>Tom
<br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26267178&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26267178&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26267024Re: JavaCC Version 5.0 released.2009-11-09T06:09:45Z2009-11-09T06:09:45ZTom CopelandThat's an interesting idea.... you might want to file an issue here:
<br><br><a href="https://javacc.dev.java.net/servlets/ProjectIssues" target="_top" rel="nofollow">https://javacc.dev.java.net/servlets/ProjectIssues</a><br><br>Just to keep it on the radar.
<br><br>Yours,
<br><br>tom
<br><br>On Sep 1, 2009, at 5:02 AM, Artur Rataj wrote:
<br><div class='shrinkable-quote'><br>> Hello.
<br>>
<br>> What about a way to ensure that, at certain points of the grammar,
<br>> backing up a single token is safe and, that a next token exists?
<br>>
<br>> I use the former for error recovery, but I do not know if it won't
<br>> cause weird lexer/parser behavior in some cases.
<br>>
<br>> I can ensure the latter by adding artificial choices to the grammar,
<br>> but I still do not know if it works in all cases.
<br>>
<br>> As of me, such features would add a bit of flexibility that one has
<br>> with hand--made parsers.
<br>>
<br>> But, perhaps there is virtually no demand for the features among
<br>> JavaCC users.
<br>>
<br>> Best regards,
<br>> Artur
<br>>
</div><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26267024&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26267024&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26262700RE: Dynamic tokenizing2009-11-09T00:25:59Z2009-11-09T00:25:59ZMazas Marc<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML xmlns="http://www.w3.org/TR/REC-html40" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word"><HEAD>
<META http-equiv=Content-Type content="text/html; charset=iso-8859-1">
<META content="MSHTML 6.00.6000.16825" name=GENERATOR>
</HEAD>
<BODY lang=EN-US vLink=purple link=blue>
<DIV dir=ltr align=left><SPAN class=094012108-09112009><FONT face=Arial color=#0000ff size=2>I do not believe you can do dynamic tokenizing directly in
JavaCC.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=094012108-09112009><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=094012108-09112009><FONT face=Arial color=#0000ff size=2>I would think about </FONT></SPAN><SPAN class=094012108-09112009><FONT face=Arial color=#0000ff size=2>writing a first
parser that recognizes your UserKeywords syntax, and in the corresponding
production a) generates a second parser (through println() statements, or a
through a template engine) which will contain all the keywords token
definitions, b) compile it, and c) execute it on the remaining of the input
file.</FONT></SPAN></DIV>
<DIV dir=ltr align=left><SPAN class=094012108-09112009><FONT face=Arial color=#0000ff size=2></FONT></SPAN> </DIV>
<DIV dir=ltr align=left><SPAN class=094012108-09112009><FONT face=Arial color=#0000ff size=2>Marc</FONT></SPAN></DIV>
<DIV> </DIV><FONT face=Arial color=#0000ff size=2></FONT><BR>
<BLOCKQUOTE dir=ltr style="MARGIN-RIGHT: 0px">
<DIV class=OutlookMessageHeader lang=fr dir=ltr align=left>
<HR tabIndex=-1>
<FONT face=Tahoma size=2><B>De :</B> Bachelier, Georges
[mailto:<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26262700&i=0" target="_top" rel="nofollow">georges.bachelier@...</a>] <BR><B>Envoyé :</B> vendredi 6
novembre 2009 20:14<BR><B>À :</B>
<a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26262700&i=1" target="_top" rel="nofollow">users@...</a><BR><B>Objet :</B> [JavaCC] Dynamic
tokenizing<BR></FONT><BR></DIV>
<DIV></DIV>
<DIV class=Section1>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Hi!<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">I have been searching a solution
for three days to the following problem:<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">I am building a JavaCC parser for
a language having an original feature: the language allows you to define user
key words which may be found anywhere in the input file after their
declaration.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">User key words are declared with
the following statement : UserKeywords <my_user_kw1>
<my_user_kwn> ;<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Once keywords have been declared,
they may be used like this :<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><my_user_kw1>
{<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">
<any_sequence_of_characters><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">}<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><my_user_kwn>
{<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">
<any_sequence_of_characters><o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">}<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">The sequences of characters do not
matter; we do not use them, but they are surrounded by
braces.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">I have written a parser production
in order to get the user key words and I have tried to play with the tokenizer
</SPAN></FONT><FONT face="Courier New" color=black size=1><SPAN style="FONT-SIZE: 9pt; BACKGROUND: lime; COLOR: black; FONT-FAMILY: 'Courier New'">CommonTokenAction</SPAN></FONT><FONT face="Courier New" color=black size=1><SPAN style="FONT-SIZE: 9pt; COLOR: black; FONT-FAMILY: 'Courier New'"> function,
but without success.<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Could someone help me on this
issue, please?<o:p></o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial"><o:p> </o:p></SPAN></FONT></P>
<P class=MsoNormal><FONT face=Arial size=2><SPAN style="FONT-SIZE: 10pt; FONT-FAMILY: Arial">Georges
<o:p></o:p></SPAN></FONT></P></DIV></BLOCKQUOTE></BODY></HTML>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26251905Re: (jjtree) Problem with TRACK_TOKENS - firstToken is wrong2009-11-08T01:08:06Z2009-11-08T01:08:06ZCarsten LohmannHi Paul,
<br><br>I created issue 232 for it
<br><a href="https://javacc.dev.java.net/issues/show_bug.cgi?id=232" target="_top" rel="nofollow">https://javacc.dev.java.net/issues/show_bug.cgi?id=232</a><br><br>Cheers,
<br>Carsten
<br><br>Paul Cager wrote:
<div class='shrinkable-quote'><br>>> Hi,
<br>>>
<br>>> I have issues concerning TRACK_TOKENS in jjtree as it sets the wrong
<br>>> "firstToken" on some nodes in my case.
<br>>
<br>> .....
<br>>> Is this all a problem with the grammar or a bug in JJTree?
<br>>>
<br>>> TIA
<br>>> Carsten
<br>>
<br>>
<br>> Hi Carsten,
<br>>
<br>> That sounds like a bug in jjtree (I've not checked the details, but using
<br>> getToken(1) doesn't sound right).
<br>>
<br>> Would you mind creating an issue
<br>> (<a href="https://javacc.dev.java.net/servlets/ProjectIssues" target="_top" rel="nofollow">https://javacc.dev.java.net/servlets/ProjectIssues</a>) so that we remember
<br>> to fix it in the next release?
<br>>
<br>> Cheers.
<br>> Paul
<br>>
<br>>
<br>> ---------------------------------------------------------------------
<br>> To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26251905&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>> For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26251905&i=1" target="_top" rel="nofollow">users-help@...</a>
<br>>
</div><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26251905&i=2" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26251905&i=3" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26251867[Issue 232] New - Problem with TRACK_TOKENS - firstToken is wrong2009-11-08T00:59:58Z2009-11-08T00:59:58Zcarstensis<a href="https://javacc.dev.java.net/issues/show_bug.cgi?id=232" target="_top" rel="nofollow">https://javacc.dev.java.net/issues/show_bug.cgi?id=232</a><br> Issue #|232
<br> Summary|Problem with TRACK_TOKENS - firstToken is wrong
<br> Component|javacc
<br> Version|5.0
<br> Platform|All
<br> OS/Version|All
<br> URL|
<br> Status|NEW
<br> Status whiteboard|
<br> Keywords|
<br> Resolution|
<br> Issue type|DEFECT
<br> Priority|P3
<br> Subcomponent|jjtree
<br> Assigned to|sreeni
<br> Reported by|carstensis
<br><br><br><br><br><br><br>------- Additional comments from <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26251867&i=0" target="_top" rel="nofollow">carstensis@...</a> Sun Nov 8 08:59:58 +0000 2009 -------
<br>I have an issue concerning TRACK_TOKENS in jjtree as it sets the wrong
<br>"firstToken" on some nodes in my case.
<br><br>I'am using the XPath grammar from
<br><a href="http://www.w3.org/2007/01/applets/xpathApplet.html" target="_top" rel="nofollow">http://www.w3.org/2007/01/applets/xpathApplet.html</a><br>and added the TRACK_TOKENS option and an UnparseVisitor just as in the
<br>"VTransformer" javacc example.
<br><br>When I parse the XPath expression "1+2", I get this result as a dump of the
<br>created SimpleNode:
<br><br>|XPath2 (firstToken '1', lastToken EOF)
<br>| XPath (firstToken '1', lastToken '2')
<br>| Expr (firstToken '1', lastToken '2')
<br>| AdditiveExpr (firstToken EOF, lastToken '2') +
<br>| IntegerLiteral (firstToken '+', lastToken '1') 1
<br>| IntegerLiteral (firstToken EOF, lastToken '2') 2
<br><br>Two things are wrong here:
<br># firstToken on Literals is wrong (should be the same as lastToken)
<br># firstToken on "Expr" subnode is wrong (ie. "AdditiveExpr" should have '1' as
<br>firstToken)
<br><br>When parsing an "Expr" (deciding if it's a "AdditiveExpr" or
<br>"MultiplicativeExpr", etc.), the parser creates the corresponding "AdditiveExpr"
<br>node only when it has already reached the end of the expression. And at that point
<br>jjtn001.jjtSetFirstToken(getToken(1));
<br>is already the token *after the end* of the "AdditiveExpr".
<br><br><br>When looking at the code, I see that 2 "AdditiveExpr" nodes are created, the
<br>first has the right firstToken and the second (the one that survives) has the
<br>wrong one.
<br>----------------------------
<br> final public void AdditiveExpr() throws ParseException {
<br> /*@bgen(jjtree) #AdditiveExpr(> 1) */
<br>// *1st*
<br> SimpleNode jjtn000 = new SimpleNode(this, JJTADDITIVEEXPR);
<br> boolean jjtc000 = true;
<br> jjtree.openNodeScope(jjtn000);
<br> jjtn000.jjtSetFirstToken(getToken(1));
<br> try {
<br> MultiplicativeExpr();
<br> label_6:
<br> while (true) {
<br> switch ((jj_ntk==-1)?jj_ntk():jj_ntk) {
<br> case Minus:
<br>[..]
<br> }
<br> switch ((jj_ntk==-1)?jj_ntk():jj_ntk) {
<br> case Plus:
<br>[..]
<br> }
<br> MultiplicativeExpr();
<br>// *2nd*
<br> SimpleNode jjtn001 = new SimpleNode(this, JJTADDITIVEEXPR);
<br> boolean jjtc001 = true;
<br> jjtree.openNodeScope(jjtn001);
<br>
<br>// => *wrong token here*, should use the token "jjtn000.jjtgetFirstToken()"
<br>instead of "getToken(1)"???
<br> jjtn001.jjtSetFirstToken(getToken(1));
<br>----------------------------
<br><br>Here's how I changed the created XPath parser code as a workaround:
<br>- for terminal symbols I replaced
<br>jjtn00x.jjtSetFirstToken(getToken(1));
<br>with
<br>jjtn00x.jjtSetFirstToken(getToken(0));
<br>(the last token was always correct, and for a terminal symbol lastToken ==
<br>firstToken)
<br><br>- replaced the remaining occurences of jjtn00x.jjtSetFirstToken(getToken(1))
<br>with
<br>jjtn00x.jjtSetFirstToken(jjtn000FirstToken);
<br>and put
<br>Token jjtn000FirstToken = getToken(1);
<br>at the beginning of the method like this:
<br>----
<br> final public void RelativePathExpr() throws ParseException {
<br> StepExpr();
<br> Token jjtn000FirstToken = getToken(1);
<br> label_11:
<br> while (true) {
<br> switch ((jj_ntk==-1)?jj_ntk():jj_ntk) {
<br>----
<br>According to my tests, this fixes the problem for the XPath parser, but of
<br>course I'd be happy to abandon this workaround once the issue has been solved in
<br>JJTree :)
<br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26251867&i=1" target="_top" rel="nofollow">issues-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26251867&i=2" target="_top" rel="nofollow">issues-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-issues-f13494.html" embed="fixTarget[13494]" target="_top" >java.net - javacc issues</a></p>tag:old.nabble.com,2006:post-26238408Re: Dynamic tokenizing2009-11-06T12:55:05Z2009-11-06T12:55:05ZFarrukh Najmi<br>This sounds doable. You can probably use LOOKAHEAD with a function call
<br>as parameter instead of a constant value.
<br>See an example here:
<br><br><<a href="http://old.nabble.com/Problem:-semantic-lookahead-calling-boolean-functions-td23087841.html" target="_top" rel="nofollow">http://old.nabble.com/Problem:-semantic-lookahead-calling-boolean-functions-td23087841.html</a>>
<br><br>Bachelier, Georges wrote:
<div class='shrinkable-quote'><br>>
<br>> Hi!
<br>>
<br>> I have been searching a solution for three days to the following problem:
<br>>
<br>> I am building a JavaCC parser for a language having an original
<br>> feature: the language allows you to define user key words which may be
<br>> found anywhere in the input file after their declaration.
<br>>
<br>> User key words are declared with the following statement :
<br>> UserKeywords <my_user_kw1> … <my_user_kwn> ;
<br>>
<br>> Once keywords have been declared, they may be used like this :
<br>>
<br>> <my_user_kw1> {
<br>>
<br>> <any_sequence_of_characters>
<br>>
<br>> }
<br>>
<br>> <my_user_kwn> {
<br>>
<br>> <any_sequence_of_characters>
<br>>
<br>> }
<br>>
<br>> The sequences of characters do not matter; we do not use them, but
<br>> they are surrounded by braces.
<br>>
<br>> I have written a parser production in order to get the user key words
<br>> and I have tried to play with the tokenizer CommonTokenAction
<br>> function, but without success.
<br>>
<br>> Could someone help me on this issue, please?
<br>>
<br>> Georges
<br>>
</div><br><br>--
<br>Regards,
<br>Farrukh
<br><br>Web: <a href="http://www.wellfleetsoftware.com" target="_top" rel="nofollow">http://www.wellfleetsoftware.com</a><br><br><br><br>---------------------------------------------------------------------
<br>To unsubscribe, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26238408&i=0" target="_top" rel="nofollow">users-unsubscribe@...</a>
<br>For additional commands, e-mail: <a href="http://old.nabble.com/user/SendEmail.jtp?type=post&post=26238408&i=1" target="_top" rel="nofollow">users-help@...</a>
<br><br><p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>tag:old.nabble.com,2006:post-26238194Dynamic tokenizing2009-11-06T11:13:50Z2009-11-06T11:13:50ZBachelier, Georges<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
</head>
<body lang=EN-US link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Hi!<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>I have been searching a solution for three days to the
following problem:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>I am building a JavaCC parser for a language having an
original feature: the language allows you to define user key words which may be
found anywhere in the input file after their declaration.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>User key words are declared with the following statement :
UserKeywords <my_user_kw1> … <my_user_kwn> ;<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Once keywords have been declared, they may be used like this
:<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><my_user_kw1> {<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> <any_sequence_of_characters><o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>}<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><my_user_kwn> {<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'> <any_sequence_of_characters><o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>}<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>The sequences of characters do not matter; we do not use
them, but they are surrounded by braces.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>I have written a parser production in order to get the user
key words and I have tried to play with the tokenizer </span></font><font size=1 color=black face="Courier New"><span style='font-size:9.0pt;font-family:
"Courier New";color:black;background:lime'>CommonTokenAction</span></font><font size=1 color=black face="Courier New"><span style='font-size:9.0pt;font-family:
"Courier New";color:black'> function, but without success.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Could someone help me on this issue, please?<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face=Arial><span style='font-size:10.0pt;
font-family:Arial'>Georges <o:p></o:p></span></font></p>
</div>
</body>
</html>
<p>From forum: <a href="http://old.nabble.com/java.net---javacc-users-f13495.html" embed="fixTarget[13495]" target="_top" >java.net - javacc users</a></p>