[janino-dev] [jira] Created: (JANINO-93) Add Token.getNextToken for walking the parsed file

Add Token.getNextToken for walking the parsed file
--------------------------------------------------

                 Key: JANINO-93
                 URL: http://jira.codehaus.org/browse/JANINO-93
             Project: Janino
          Issue Type: New Feature
            Reporter: Adam Heath
            Assignee: Arno Unkrig
         Attachments: feature_add_nextToken_to_Token.patch

Add a method to Token, getNextToken, that maintains the link of tokens that are parsed from a stream.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JANINO-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_96326 ]

Arno Unkrig commented on JANINO-93:
-----------------------------------

This is an intrusive list, and intrusive lists are not nice. Couldn't you "read()"... and store the tokens in a List?

> Add Token.getNextToken for walking the parsed file
> --------------------------------------------------
>
>                 Key: JANINO-93
>                 URL: http://jira.codehaus.org/browse/JANINO-93
>             Project: Janino
>          Issue Type: New Feature
>            Reporter: Adam Heath
>            Assignee: Arno Unkrig
>         Attachments: feature_add_nextToken_to_Token.patch
>
>
> Add a method to Token, getNextToken, that maintains the link of tokens that are parsed from a stream.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JANINO-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_96421 ]

Adam Heath commented on JANINO-93:
----------------------------------

I assume you mean patch, not list.

Why are intrusive patches not nice?  If a particular feature requires lots of changes, and the changes themselves are *only* constrained with adding that feature, calling it intrusive just because of it's size is a poor evaluation.  Instead, just comment on the issues you see with the implementation, and that problems that might occur.

You missed a critical component of the patch; setSwallowIgnorable.  Please see the following example code:

{noformat}
protected String removeStart(String text, StringBuffer imports, StringBuffer staticDefs) throws IOException, Scanner.LocatedException {}}
        Scanner textScanner = new Scanner(null, new StringReader(text));
        textScanner.setSwallow(false);
        StringWriter importWriter = new StringWriter();
        StringWriter staticWriter = new StringWriter();
        StringWriter writer = importWriter;
        Parser textParser = new Parser(textScanner);
        do {
            Scanner.Token token = textScanner.peek();
            if (token.isComment()) {
                textScanner.read();
            } else if (token.isWhitespace()) {
                textScanner.read();
            } else if (writer == importWriter && token.isKeyword("import")) {
                textScanner.setSwallow(true);
                textParser.parseImportDeclaration();
                textScanner.setSwallow(false);
            } else if (token.isKeyword("static")) {
                writer = staticWriter;
                textScanner.read();
                token = textScanner.peek();
                textScanner.setSwallow(true);
                short mod = (short) (Mod.STATIC | textParser.parseModifiersOpt());
                if (textParser.peekOperator("{")) {
                    textScanner.read();
                    textParser.parseBlock();
                } else if (textParser.peekKeyword("class")) {
                    textScanner.read();
                    textParser.parseClassDeclarationRest(
                        null,
                        mod,
                        Parser.ClassDeclarationContext.TYPE_DECLARATION
                    );
                } else if (textParser.peekKeyword("interface")) {
                    textScanner.read();
                    textParser.parseInterfaceDeclarationRest(
                        null,
                        mod,
                        Parser.InterfaceDeclarationContext.NAMED_TYPE_DECLARATION
                    );
                } else if (textParser.peekKeyword("void")) {
                    textScanner.read();
                    Location location = textParser.location();
                    String name = textParser.readIdentifier();
                    textParser.parseMethodDeclarationRest(
                        null,
                        mod,
                        new Java.BasicType(location, Java.BasicType.VOID),
                        name
                    );
                } else {
                    Java.Type type = textParser.parseType();
                    Location location = textParser.location();
                    String name = textParser.readIdentifier();
                    if (textParser.peekOperator("(")) {
                        textParser.parseMethodDeclarationRest(
                            null,
                            mod,
                            type,
                            name
                        );
                    } else {
                        new Java.FieldDeclaration(
                            location,
                            null,
                            mod,
                            type,
                            textParser.parseFieldDeclarationRest(name)
                        );
                        textParser.readOperator(";");
                    }
                }
                textScanner.setSwallow(false);
            } else {
                break;
            }
            writer.write(token.toString());
            while (token.getNextToken() != null) {
                token = token.getNextToken();
                if (token.getNextToken() == null) break;
                writer.write(token.toString());
            }
        } while (true);
        imports.append(importWriter.getBuffer());
        staticDefs.append(staticWriter.getBuffer());
        textScanner.setSwallow(false);
        StringBuffer newText = new StringBuffer();
        while (!textScanner.peek().isEOF()) {
            Scanner.Token token = textScanner.read();
            //System.err.println("token=" + token);
            String tokenText = token.toString();
            newText.append(tokenText);
        }
        return newText.toString();
    }
{noformat}

> Add Token.getNextToken for walking the parsed file
> --------------------------------------------------
>
>                 Key: JANINO-93
>                 URL: http://jira.codehaus.org/browse/JANINO-93
>             Project: Janino
>          Issue Type: New Feature
>            Reporter: Adam Heath
>            Assignee: Arno Unkrig
>         Attachments: feature_add_nextToken_to_Token.patch
>
>
> Add a method to Token, getNextToken, that maintains the link of tokens that are parsed from a stream.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JANINO-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_99242 ]

Arno Unkrig commented on JANINO-93:
-----------------------------------

{quote}
I assume you mean patch, not list.
{quote}
Nope, I mean "intrusive list": A list that is implemented by adding a "nextToken" field to the element, instead of keep the references to the elements in element-external data structures, as java.util.List does.
{quote}
Why are intrusive patches not nice? If a particular feature requires lots of changes, and the changes themselves are only constrained with adding that feature, calling it intrusive just because of it's size is a poor evaluation. Instead, just comment on the issues you see with the implementation, and that problems that might occur.
{quote}
See above. I don't quite understand what an "intrusive patch" is supposed to be (isn't every path intrusive by definition?), but anyway.
{quote}
You missed a critical component of the patch; setSwallowIgnorable. Please see the following example code:
{quote}
I'm confused. The patch DOES NOT contain anything like "setSwallowableIgnorable()". Can you please explain?


CU

Arno

> Add Token.getNextToken for walking the parsed file
> --------------------------------------------------
>
>                 Key: JANINO-93
>                 URL: http://jira.codehaus.org/browse/JANINO-93
>             Project: Janino
>          Issue Type: New Feature
>            Reporter: Adam Heath
>            Assignee: Arno Unkrig
>         Attachments: feature_add_nextToken_to_Token.patch
>
>
> Add a method to Token, getNextToken, that maintains the link of tokens that are parsed from a stream.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

     [ http://jira.codehaus.org/browse/JANINO-93?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Arno Unkrig resolved JANINO-93.
-------------------------------

    Resolution: Won't Fix

Hi Adam,

I got no feedback on this one, so I set it to "resolved". Feel free to re-open it if you like.


CU

Arno

> Add Token.getNextToken for walking the parsed file
> --------------------------------------------------
>
>                 Key: JANINO-93
>                 URL: http://jira.codehaus.org/browse/JANINO-93
>             Project: Janino
>          Issue Type: New Feature
>            Reporter: Adam Heath
>            Assignee: Arno Unkrig
>         Attachments: feature_add_nextToken_to_Token.patch
>
>
> Add a method to Token, getNextToken, that maintains the link of tokens that are parsed from a stream.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

    [ http://jira.codehaus.org/browse/JANINO-93?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=146051#action_146051 ]

Adam Heath commented on JANINO-93:
----------------------------------

No longer need this anyways, as I rewrote my code to extend SimpleCompiler.

> Add Token.getNextToken for walking the parsed file
> --------------------------------------------------
>
>                 Key: JANINO-93
>                 URL: http://jira.codehaus.org/browse/JANINO-93
>             Project: Janino
>          Issue Type: New Feature
>            Reporter: Adam Heath
>            Assignee: Arno Unkrig
>         Attachments: feature_add_nextToken_to_Token.patch
>
>
> Add a method to Token, getNextToken, that maintains the link of tokens that are parsed from a stream.

--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email