White space agnostics prevents debugging from working?

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

After a bit more investigation it looks like the WSA pipeline is not emitting the IL ".line" instructions which is why the debugging is not working. (Both compiles create PDBs with the exact same size)

E.g. compare these two sections from the same boo script

Any ideas on how I can fix this?


Compiled with white space agnostic mode
-------------------------------------------
    // Method begins at RVA 0x2050
    // Code size       24 (0x18)
    .maxstack  1
    IL_0000:  /* 17   |                  */ ldc.i4.1
    IL_0001:  /* 28   | (0A)000001       */ call       void [mscorlib/*23000001*/]System.Console/*01000002*/::WriteLine(int32) /* 0A000001 */
    IL_0006:  /* 28   | (0A)000002       */ call       void [mscorlib/*23000001*/]System.Diagnostics.Debugger/*01000003*/::Break() /* 0A000002 */
-------------------------------------------



Compiled with the standard pipeline (notice the .line directives)
-------------------------------------------
    // Method begins at RVA 0x2050
    // Code size       24 (0x18)
    .maxstack  1
    .language '{00000000-0000-0000-0000-000000000000}', '{00000000-0000-0000-0000-000000000000}', '{5A869D0B-6611-11D3-BD2A-0000F80849BD}'
// Source File 'C:\...\wss\test.boo'
    .line 3,4 : 0,0 'C:\\...\\wss\\test.boo'
//000003:         print 1
    IL_0000:  /* 17   |                  */ ldc.i4.1
    IL_0001:  /* 28   | (0A)000001       */ call       void [mscorlib/*23000001*/]System.Console/*01000002*/::WriteLine(int32) /* 0A000001 */
    .line 4,5 : 0,0 ''
//000004:         System.Diagnostics.Debugger.Break()
    IL_0006:  /* 28   | (0A)000002       */ call       void [mscorlib/*23000001*/]System.Diagnostics.Debugger/*01000003*/::Break() /* 0A000002 */
-------------------------------------------




------------
From: boolang@... [mailto:boolang@...] On Behalf Of André van der Merwe
Subject: White space agnostics prevents debugging from working?

When I try debug a boo script with white space agnostic mode on I am then unable to debug. This seems impossible but I can very easily reproduce the problem.

Am I perhaps setting up white space agnostic mode incorrectly? I've always done this
       booc.Parameters.Pipeline.RemoveAt(0);
       booc.Parameters.Pipeline.Insert(0, new WSABooParsingStep());


Does anyone have any idea how I can fix this?  

Thanks in advance



Here is the code that will not debug.
---------------------------------------------------------------------
string code = @"class Tester:
    public def TestMethod():
        print 1
        System.Diagnostics.Debugger.Break()
        print 2
        print 3
    end  //** remove to fix
end  //** remove to fix

print 2";

var codeReader = new StringReader(code);

var booc = new BooCompiler();
booc.Parameters.GenerateInMemory = true;
booc.Parameters.Input.Add(new ReaderInput("test.boo", codeReader));
booc.Parameters.OutputType = CompilerOutputType.ConsoleApplication;
booc.Parameters.Debug = true;
booc.Parameters.WhiteSpaceAgnostic = true;  //** remove to fix
booc.Parameters.Pipeline = new CompileToMemory();

booc.Parameters.Pipeline.RemoveAt(0);  //** remove to fix
booc.Parameters.Pipeline.Insert(0, new WSABooParsingStep());  //** remove to fix

CompilerContext context = booc.Run();

if (context.Errors.Count == 0)
{
                 var asm = context.GeneratedAssembly;
                 var type = asm.GetType("Tester");
                 var o = Activator.CreateInstance(type);
                 var method = type.GetMethod("TestMethod");
                 method.Invoke(o, new object[] { });
}
---------------------------------------------------------------------

If you remove the "//** remove to fix" lines it will debug happily.





--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

2009/11/5 André van der Merwe <AndrevdM@...>:
>
> After a bit more investigation it looks like the WSA pipeline is not emitting the IL ".line" instructions which is why the debugging is not working. (Both compiles create PDBs with the exact same size)
>
> E.g. compare these two sections from the same boo script
>
> Any ideas on how I can fix this?
>

I think the parser might not be counting lines properly but I didn't
have time to confirm this.

Try to just parse something (other than compile it) and see if the
node's LexicalInfo is pointing to the correct place...

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

>I think the parser might not be counting lines properly but I didn't
>have time to confirm this.
>
>Try to just parse something (other than compile it) and see if the
>node's LexicalInfo is pointing to the correct place...

Hi,

I added a compiler step that calls a visitor to check the node's LexicalInfo.Line property in EnterMemberReferenceExpression and for both the WSA and non-WSA the line numbers look correct. (Hope this is what you were suggesting...)

Thanks for the help.




Full test code
--------------------------


class Program
{
    static void Main(string[] args)
    {
        string code = @"
class Tester:
    public def TestMethod():
        print 1
        System.Diagnostics.Debugger.Break()
        print 2
        print 3
    end
end

System.Diagnostics.Debugger.Break()
print 2";

        var codeReader = new StringReader(code);
        File.WriteAllText("test.boo", code);

        var booc = new BooCompiler();
        booc.Parameters.GenerateInMemory = true;
        booc.Parameters.Input.Add(new ReaderInput("test.boo", codeReader));
        booc.Parameters.OutputType = CompilerOutputType.ConsoleApplication;
        booc.Parameters.Debug = true;
        booc.Parameters.WhiteSpaceAgnostic = true;
        //booc.Parameters.Pipeline = new CompileToMemory();
        booc.Parameters.Pipeline = new CompileToFile();
        booc.Parameters.OutputAssembly = "test.exe";

        //booc.Parameters.Pipeline.RemoveAt(0);
        //booc.Parameters.Pipeline.Insert(0, new WSABooParsingStep());
        booc.Parameters.Pipeline[0] = new WSABooParsingStep();

        booc.Parameters.Pipeline.Add(new RunVisitorCompilerStep(new TestVisitor()));

        CompilerContext context = booc.Run();

        if (context.Errors.Count == 0)
        {
            var asm = context.GeneratedAssembly;
            var type = asm.GetType("Tester");
            var o = Activator.CreateInstance(type);
            var method = type.GetMethod("TestMethod");
            method.Invoke(o, new object[] { });
        }
    }
}

class TestVisitor : DepthFirstVisitor
{
    public override bool EnterMemberReferenceExpression(MemberReferenceExpression node)
    {
        Console.WriteLine("{0} {1}", node.LexicalInfo.Line, node.ToString());
        return base.EnterMemberReferenceExpression(node);
    }
}

public class RunVisitorCompilerStep : AbstractCompilerStep
{
    private readonly DepthFirstVisitor m_visitor;

    public RunVisitorCompilerStep(DepthFirstVisitor visitor)
    {
        m_visitor = visitor;
    }

    public override void Run()
    {
        m_visitor.Visit(CompileUnit);
    }
}
--------------------------

Both output this

3 System.Console.WriteLine
3 System.Console
4 System.Diagnostics.Debugger.Break
4 System.Diagnostics.Debugger
4 System.Diagnostics
5 System.Console.WriteLine
5 System.Console
6 System.Console.WriteLine
6 System.Console
10 System.Diagnostics.Debugger.Break
10 System.Diagnostics.Debugger
10 System.Diagnostics
11 System.Console.WriteLine
11 System.Console

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

Ah! So the line number is correct however for the WSA compile the LexicalInfo's FileName and FullPath properties are empty. So I assume this is causing the problem.


I'll look at the source and see if I can spot where this is happening. Any pointers will be welcome :)

Thanks again





-----Original Message-----
From: boolang@... [mailto:boolang@...] On Behalf Of André van der Merwe
Sent: 05 November 2009 21:00
To: boolang@...
Subject: RE: White space agnostics prevents debugging from working?

>I think the parser might not be counting lines properly but I didn't
>have time to confirm this.
>
>Try to just parse something (other than compile it) and see if the
>node's LexicalInfo is pointing to the correct place...

Hi,

I added a compiler step that calls a visitor to check the node's LexicalInfo.Line property in EnterMemberReferenceExpression and for both the WSA and non-WSA the line numbers look correct. (Hope this is what you were suggesting...)

Thanks for the help.




Full test code
--------------------------


class Program
{
    static void Main(string[] args)
    {
        string code = @"
class Tester:
    public def TestMethod():
        print 1
        System.Diagnostics.Debugger.Break()
        print 2
        print 3
    end
end

System.Diagnostics.Debugger.Break()
print 2";

        var codeReader = new StringReader(code);
        File.WriteAllText("test.boo", code);

        var booc = new BooCompiler();
        booc.Parameters.GenerateInMemory = true;
        booc.Parameters.Input.Add(new ReaderInput("test.boo", codeReader));
        booc.Parameters.OutputType = CompilerOutputType.ConsoleApplication;
        booc.Parameters.Debug = true;
        booc.Parameters.WhiteSpaceAgnostic = true;
        //booc.Parameters.Pipeline = new CompileToMemory();
        booc.Parameters.Pipeline = new CompileToFile();
        booc.Parameters.OutputAssembly = "test.exe";

        //booc.Parameters.Pipeline.RemoveAt(0);
        //booc.Parameters.Pipeline.Insert(0, new WSABooParsingStep());
        booc.Parameters.Pipeline[0] = new WSABooParsingStep();

        booc.Parameters.Pipeline.Add(new RunVisitorCompilerStep(new TestVisitor()));

        CompilerContext context = booc.Run();

        if (context.Errors.Count == 0)
        {
            var asm = context.GeneratedAssembly;
            var type = asm.GetType("Tester");
            var o = Activator.CreateInstance(type);
            var method = type.GetMethod("TestMethod");
            method.Invoke(o, new object[] { });
        }
    }
}

class TestVisitor : DepthFirstVisitor
{
    public override bool EnterMemberReferenceExpression(MemberReferenceExpression node)
    {
        Console.WriteLine("{0} {1}", node.LexicalInfo.Line, node.ToString());
        return base.EnterMemberReferenceExpression(node);
    }
}

public class RunVisitorCompilerStep : AbstractCompilerStep
{
    private readonly DepthFirstVisitor m_visitor;

    public RunVisitorCompilerStep(DepthFirstVisitor visitor)
    {
        m_visitor = visitor;
    }

    public override void Run()
    {
        m_visitor.Visit(CompileUnit);
    }
}
--------------------------

Both output this

3 System.Console.WriteLine
3 System.Console
4 System.Diagnostics.Debugger.Break
4 System.Diagnostics.Debugger
4 System.Diagnostics
5 System.Console.WriteLine
5 System.Console
6 System.Console.WriteLine
6 System.Console
10 System.Diagnostics.Debugger.Break
10 System.Diagnostics.Debugger
10 System.Diagnostics
11 System.Console.WriteLine
11 System.Console



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

2009/11/5 André van der Merwe <AndrevdM@...>:
> Ah! So the line number is correct however for the WSA compile the LexicalInfo's FileName and FullPath properties are empty. So I assume this is causing the problem.
>

Hmm... I think there's a special token factory that's required for
file names to be preserved during antlr parsing. Maybe it's not being
configured for the wsa parser....

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

>Hmm... I think there's a special token factory that's required for
>file names to be preserved during antlr parsing. Maybe it's not being
>configured for the wsa parser....

Found it.

In Boo.Lang.Parser.WSABooParser: in the CreateBooLexer method (line 77 boo 0.9.2.3383)

Change: lexer.Initialize(selector, tabSize, BooToken.Creator);
To: lexer.Initialize(selector, tabSize, BooToken.TokenCreator);


If I change that my script debugs perfectly.

Thanks for the help.




-----Original Message-----
From: boolang@... [mailto:boolang@...] On Behalf Of Rodrigo B. de Oliveira
Sent: 05 November 2009 21:23
To: boolang@...
Subject: Re: White space agnostics prevents debugging from working?


2009/11/5 André van der Merwe <AndrevdM@...>:
> Ah! So the line number is correct however for the WSA compile the LexicalInfo's FileName and FullPath properties are empty. So I assume this is causing the problem.
>

Hmm... I think there's a special token factory that's required for
file names to be preserved during antlr parsing. Maybe it's not being
configured for the wsa parser....



--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

I've raised an issue and submitted a patch
        http://jira.codehaus.org/browse/BOO-1262


Thanks again




-----Original Message-----
From: boolang@... [mailto:boolang@...] On Behalf Of André van der Merwe
Sent: 05 November 2009 22:11
To: boolang@...
Subject: RE: White space agnostics prevents debugging from working?

>Hmm... I think there's a special token factory that's required for
>file names to be preserved during antlr parsing. Maybe it's not being
>configured for the wsa parser....

Found it.

In Boo.Lang.Parser.WSABooParser: in the CreateBooLexer method (line 77 boo 0.9.2.3383)

Change: lexer.Initialize(selector, tabSize, BooToken.Creator);
To: lexer.Initialize(selector, tabSize, BooToken.TokenCreator);


If I change that my script debugs perfectly.

Thanks for the help.




-----Original Message-----
From: boolang@... [mailto:boolang@...] On Behalf Of Rodrigo B. de Oliveira
Sent: 05 November 2009 21:23
To: boolang@...
Subject: Re: White space agnostics prevents debugging from working?


2009/11/5 André van der Merwe <AndrevdM@...>:
> Ah! So the line number is correct however for the WSA compile the LexicalInfo's FileName and FullPath properties are empty. So I assume this is causing the problem.
>

Hmm... I think there's a special token factory that's required for
file names to be preserved during antlr parsing. Maybe it's not being
configured for the wsa parser....





--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---

Great! I'll be applying it shortly...

2009/11/5 André van der Merwe <AndrevdM@...>:
> I've raised an issue and submitted a patch
>        http://jira.codehaus.org/browse/BOO-1262
>
>
> Thanks again
> ....

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups "Boo Programming Language" group.
To post to this group, send email to boolang@...
To unsubscribe from this group, send email to boolang+unsubscribe@...
For more options, visit this group at http://groups.google.com/group/boolang?hl=en
-~----------~----~----~----~------~----~------~--~---