Java subclasses in Ruby

[backspaces]

In the jruby - dev forum I found:

Kresten Krab Thorup (Trifork) wrote:

So, I just submitted fix for JRUBY-71 (against current trunk, 2225).  
This allows one to write a subclass of a Java class in ruby, override/
extend methods, call super, etc., and pass such objects back out to  
java.

<snip>

Enjoy!

Kresten

But I've seen other emails which indicate that JRuby/Java interoperability in terms of subclassing is not working.

So two questions:
1 - Is the goal for JRuby to be able to have complete bi-directionality in terms of subclassing of Java classes?  I.e. much the way quoted above?  
2 - If so, does it currently work?

My specific interest is to use a graphics library, Processing.org, which requires me to subclass its PApplet class, and have that subclass be callable from the Processing library, .. i.e. from Java.

Owen

[Charles Oliver Nutter-2]

backspaces wrote:

> In the jruby - dev forum I found:
>
> Kresten Krab Thorup (Trifork) wrote:
>> So, I just submitted fix for JRUBY-71 (against current trunk, 2225).  
>> This allows one to write a subclass of a Java class in ruby, override/
>> extend methods, call super, etc., and pass such objects back out to  
>> java.
>>
>> <snip>
>>
>> Enjoy!
>>
>> Kresten
>>
>
> But I've seen other emails which indicate that JRuby/Java interoperability
> in terms of subclassing is not working.
>
> So two questions:
> 1 - Is the goal for JRuby to be able to have complete bi-directionality in
> terms of subclassing of Java classes?  I.e. much the way quoted above?  
> 2 - If so, does it currently work?

If by bi-directionality, you mean such subclasses are usable in both
Ruby and Java...yes, that is the intent. I believe it's *partially*
working, but this is going to be a big area of focus over the next
several months.

If by bi-directionality, you mean both Ruby can extend Java and Java can
extend Ruby, well I'd argue the latter isn't possible with a dynamic
language. The subclass can never fulfill the contracts of the Ruby
superclass since it can't add and remove methods, can't be re-opened,
and so on. I believe Groovy supports this, but the superclass you extend
must be made quite a bit less dynamic, and the subclass still doesn't
support all the more dynamic features of Groovy. We'll support Ruby
extending Java, but I doubt we'll support the other direction.

> My specific interest is to use a graphics library, Processing.org, which
> requires me to subclass its PApplet class, and have that subclass be
> callable from the Processing library, .. i.e. from Java.

- Charlie


---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[backspaces]

Charles Oliver Nutter-2 wrote:

backspaces wrote:
> In the jruby - dev forum I found:
> ...
> So two questions:
> 1 - Is the goal for JRuby to be able to have complete bi-directionality in
> terms of subclassing of Java classes?  I.e. much the way quoted above?  
> 2 - If so, does it currently work?

If by bi-directionality, you mean such subclasses are usable in both
Ruby and Java...yes, that is the intent. I believe it's *partially*
working, but this is going to be a big area of focus over the next
several months.

Yup, that's exactly my goal: I want to subclass a Java class within JRuby.  The JRuby class will call the Java superclass methods for graphics primitives.  And the Java superclass will call certain methods in the JRuby subclass for initialization and stepping the graphics through its animation.  Specifically, the protocol is that the superclass will call two methods, setup() and draw() in the JRuby subclass.

If by bi-directionality, you mean both Ruby can extend Java and Java can
extend Ruby, well I'd argue the latter isn't possible with a dynamic
language. The subclass can never fulfill the contracts of the Ruby
superclass since it can't add and remove methods, can't be re-opened,
and so on. I believe Groovy supports this, but the superclass you extend
must be made quite a bit less dynamic, and the subclass still doesn't
support all the more dynamic features of Groovy. We'll support Ruby
extending Java, but I doubt we'll support the other direction.

- Charlie

I'm pretty sure I'll not need this level of sophistication.  Actually it hadn't occurred to me!

    -- Owen
"You can do Anything, but not Everything!"

[Charles Oliver Nutter-2]

backspaces wrote:

>
> Charles Oliver Nutter-2 wrote:
>> backspaces wrote:
>>> In the jruby - dev forum I found:
>>> ...
>>> So two questions:
>>> 1 - Is the goal for JRuby to be able to have complete bi-directionality
>>> in
>>> terms of subclassing of Java classes?  I.e. much the way quoted above?  
>>> 2 - If so, does it currently work?
>> If by bi-directionality, you mean such subclasses are usable in both
>> Ruby and Java...yes, that is the intent. I believe it's *partially*
>> working, but this is going to be a big area of focus over the next
>> several months.
>>
> Yup, that's exactly my goal: I want to subclass a Java class within JRuby.
> The JRuby class will call the Java superclass methods for graphics
> primitives.  And the Java superclass will call certain methods in the JRuby
> subclass for initialization and stepping the graphics through its animation.
> Specifically, the protocol is that the superclass will call two methods,
> setup() and draw() in the JRuby subclass.

That's good! We should support this. If we do not support it correctly
today, we need to work to support it, and that's what this next
development cycle is going to focus on. So any and all bugs you can feed
us about this type of integration will be very important and useful.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[backspaces]

Charles Oliver Nutter-2 wrote:

backspaces wrote:
>
> Charles Oliver Nutter-2 wrote:
>> backspaces wrote:
>>> In the jruby - dev forum I found:
>>> ...
>>> So two questions:
>>> 1 - Is the goal for JRuby to be able to have complete bi-directionality
>>> in
>>> terms of subclassing of Java classes?  I.e. much the way quoted above?  
>>> 2 - If so, does it currently work?
>> If by bi-directionality, you mean such subclasses are usable in both
>> Ruby and Java...yes, that is the intent. I believe it's *partially*
>> working, but this is going to be a big area of focus over the next
>> several months.
>>
> Yup, that's exactly my goal: I want to subclass a Java class within JRuby.
> The JRuby class will call the Java superclass methods for graphics
> primitives.  And the Java superclass will call certain methods in the JRuby
> subclass for initialization and stepping the graphics through its animation.
> Specifically, the protocol is that the superclass will call two methods,
> setup() and draw() in the JRuby subclass.

That's good! We should support this. If we do not support it correctly
today, we need to work to support it, and that's what this next
development cycle is going to focus on. So any and all bugs you can feed
us about this type of integration will be very important and useful.

- Charlie

OK, I've done two tests: one a trivial check to see whether subclassing works the way I think it does, and a second actually using the Processing library.  The first works fine, yeehaaa!  That's wonderful news.   The second fails on an exception that should be caught by the Processing code but somehow does not.  

Here's the first test (note this was also discussed in the groovy forum where I found out that the groovy method signature needs to use "void" rather than "def").

First, I build a trivial java class with a setup method and a util method which simply prints a string.  In addition, I add a method that simply calls this instance's setup method:
public class JavaClass {
  public void setup() {
    util("JavaClass: setup called.");
  }
  public void util(String s) {
    System.out.println(s);
  }
  public void callSetup() {
    this.setup();
  }
}

I then build a java subclass overriding setup(), and include a main which lets me test it.  The test creates an instance of JavaClass and an instance of the subclass.  It then calls setup directly, and indirectly via callSetup:
public class JavaSubClass extends JavaClass {
    public void setup() {
      util("JavaSubClass: setup called.");
    }
    public static void main(String[] args) {
      JavaClass jc = new JavaClass();
      jc.setup();
      jc.callSetup();
      System.out.println("");
      JavaSubClass jsc = new JavaSubClass();
      jsc.setup();
      jsc.callSetup();
    }
}

When I run it, I get what you'd expect:
owen|~/src/jruby[1110]: java JavaSubClass      
JavaClass: setup called.
JavaClass: setup called.

JavaSubClass: setup called.
JavaSubClass: setup called.

Next for the JRuby test:
require 'java'
include_class "JavaClass"

javaClass = JavaClass.new
javaClass.setup()
javaClass.callSetup()

class RubyClass < JavaClass
  def setup()
    util('RubyClass: setup called')
  end
end

puts
rubyClass = RubyClass.new
rubyClass.setup()
rubyClass.callSetup()

.. which works just fine:
owen|~/src/jruby[1111]: jruby testjava.rb
JavaClass: setup called.
JavaClass: setup called.

RubyClass: setup called
RubyClass: setup called

I'll report in another post on the Processing problem.  Briefly, however, it appears to be a problem with catching an exception within the PApplet code:
Applet.java:181:in `java.applet.Applet.getAppletContext': java.lang.NullPointerException: null (NativeException)
   from null:-1:in `processing.core.PApplet$Proxy0.__super$getAppletContext'
   from NativeMethodAccessorImpl.java:-2:in `sun.reflect.NativeMethodAccessorImpl.invoke0'
   from NativeMethodAccessorImpl.java:39:in `sun.reflect.NativeMethodAccessorImpl.invoke'
   from DelegatingMethodAccessorImpl.java:25:in `sun.reflect.DelegatingMethodAccessorImpl.invoke'
   from Method.java:585:in `java.lang.reflect.Method.invoke'
   from JavaMethod.java:204:in `org.jruby.javasupport.JavaMethod.invokeWithExceptionHandling'
..... and so on

Owen

[Nick Sieger-2]

On 3/7/07, backspaces <owen@...> wrote:

> First, I build a trivial java class with a setup method and a util method
> which simply prints a string.  In addition, I add a method that simply calls
> this instance's setup method:
> public class JavaClass {
>   public void setup() {
>     util("JavaClass: setup called.");
>   }
>   public void util(String s) {
>     System.out.println(s);
>   }
>   public void callSetup() {
>     this.setup();
>   }
> }
>
> I then build a java subclass overriding setup(), and include a main which
> lets me test it.  The test creates an instance of JavaClass and an instance
> of the subclass.  It then calls setup directly, and indirectly via
> callSetup:
> public class JavaSubClass extends JavaClass {
>     public void setup() {
>       util("JavaSubClass: setup called.");
>     }
>     public static void main(String[] args) {
>       JavaClass jc = new JavaClass();
>       jc.setup();
>       jc.callSetup();
>       System.out.println("");
>       JavaSubClass jsc = new JavaSubClass();
>       jsc.setup();
>       jsc.callSetup();
>     }
> }
>
> When I run it, I get what you'd expect:
> owen|~/src/jruby[1110]: java JavaSubClass
> JavaClass: setup called.
> JavaClass: setup called.
>
> JavaSubClass: setup called.
> JavaSubClass: setup called.
>
> Next for the JRuby test:
> require 'java'
> include_class "JavaClass"
>
> javaClass = JavaClass.new
> javaClass.setup()
> javaClass.callSetup()
>
> class RubyClass < JavaClass
>   def setup()
>     util('RubyClass: setup called')
>   end
> end
>
> puts
> rubyClass = RubyClass.new
> rubyClass.setup()
> rubyClass.callSetup()
>
> .. which works just fine:
> owen|~/src/jruby[1111]: jruby testjava.rb
> JavaClass: setup called.
> JavaClass: setup called.
>
> RubyClass: setup called
> RubyClass: setup called

I'm not surprised that this works, calling the Ruby subclass from the
Ruby side.  What happens if you pass that object back to Java and try
to call the setup method from there?

/Nick

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[backspaces]

Just a quick pointer to a parallel discussion on the Processing board to see if I'm understanding Processing's internals correctly:
  http://tinyurl.com/32hewk

It includes a pointer to the ruby file, and a test without subclassing, just calling PApplet directly .. which did not have the exception problem.  This leads me to think exception handling works ok in the java classes, but not in a subclass of such a class.  I think.  

But no worries for now, let me dig a bit deeper.

Thanks for your patience.

Owen

[backspaces]

Interesting you suggest that, indeed in the initial tests, "callSetup()" took an argument, the instance on which setup was to be called.  I changed to the simpler approach where the Java callSetup just calls the currently visible setup method directly.  This worked for testing groovy so I presumed it'd be OK for JRuby.

So when I call: rubyClass.callSetup() I presume this causes the Java class to call the currently visible setup() method, which in the Ruby case is the Ruby subclass of the Java superclass.  The only way I think that might not test correctly is if JRuby "flattens" the classes, building a Ruby class with callSetup() promoted into Ruby.

I could try the old method where the instance gets passed around if you think it'd make a  difference.

Owen

Nick Sieger-2 wrote:

On 3/7/07, backspaces <[e-mail]> wrote:
> First, I build a trivial java class with a setup method and a util method
> which simply prints a string.  In addition, I add a method that simply calls
> this instance's setup method:
> public class JavaClass {
>   public void setup() {
>     util("JavaClass: setup called.");
>   }
>   public void util(String s) {
>     System.out.println(s);
>   }
>   public void callSetup() {
>     this.setup();
>   }
> }
>
> I then build a java subclass overriding setup(), and include a main which
> lets me test it.  The test creates an instance of JavaClass and an instance
> of the subclass.  It then calls setup directly, and indirectly via
> callSetup:
> public class JavaSubClass extends JavaClass {
>     public void setup() {
>       util("JavaSubClass: setup called.");
>     }
>     public static void main(String[] args) {
>       JavaClass jc = new JavaClass();
>       jc.setup();
>       jc.callSetup();
>       System.out.println("");
>       JavaSubClass jsc = new JavaSubClass();
>       jsc.setup();
>       jsc.callSetup();
>     }
> }
>
> When I run it, I get what you'd expect:
> owen|~/src/jruby[1110]: java JavaSubClass
> JavaClass: setup called.
> JavaClass: setup called.
>
> JavaSubClass: setup called.
> JavaSubClass: setup called.
>
> Next for the JRuby test:
> require 'java'
> include_class "JavaClass"
>
> javaClass = JavaClass.new
> javaClass.setup()
> javaClass.callSetup()
>
> class RubyClass < JavaClass
>   def setup()
>     util('RubyClass: setup called')
>   end
> end
>
> puts
> rubyClass = RubyClass.new
> rubyClass.setup()
> rubyClass.callSetup()
>
> .. which works just fine:
> owen|~/src/jruby[1111]: jruby testjava.rb
> JavaClass: setup called.
> JavaClass: setup called.
>
> RubyClass: setup called
> RubyClass: setup called

I'm not surprised that this works, calling the Ruby subclass from the
Ruby side.  What happens if you pass that object back to Java and try
to call the setup method from there?

/Nick

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[Charles Oliver Nutter-2]

backspaces wrote:

> Just a quick pointer to a parallel discussion on the Processing board to see
> if I'm understanding Processing's internals correctly:
>   http://tinyurl.com/32hewk
>
> It includes a pointer to the ruby file, and a test without subclassing, just
> calling PApplet directly .. which did not have the exception problem.  This
> leads me to think exception handling works ok in the java classes, but not
> in a subclass of such a class.  I think.  
>
> But no worries for now, let me dig a bit deeper.
>
> Thanks for your patience.

Ok, sounds like good progress so far Owen. Let us know what you find.
That NPE is almost certainly a bug somewhere, but I'm not sure whether
it would be us or processing or your code or what. But the general
structure of what you want to do appears to be working correctly, so
that's great!

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[backspaces]

Nick Sieger-2 wrote:

...
I'm not surprised that this works, calling the Ruby subclass from the
Ruby side.  What happens if you pass that object back to Java and try
to call the setup method from there?

/Nick
...

I modified the test to include passing the instance in to the JavaClass.  I also tested for null/nil.  All works just as we'd hope.

Details:

public class JavaClass {
  public void setup() {
    util("JavaClass: setup called.");
  }
  public void util(String s) {
    System.out.println(s);
  }
  public void callSetup() {
    this.setup();
  }
  public void callSetup(JavaClass ins) {
    try {
      ins.setup();
    } catch (Exception e) {
      util(e+" .. skipping");
    }
  }
}
public class JavaSubClass extends JavaClass {
    public void setup() {
      util("JavaSubClass: setup called.");
    }
    public static void main(String[] args) {
      JavaClass jc = new JavaClass();
      jc.setup();
      jc.callSetup();
      System.out.println("");
      JavaSubClass jsc = new JavaSubClass();
      jsc.setup();
      jsc.callSetup();
      System.out.println("");
      jc.callSetup(jsc);
      jsc.callSetup(jsc);
      System.out.println("");
      jc.callSetup(null);
      jsc.callSetup(null);
    }
}
...
require 'java'
include_class "JavaClass"

javaClass = JavaClass.new
javaClass.setup()
javaClass.callSetup()

class RubyClass < JavaClass
  def setup()
    util('RubyClass: setup called')
  end
end

puts
rubyClass = RubyClass.new
rubyClass.setup()
rubyClass.callSetup()

puts
javaClass.callSetup(rubyClass)
rubyClass.callSetup(rubyClass)

puts
javaClass.callSetup(nil)
rubyClass.callSetup(nil)
...
/src/jruby[1180]: java JavaSubClass
JavaClass: setup called.
JavaClass: setup called.

JavaSubClass: setup called.
JavaSubClass: setup called.

JavaSubClass: setup called.
JavaSubClass: setup called.

java.lang.NullPointerException .. skipping
java.lang.NullPointerException .. skipping
...
owen|~/src/jruby[1181]: jruby testjava.rb
JavaClass: setup called.
JavaClass: setup called.

RubyClass: setup called
RubyClass: setup called

RubyClass: setup called
RubyClass: setup called

java.lang.NullPointerException .. skipping
java.lang.NullPointerException .. skipping

[backspaces]

OK, I looked into the Processing problem.  Several points of interest.

1 - The init() call in PApplet, which does things like launch a thread for animation, performs a test to see if the PApplet is really an applet, or is being used as a panel within an application.  The test looks like:
try {
  getAppletContext();
  online = true;
} catch (NullPointerException e) {
  online = false;
}
The online flag is used within the rest of the library to know how the PApplet/panel is being used.

2 - But the getAppletContext() call does not advertise throwing an exception.  I don't know if that would impact JRuby's java integration subsystem.  The method signature is simply:
  public AppletContext getAppletContext()
.. the return value is simply ignored by Processing .. for good reason, see next:

3 - The code that fails in getAppletContext() is:
public AppletContext getAppletContext() {
  return stub.getAppletContext(); <-- line 181
}
.. which causes the exception to occur:
java.lang.NullPointerException
        at java.applet.Applet.getAppletContext(Applet.java:181)
.. I.e. the stub is not set, thus signals the use of PApplet as a panel, not a true applet.

I can probably figure out a reasonable work around, but for now wanted to report what the problem is.  Possibly JRuby only is aware of advertised exceptions, for example.

Owen

Charles Oliver Nutter-2 wrote:

backspaces wrote:
> Just a quick pointer to a parallel discussion on the Processing board to see
> if I'm understanding Processing's internals correctly:
>   http://tinyurl.com/32hewk
>
> It includes a pointer to the ruby file, and a test without subclassing, just
> calling PApplet directly .. which did not have the exception problem.  This
> leads me to think exception handling works ok in the java classes, but not
> in a subclass of such a class.  I think.  
>
> But no worries for now, let me dig a bit deeper.
>
> Thanks for your patience.

Ok, sounds like good progress so far Owen. Let us know what you find.
That NPE is almost certainly a bug somewhere, but I'm not sure whether
it would be us or processing or your code or what. But the general
structure of what you want to do appears to be working correctly, so
that's great!

- Charlie

[Charles Oliver Nutter-2]

backspaces wrote:
> I can probably figure out a reasonable work around, but for now wanted to
> report what the problem is.  Possibly JRuby only is aware of advertised
> exceptions, for example.

Ok, I think I follow what's going on...but is there something JRuby's
not doing right or handling incorrectly here? I think maybe I missed
that part.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[backspaces]

Charles Oliver Nutter-2 wrote:

backspaces wrote:
> I can probably figure out a reasonable work around, but for now wanted to
> report what the problem is.  Possibly JRuby only is aware of advertised
> exceptions, for example.

Ok, I think I follow what's going on...but is there something JRuby's
not doing right or handling incorrectly here? I think maybe I missed
that part.

- Charlie

Its hard to say, its clearly subtle.

But the situation is this: in JRuby, the NPE is not caught by the Processing catch .. it "escapes" somehow and is caught by the JRuby runtime rather than Processing.  I.e. this piece of code in Applet:
        public AppletContext getAppletContext() {
          return stub.getAppletContext(); # line 181
        }
fails because stub is (apparently) null.  Processing uses this to set the "online" flag:
        try {
          getAppletContext();
          online = true;
        } catch (NullPointerException e) {
          online = false;
        }
but somehow the catch fails and the JRuby runtime catches it:
     java.lang.NullPointerException
        at java.applet.Applet.getAppletContext(Applet.java:181)
The whole trace is here:
    http://backspaces.net/files/stacktrace
.. there are some parts of the trace that I don't understand such as a reference to:
    processing.core.PApplet$Proxy0.__super$getAppletContext
I can't find Proxy0 in the Processing code.  Ditto with other JRuby-specific parts of the trace.

Bottom line is that it works in three other environments (groovy, rhino, jython) and in Java itself.  But in my earlier tests, I showed JRuby does fine with catches of this sort:
  http://www.nabble.com/Re%3A-Java-subclasses-in-Ruby-p9368181.html
so its hard to say its strictly a JRuby problem.

Let me know if I can help, for example bundling up the Processing core.jar and other stuff to make a test you could run on your systems.  In the mean time I'll prod more.

Owen

[Charles Oliver Nutter-2]

backspaces wrote:

>
> Charles Oliver Nutter-2 wrote:
>> backspaces wrote:
>>> I can probably figure out a reasonable work around, but for now wanted to
>>> report what the problem is.  Possibly JRuby only is aware of advertised
>>> exceptions, for example.
>> Ok, I think I follow what's going on...but is there something JRuby's
>> not doing right or handling incorrectly here? I think maybe I missed
>> that part.
>>
>> - Charlie
>>
> Its hard to say, its clearly subtle.
>
> But the situation is this: in JRuby, the NPE is not caught by the Processing
> catch .. it "escapes" somehow and is caught by the JRuby runtime rather than
> Processing.  I.e. this piece of code in Applet:

Ok, I think I might know what's happening here. Inside JRuby, all
exceptions are RuntimeException, since that's what we pass around.
Exceptions in Java code would get wrapped as a NativeException and
propogated out. My guess is that we're probably not unwrapping and
re-throwing the contained Java exception correctly in these cases.

Maybe you could provide a test harness and we'll see if we can figure it
out. The recently-added code for extending concrete/abstract classes is
a little involved, so I haven't wrapped my brain around it, but I think
this theory is probably on the right track.

You could also test it by modifying that catch to catch everything and
see if, in fact, a RuntimeException is coming out.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[backspaces]

Charles Oliver Nutter-2 wrote:

Ok, I think I might know what's happening here. Inside JRuby, all
exceptions are RuntimeException, since that's what we pass around.
Exceptions in Java code would get wrapped as a NativeException and
propogated out. My guess is that we're probably not unwrapping and
re-throwing the contained Java exception correctly in these cases.

Maybe you could provide a test harness and we'll see if we can figure it
out. The recently-added code for extending concrete/abstract classes is
a little involved, so I haven't wrapped my brain around it, but I think
this theory is probably on the right track.

You could also test it by modifying that catch to catch everything and
see if, in fact, a RuntimeException is coming out.

- Charlie

Sorry for being a bit late getting back to this.  I've done my first homework problem .. building a test harness.  But I haven't yet figured out some of the details of rebuilding Processing correctly .. will work on it next.

Here's the test harness:
  http://backspaces.net/files/rtest.tar.gz
It contains
JavaClass.java            core.jar               testframe.rb
JavaSubClass.java       readme               testjava.rb
RandBoxes.java          src                     testprocessing.rb

The core.jar is the Processing jar file, whose source is in the src folder.  The readme is attached to this email and shows how to do a complete set of tests.  Note that the tests are done using 1.4.  I did this because Processing is built in 1.4, but runs fine within 1.5.  Not sure that level of certainty is needed but I'll need it for rebuilding the processing jar if I ever succeed!

Also, I run the jruby processing test twice, once using the RandBoxes subclass of PApplet, one using PApplet directly.  The first gets the NPE, the second does not.  May be telling us something?

Let me know if there is any difficulty downloading and testing.  I'll also press on into rebuilding Processing if I can do so.

Owen

readme:
# Expand http://backspaces.net/files/rtest.tar.gz and cd to rtest dir
# Start clean.  Note, I'll reset CLASSPATH several times to make sure we don't
# inadvertently use unexpected classes.
unset CLASSPATH
rm *.class
ls
# Should look like:
# JavaClass.java          core.jar                testframe.rb
# JavaSubClass.java       readme                  testjava.rb
# RandBoxes.java          src                     testprocessing.rb

# Note src contains all the processing core source files
# PApplet.java            PGraphics3D.java        PPolygon.java
# PConstants.java         PGraphicsJava2D.java    PShape.java
# PFont.java              PImage.java             PTriangle.java
# PGraphics.java          PLine.java
# PGraphics2D.java        PMatrix.java

# Switch to using 1.4 just to be sure.  (Processing is built on 1.4)
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Versions/1.4.2/Home
export PATH=$JAVA_HOME/bin:$PATH
java -version

# Simple test to make sure subclassing and NPE works as expected
javac JavaSubClass.java
java JavaSubClass

# Two preliminary jruby tests, one of frames, that we'll use for our Processing test
jruby testframe.rb # make sure frames OK
# .. and one mimicking the JavaSubClass test
export CLASSPATH=. # so jruby sees JavaSubClass
jruby testjava.rb  # subclassing and NPE work as expected

export CLASSPATH=core.jar
javac RandBoxes.java
export CLASSPATH=core.jar:.
java RandBoxes

export CLASSPATH=core.jar
jruby testprocessing.rb
# change line 10 to use PApplet, not RandBoxes.  No NPE!
# This works because PApplet in not abstract and uses defaults for size etc
sed 's/RandBoxes.new/PApplet.new/' <testprocessing.rb | jruby

# Here's the jar file contents:
jar tvf core.jar
#      0 Mon Nov 13 18:57:26 MST 2006 processing/
#      0 Thu Nov 30 16:29:44 MST 2006 processing/core/
#    591 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$1.class
#    818 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$2.class
#   1140 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$3.class
#    752 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$4.class
#   1064 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$5.class
#    487 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$6.class
#    490 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$7.class
#    535 Thu Nov 30 16:36:18 MST 2006 processing/core/PApplet$8.class
#   1286 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$RegisteredMethods.class
#    945 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$Worker$1.class
#   2025 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$Worker.class
#    499 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet$WorkerVar.class
#  84537 Thu Nov 30 17:33:08 MST 2006 processing/core/PApplet.class
#   5271 Thu Nov 30 17:33:08 MST 2006 processing/core/PConstants.class
#  10927 Thu Nov 30 17:33:08 MST 2006 processing/core/PFont.class
#  37829 Thu Nov 30 17:33:08 MST 2006 processing/core/PGraphics.class
#  16786 Thu Nov 30 17:33:08 MST 2006 processing/core/PGraphics2D.class
#  37665 Thu Nov 30 17:33:08 MST 2006 processing/core/PGraphics3D.class
#   1898 Thu Nov 30 17:33:10 MST 2006 processing/core/PGraphicsJava2D$ImageCache.class
#  16073 Thu Nov 30 17:33:10 MST 2006 processing/core/PGraphicsJava2D.class
#  23073 Thu Nov 30 17:33:08 MST 2006 processing/core/PImage.class
#  11827 Thu Nov 30 17:33:08 MST 2006 processing/core/PLine.class
#  10391 Thu Nov 30 17:33:08 MST 2006 processing/core/PMatrix.class
#   8824 Thu Nov 30 17:33:08 MST 2006 processing/core/PPolygon.class
#   2972 Thu Nov 30 17:33:10 MST 2006 processing/core/PShape.class
#  27970 Thu Nov 30 17:33:10 MST 2006 processing/core/PTriangle.class

[backspaces]

backspaces wrote:

Sorry for being a bit late getting back to this.  I've done my first homework problem .. building a test harness.  But I haven't yet figured out some of the details of rebuilding Processing correctly .. will work on it next.

Here's the test harness:
  http://backspaces.net/files/rtest.tar.gz
It contains
JavaClass.java            core.jar               testframe.rb
JavaSubClass.java       readme               testjava.rb
RandBoxes.java          src                     testprocessing.rb

The core.jar is the Processing jar file, whose source is in the src folder.  The readme is attached to this email and shows how to do a complete set of tests.  Note that the tests are done using 1.4.  I did this because Processing is built in 1.4, but runs fine within 1.5.  Not sure that level of certainty is needed but I'll need it for rebuilding the processing jar if I ever succeed!

Also, I run the jruby processing test twice, once using the RandBoxes subclass of PApplet, one using PApplet directly.  The first gets the NPE, the second does not.  May be telling us something?

Let me know if there is any difficulty downloading and testing.  I'll also press on into rebuilding Processing if I can do so.

Owen

OK, I got the Processing compile to work .. using the 1.4 compiler setup as described in the readme file.  I then made a copy of the source folder and compiled it all.  Set the classpath to use that rather than the core.jar.  Worked fine.  Then replaced the NPE with a vanilla Exception, along with a few print statements.  Got this trace:
Test: Init() called
Test: Try start
Test: Try exception, e=org.jruby.exceptions.RaiseException: Native Exception: 'class java.lang.NullPointerException'; Message: null; StackTrace: java.lang.NullPointerException
        at java.applet.Applet.getAppletContext(Applet.java:181)
        at processing.core.PApplet$Proxy0.__super$getAppletContext(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
The complete dump is http://backspaces.net/files/dump

This looks like your suggested possibility: the NPE was not converted back to a NPE by the JRuby java support libraries, if I understand how it works.

Details of how to do this using the test harness, let me know if there are any problems or if I can do further tests:
cp -Rp src testsrc
javac -deprecation  -d . testsrc/*
ls processing/core
# Edit testsrc/PApplet:
  public void init() {
System.out.println("Test: Init() called");
..
    try {
System.out.println("Test: Try start");
      getAppletContext();
System.out.println("Test: Try OK");
      online = true;
    } catch (Exception e) { // NullPointerException
System.out.println("Test: Try exception, e="+e);
      online = false;
    }

export CLASSPATH=. # should only catch stuff in ./processing/core
java RandBoxes     # make sure java version still works and prints out messages
jruby testprocessing.rb # test ruby

[backspaces]

backspaces wrote:

backspaces wrote:

Sorry for being a bit late getting back to this.  I've done my first homework problem .. building a test harness.  But I haven't yet figured out some of the details of rebuilding Processing correctly .. will work on it next.

Here's the test harness:
  http://backspaces.net/files/rtest.tar.gz
It contains
JavaClass.java            core.jar               testframe.rb
JavaSubClass.java       readme               testjava.rb
RandBoxes.java          src                     testprocessing.rb

The core.jar is the Processing jar file, whose source is in the src folder.  The readme is attached to this email and shows how to do a complete set of tests.  Note that the tests are done using 1.4.  I did this because Processing is built in 1.4, but runs fine within 1.5.  Not sure that level of certainty is needed but I'll need it for rebuilding the processing jar if I ever succeed!

Also, I run the jruby processing test twice, once using the RandBoxes subclass of PApplet, one using PApplet directly.  The first gets the NPE, the second does not.  May be telling us something?

Let me know if there is any difficulty downloading and testing.  I'll also press on into rebuilding Processing if I can do so.

Owen

OK, I got the Processing compile to work .. using the 1.4 compiler setup as described in the readme file.  I then made a copy of the source folder and compiled it all.  Set the classpath to use that rather than the core.jar.  Worked fine.  Then replaced the NPE with a vanilla Exception, along with a few print statements.  Got this trace:
....

I've made all this more simple by rebuilding the rtest harness.  It still lives on:
  http://backspaces.net/files/rtest.tar.gz
but it now contains the rebuilt processing library.  The readme has been expanded, and corrected.  The simplest test is to simply uncompress rtest.tar.gz, cd to rtest, then do:

export CLASSPATH=. # should only catch stuff in ./processing/core
java RandBoxes     # make sure java version still works and prints out messages
jruby testprocessing.rb # test ruby

This results in the desired catch in either java 1.4 or 1.5

Owen

[backspaces]

This does not relate directly to the bug here, but is a puzzling result of trying to work around it.

The bug has been entered in the JIRA database,
  http://jira.codehaus.org/browse/JRUBY-704
and by looking at the java.applet.Applet source, I figured out how to modify the Processing PApplet code so as to not raise an exception.  This is the code fragment used:

//    try {
//System.out.println("Test: Try start");
//      getAppletContext();
//      online = true;
//System.out.println("Test: Try OK");
//    } catch (Exception e) { // NullPointerException
//System.out.println("Test: Try exception, e="+e);
//      online = false;
//    }
        online = isActive();
System.out.println("Test: online="+online);

..which basically calls the java.applet.Applet method isActive which lets us know whether or not we're being run in an applet or just in a pane/application.  isActive is graceful enough to check for a null "stub" before using it!

When I rebuild the processing library, I run into another problem .. apparently a stack overflow.  

If I run jruby in java 1.5, I get:
Test: Init() called
Test: online=false
/Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:464:in `__jcreate!': stack level too deep (SystemStackError)
        from /Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:464

But if I run in java 1.4, I get:
Test: Init() called
Test: online=false
/Users/owen/local/jruby/bin/jruby: line 153: 15476 Trace/BPT trap          "$JAVA_CMD" $JAVA_OPTS $DEBUG -Xmx256m -Xss1024k -da -classpath "$CP" "-Djruby.base=$JRUBY_BASE" "-Djruby.home=$JRUBY_HOME" "-Djruby.lib=$JRUBY_BASE/lib" -Djruby.script=jruby "-Djruby.shell=$JRUBY_SHELL" $EN_US org.jruby.Main $JRUBY_OPTS "$@"

In either case, is there some sort of flag I can set in the jruby launch script to avoid this?

Owen

[Charles Oliver Nutter-2]

backspaces wrote:

> When I rebuild the processing library, I run into another problem ..
> apparently a stack overflow.  
>
> If I run jruby in java 1.5, I get:
> Test: Init() called
> Test: online=false
> /Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:464:in
> `__jcreate!': stack level too deep (SystemStackError)
>         from
> /Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:464
>
> But if I run in java 1.4, I get:
> Test: Init() called
> Test: online=false
> /Users/owen/local/jruby/bin/jruby: line 153: 15476 Trace/BPT trap        
> "$JAVA_CMD" $JAVA_OPTS $DEBUG -Xmx256m -Xss1024k -da -classpath "$CP"
> "-Djruby.base=$JRUBY_BASE" "-Djruby.home=$JRUBY_HOME"
> "-Djruby.lib=$JRUBY_BASE/lib" -Djruby.script=jruby
> "-Djruby.shell=$JRUBY_SHELL" $EN_US org.jruby.Main $JRUBY_OPTS "$@"
>
> In either case, is there some sort of flag I can set in the jruby launch
> script to avoid this?

Try modifying the script to set -Xss1024k to something higher...we've
had our battles with stack depth over time. What platform and Java 5
version are you running on?

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list please visit:

    http://xircles.codehaus.org/manage_email

[backspaces]

Charles Oliver Nutter-2 wrote:

Try modifying the script to set -Xss1024k to something higher...we've
had our battles with stack depth over time. What platform and Java 5
version are you running on?

- Charlie

I tried a larger stack size, -Xss100m, and no go .. added a comment to the JIRA.  The dump looks like:
owen|~/src/rtest[1157]: jruby testprocessing.rb
Stack setting =  -Xss100m
/Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:141 warning: already initialized constant EXIT_ON_CLOSE
Test: Init() called
Test: online=false
/Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:464:in `__jcreate!': stack level too deep (SystemStackError)
        from /Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:464

Going to a higher stack size just gets an eventual out-of-memory failure.

I did a further study by modifying the ruby code to print where the error occurs:
panel.init() # Needed to start Processing thread and initialize data
puts "out of init()"
java.lang.Thread.sleep(1000) while panel.defaultSize && !panel.finished
puts "past sleep()"

frame.pack()
puts "past pack()"
frame.defaultCloseOperation=javax.swing.JFrame::EXIT_ON_CLOSE
puts "past frame.defaultCloseOperation"
frame.resizable = false
puts "past frame.resizable"
frame.visible = true

The last printout is past sleep(), which surprised me a bit, I thought that might be the problem.  Presumably this means the problem is in frame.pack()?!  The printout:
owen|~/src/rtest[1163]: jruby testprocessing.rb
Stack setting =  -Xss100m
/Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:141 warning: already initialized constant EXIT_ON_CLOSE
Test: Init() called
Test: online=false
out of init()
past sleep()
/Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:464:in `__jcreate!': stack level too deep (SystemStackError)
        from /Users/enebo/jruby/releases/jruby-0_9_8/src/builtin/javasupport.rb:464

If I comment out the frame.pack(), it completes the script but still dies trying to bring up the window.  Probably simply delays a lazy evaluation/event inside Swing, likely causing Processing to do something else bad!

In terms of platform: I'm running on Mac OS X, 10.4.8, with a new macbook pro 2.33 ghz core 2 duo, 2GB memory.

When running 1.5, Java sez:
java version "1.5.0_07"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0_07-164)
Java HotSpot(TM) Client VM (build 1.5.0_07-87, mixed mode, sharing)