New Java integration features

[Charles Oliver Nutter-2]

I've added a couple enhancements over the past few weeks I figured I
should list here for discussion.

0. Obviously, there's been a lot of performance work.

1. Closures can be passed to any method with an interface as its last
argument; the closure will be converted to the target interface type.

thread = java.lang.Thread.new { puts 'here' }

2. Interfaces can be implemented using Ruby-cased (underscored) names
for all methods now.

class Foo
   include java.awt.event.ActionListener
   def action_performed(event)
     ...
   end
end

3. Interfaces with bean-like methods can be implemented using attr*.

Java:
public interface Blah {
   public Object getMyX();
   public void setMyX(Object x);

   public boolean isMyY();
   public void setMyY();

   public boolean yumYum();
}

Ruby:
class Foo
   include Blah
   attr_accessor :my_x
   attr_accessor :my_y
   # attr_accessor :myX or :myY also work
end

Of course this implies you can also def my_x=(i) just the same.

4. Interfaces with boolean methods can be implemented using the
question-marked version of the method name.

class Foo
   include Blah
   def isMyY?; end # works
   def is_my_y?; end # works
   def myY?; end # works
   def my_y?; end # works
   def yum_yum?; end # works
   def yumYum?; end # works

*** NOTE ***

This extensive support of Ruby naming may be going too far. I'm really
looking for opinions on this, to see whether it's too much. In general
my primary goal was to allow users to use either straight-up Java names
or various gradations of Ruby names, all the way to the most Ruby-like =
or ? underscored names. So those extreme cases work and all intermediate
cases work. But is it excessive?

Also, it's currently rather undefined what happens if you're mixing
forms. It currently will search for methods in this order if they
haven't been found previously:

1. normal Java name
2. ruby name with underscores
IF PROPERTY {
3. java property name (minus get/set/is)
4. java property name with ? if boolean
5. ruby property name (java prop name with underscores)
6. ruby property name with ? if boolean
}
7. java name with ? if boolean
8. ruby name with ? if boolean

Now there's two implications here.

First, defining as two different forms will have unusual effects, since
redefining a method will try to update what the Java interface
dispatches to. So you could have two different forms with different
bodies, but only the last one wins.

Second, for cases where you want to use method_missing to handle all
calls through that interface, there's going to be up to 8 times as many
method table searches before falling back on method_missing. This may
impact performance of "anonymous interfaces" like the converted closures
above, albeit by an unknown amount (benchmarking help for all this is
requested :)

So I think we should try to talk through some of this stuff.

Specs have been added in
spec/java_integration/interfaces/implementation_spec.rb

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Charles Oliver Nutter-2]

Charles Oliver Nutter wrote:
> This extensive support of Ruby naming may be going too far. I'm really
> looking for opinions on this, to see whether it's too much. In general
> my primary goal was to allow users to use either straight-up Java names
> or various gradations of Ruby names, all the way to the most Ruby-like =
> or ? underscored names. So those extreme cases work and all intermediate
> cases work. But is it excessive?

There is another small issue here: extending concrete classes. Because
in the extension/override case there's usually already existing methods
of that name, the Ruby naming logic I added for interfaces doesn't work.

The way I justify it for the moment is that overriding methods from a
concrete or abstract superclass is about replacing exactly the
same-named method. But of course there's an argument to be made that you
should be able to replace that method with any of its Ruby analogs. So,
what do you think? If I have this code in Java:

public class Foo {
   public Object getBar() { ... }
   public void setBar(Object bar) { ... }
}

Should I be able to do this:

class RubyFoo < Foo
   def bar; my_getBar_logic; end
   def bar=(x); my_setBar_logic; end
end

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Ola Bini-3]

Charles Oliver Nutter wrote:
> I've added a couple enhancements over the past few weeks I figured I
> should list here for discussion.
>
I like these features. Good job.

I had one thing that happened to me a few weeks ago that seemed a bit
strange. I'm writing unit tests for the Java parts of PKCS7, and the
central object have these kinds of methods:

isEnveloped();
getEnveloped();
setEnveloped();

The problem is that isEnveloped() is _not_ the same as getEnveloped().

in my code I used

p7.enveloped?      and expected isEnveloped()
p7.enveloped        and expected getEnveloped()

This used to work until some point 3 weeks ago. At that point
p7.enveloped started returning the same as p7.enveloped?.
Not sure what people think about this, but in this case I think my
intuition that get/set matches regular attributes, and is matches ? is
the right one. I should try to put together some specs on how I would
prefer this to work at some point. In the meantime, thoughts?

Cheers

--
 Ola Bini (http://ola-bini.blogspot.com)
 JRuby Core Developer
 Developer, ThoughtWorks Studios (http://studios.thoughtworks.com)
 Practical JRuby on Rails (http://apress.com/book/view/9781590598818)

 "Yields falsehood when quined" yields falsehood when quined.



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Ola Bini-3]

Charles Oliver Nutter wrote:

> Charles Oliver Nutter wrote:
>> This extensive support of Ruby naming may be going too far. I'm
>> really looking for opinions on this, to see whether it's too much. In
>> general my primary goal was to allow users to use either straight-up
>> Java names or various gradations of Ruby names, all the way to the
>> most Ruby-like = or ? underscored names. So those extreme cases work
>> and all intermediate cases work. But is it excessive?
>
> There is another small issue here: extending concrete classes. Because
> in the extension/override case there's usually already existing
> methods of that name, the Ruby naming logic I added for interfaces
> doesn't work.
>
> The way I justify it for the moment is that overriding methods from a
> concrete or abstract superclass is about replacing exactly the
> same-named method. But of course there's an argument to be made that
> you should be able to replace that method with any of its Ruby
> analogs. So, what do you think? If I have this code in Java:
>
> public class Foo {
>   public Object getBar() { ... }
>   public void setBar(Object bar) { ... }
> }
>
> Should I be able to do this:
>
> class RubyFoo < Foo
>   def bar; my_getBar_logic; end
>   def bar=(x); my_setBar_logic; end
> end
>

The only thing I'm concerned with is consistency between interfaces and
concrete classes. If we're not doing it for classes, then we shouldn't
do it for interfaces either.

Cheers

--
 Ola Bini (http://ola-bini.blogspot.com)
 JRuby Core Developer
 Developer, ThoughtWorks Studios (http://studios.thoughtworks.com)
 Practical JRuby on Rails (http://apress.com/book/view/9781590598818)

 "Yields falsehood when quined" yields falsehood when quined.



---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Thomas E Enebo]

On Fri, Aug 8, 2008 at 4:06 AM, Charles Oliver Nutter
<charles.nutter@...> wrote:

> I've added a couple enhancements over the past few weeks I figured I should
> list here for discussion.
>
> 0. Obviously, there's been a lot of performance work.
>
> 1. Closures can be passed to any method with an interface as its last
> argument; the closure will be converted to the target interface type.
>
> thread = java.lang.Thread.new { puts 'here' }
>
> 2. Interfaces can be implemented using Ruby-cased (underscored) names for
> all methods now.
>
> class Foo
>  include java.awt.event.ActionListener
>  def action_performed(event)
>    ...
>  end
> end
>
> 3. Interfaces with bean-like methods can be implemented using attr*.
>
> Java:
> public interface Blah {
>  public Object getMyX();
>  public void setMyX(Object x);
>
>  public boolean isMyY();
>  public void setMyY();
>
>  public boolean yumYum();
> }
>
> Ruby:
> class Foo
>  include Blah
>  attr_accessor :my_x
>  attr_accessor :my_y
>  # attr_accessor :myX or :myY also work
> end
>
> Of course this implies you can also def my_x=(i) just the same.
>
> 4. Interfaces with boolean methods can be implemented using the
> question-marked version of the method name.
>
> class Foo
>  include Blah
>  def isMyY?; end # works
>  def is_my_y?; end # works
>  def myY?; end # works
>  def my_y?; end # works
>  def yum_yum?; end # works
>  def yumYum?; end # works
>
> *** NOTE ***
>
> This extensive support of Ruby naming may be going too far. I'm really
> looking for opinions on this, to see whether it's too much. In general my
> primary goal was to allow users to use either straight-up Java names or
> various gradations of Ruby names, all the way to the most Ruby-like = or ?
> underscored names. So those extreme cases work and all intermediate cases
> work. But is it excessive?
>
> Also, it's currently rather undefined what happens if you're mixing forms.
> It currently will search for methods in this order if they haven't been
> found previously:
>
> 1. normal Java name
> 2. ruby name with underscores
> IF PROPERTY {
> 3. java property name (minus get/set/is)
> 4. java property name with ? if boolean
> 5. ruby property name (java prop name with underscores)
> 6. ruby property name with ? if boolean
> }
> 7. java name with ? if boolean
> 8. ruby name with ? if boolean

7 and 8 do not seem right to me...

> Now there's two implications here.
>
> First, defining as two different forms will have unusual effects, since
> redefining a method will try to update what the Java interface dispatches
> to. So you could have two different forms with different bodies, but only
> the last one wins.
>
> Second, for cases where you want to use method_missing to handle all calls
> through that interface, there's going to be up to 8 times as many method
> table searches before falling back on method_missing. This may impact
> performance of "anonymous interfaces" like the converted closures above,
> albeit by an unknown amount (benchmarking help for all this is requested :)

removing 7 and 8 will make that up to 6 times.  It is up to 4 if
!boolean.  Up to 2 if not a property.  It seems like 4x will be most
common.  Putting commonest calling convention in front will reduce
what it generally is.

I keep thinking there is a more complex way of doing this which will
end up being a single search.  Like canonicalizing method name then
doing a single lookup.

> So I think we should try to talk through some of this stuff.

-Tom

--
Blog: http://www.bloglines.com/blog/ThomasEEnebo
Email: enebo@... , tom.enebo@...

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Charles Oliver Nutter-2]

Thomas E Enebo wrote:

>> 1. normal Java name
>> 2. ruby name with underscores
>> IF PROPERTY {
>> 3. java property name (minus get/set/is)
>> 4. java property name with ? if boolean
>> 5. ruby property name (java prop name with underscores)
>> 6. ruby property name with ? if boolean
>> }
>> 7. java name with ? if boolean
>> 8. ruby name with ? if boolean
>
> 7 and 8 do not seem right to me...

So I think in misinterpreted some code added in the last release to
support ? forms of properties. The code appeared to try to add such
names for all boolean methods, but I believe the intention was to only
add it for property names. I'm going to boot 7 and 8.

>> Now there's two implications here.
>>
>> First, defining as two different forms will have unusual effects, since
>> redefining a method will try to update what the Java interface dispatches
>> to. So you could have two different forms with different bodies, but only
>> the last one wins.
>>
>> Second, for cases where you want to use method_missing to handle all calls
>> through that interface, there's going to be up to 8 times as many method
>> table searches before falling back on method_missing. This may impact
>> performance of "anonymous interfaces" like the converted closures above,
>> albeit by an unknown amount (benchmarking help for all this is requested :)
>
> removing 7 and 8 will make that up to 6 times.  It is up to 4 if
> !boolean.  Up to 2 if not a property.  It seems like 4x will be most
> common.  Putting commonest calling convention in front will reduce
> what it generally is.
>
> I keep thinking there is a more complex way of doing this which will
> end up being a single search.  Like canonicalizing method name then
> doing a single lookup.

I suppose that's possible. Currently the interface impl stuff works like
this (and this design is largely why it's going to be faster than in 1.1.4):

1. The Java implementation class has a number of static fields that hold
references to the associated Ruby class, the runtime which created the
class, and also a field for each method being implemented.
2. When first calling any of those methods from Java, it will check to
see if the field associated with the method is non-null. If it's null,
it will search for the method using the rules above and populate the field.
3. For methods that change after they've been called, a method_added
hook is provided for the class that resets the associated field to the
new method.

This essentially eliminates dynamic dispatch entirely when calling into
Ruby through a Java interface. But there's that initial hit to look up
the methods. Now we could actively search for those methods when the
class is created and pre-populate the fields, but I'm not doing that
right now.

In order to do what you're saying, we would want to search only for the
canonicalized name, and for all method_added always update that
canonicalized name. For example, the canon name could be the original
Java method name; so when you define my_foo= it would also redefine
setMyFoo.

Hmm...that could work. Anyone see any problem with doing that besides
the side effect that mixing naming styles would always overwrite the
canonical Java name?

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Charles Oliver Nutter-2]

Charles Oliver Nutter wrote:

> Thomas E Enebo wrote:
>>> 7. java name with ? if boolean
>>> 8. ruby name with ? if boolean
>>
>> 7 and 8 do not seem right to me...
>
> So I think in misinterpreted some code added in the last release to
> support ? forms of properties. The code appeared to try to add such
> names for all boolean methods, but I believe the intention was to only
> add it for property names. I'm going to boot 7 and 8.

Actually it seems that the code we have at present *does* append a ? to
any boolean-returning methods. So for example, in the JI spec fixture
BeanLikeInterfaceHandler, the boolean friendly() method can be called as
  friendly?. If that feature stays (and it probably should since it was
shipped with 1.1.3), I think you also should be able to implement
friendly? for a boolean friendly() interface method. Yes?

>> I keep thinking there is a more complex way of doing this which will
>> end up being a single search.  Like canonicalizing method name then
>> doing a single lookup.
...

> This essentially eliminates dynamic dispatch entirely when calling into
> Ruby through a Java interface. But there's that initial hit to look up
> the methods. Now we could actively search for those methods when the
> class is created and pre-populate the fields, but I'm not doing that
> right now.
>
> In order to do what you're saying, we would want to search only for the
> canonicalized name, and for all method_added always update that
> canonicalized name. For example, the canon name could be the original
> Java method name; so when you define my_foo= it would also redefine
> setMyFoo.

Ok, it also occurred to me why this isn't quite so simple. The lookup
done for each method needs to be there in the first place because we
defer creating the Java-based interface impl until the class is first
constructed. We also don't add the method_added hook until that point,
because there's no Java class yet for it to update. So at the time the
8x lookup needs to happen, it's very possible there's only one of those
styles there.

So basically we need to do the search for all 8 styles at least once per
method. That search could happen at first call, likely never happening
again since the method_added hook will keep it updated. Or that search
could happen at class creation time, with up to 8 lookups per each
interface method being implemented. Either way, you're paying the cost.

One thing I did fix on trunk is that the search wasn't actually ordered
1 thru 8 because I was stuffing the names into a HashSet during class
creation. That is now a LinkedHashSet, so the order is predictable.

I'm open for other ideas on the search issue too. But I'm skeptical as
to whether it's going to be a real hit. It will only ever happen once
per method per new interface impl class.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Thomas E Enebo]

On Fri, Aug 8, 2008 at 12:59 PM, Charles Oliver Nutter
<charles.nutter@...> wrote:

> Charles Oliver Nutter wrote:
>>
>> Thomas E Enebo wrote:
>>>>
>>>> 7. java name with ? if boolean
>>>> 8. ruby name with ? if boolean
>>>
>>> 7 and 8 do not seem right to me...
>>
>> So I think in misinterpreted some code added in the last release to
>> support ? forms of properties. The code appeared to try to add such names
>> for all boolean methods, but I believe the intention was to only add it for
>> property names. I'm going to boot 7 and 8.
>
> Actually it seems that the code we have at present *does* append a ? to any
> boolean-returning methods. So for example, in the JI spec fixture
> BeanLikeInterfaceHandler, the boolean friendly() method can be called as
>  friendly?. If that feature stays (and it probably should since it was
> shipped with 1.1.3), I think you also should be able to implement friendly?
> for a boolean friendly() interface method. Yes?

Yeah in that case it does seem to make sense...how about modifying
things like so:

1. normal Java name
2. ruby name with underscores
IF PROPERTY {
3. java property name (minus get/set/is)
4. java property name with ? if boolean
5. ruby property name (java prop name with underscores)
6. ruby property name with ? if boolean
} ELSE {
7. java name with ? if boolean
8. ruby name with ? if boolean
}

>
>>> I keep thinking there is a more complex way of doing this which will
>>> end up being a single search.  Like canonicalizing method name then
>>> doing a single lookup.
>
> ...
>>
>> This essentially eliminates dynamic dispatch entirely when calling into
>> Ruby through a Java interface. But there's that initial hit to look up the
>> methods. Now we could actively search for those methods when the class is
>> created and pre-populate the fields, but I'm not doing that right now.
>>
>> In order to do what you're saying, we would want to search only for the
>> canonicalized name, and for all method_added always update that
>> canonicalized name. For example, the canon name could be the original Java
>> method name; so when you define my_foo= it would also redefine setMyFoo.
>
> Ok, it also occurred to me why this isn't quite so simple. The lookup done
> for each method needs to be there in the first place because we defer
> creating the Java-based interface impl until the class is first constructed.
> We also don't add the method_added hook until that point, because there's no
> Java class yet for it to update. So at the time the 8x lookup needs to
> happen, it's very possible there's only one of those styles there.
>
> So basically we need to do the search for all 8 styles at least once per
> method. That search could happen at first call, likely never happening again
> since the method_added hook will keep it updated. Or that search could
> happen at class creation time, with up to 8 lookups per each interface
> method being implemented. Either way, you're paying the cost.
>
> One thing I did fix on trunk is that the search wasn't actually ordered 1
> thru 8 because I was stuffing the names into a HashSet during class
> creation. That is now a LinkedHashSet, so the order is predictable.
>
> I'm open for other ideas on the search issue too. But I'm skeptical as to
> whether it's going to be a real hit. It will only ever happen once per
> method per new interface impl class.

Right now in Rails whenever we call a base method we do quite a number
of hash misses and one hash hit.  So in many regards we can probably
consider the cost by looking a ruby dispatch cache misses.

-Tom



--
Blog: http://www.bloglines.com/blog/ThomasEEnebo
Email: enebo@... , tom.enebo@...

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[david_koontz]

First off, let me say, "thank you, thank you, thank you".  All of this  
is wonderfully welcome improvements for us at Happy Camper.

On Aug 8, 2008, at 2:06 AM, Charles Oliver Nutter wrote:

> This extensive support of Ruby naming may be going too far. I'm  
> really looking for opinions on this, to see whether it's too much.  
> In general my primary goal was to allow users to use either straight-
> up Java names or various gradations of Ruby names, all the way to  
> the most Ruby-like = or ? underscored names. So those extreme cases  
> work and all intermediate cases work. But is it excessive?

In all the time I've been doing JI work, I've never wanted to write a  
method name that wasn't the canonical Ruby style name.  So, for me and  
the people I've been working with, I don't think any intermediary name  
is really wanted or needed.  Ideally we would use the Ruby name in  
every case  and just pretend that setFoo on the Java class/interface  
is really just written as foo=.  So you would get my vote on making  
concrete classes work like interfaces with regards to overriding a  
method.  I do agree that this presents some pitfalls if you implement  
both a foo= and a setFoo method, although I would say you can do lots  
of stupid things in Ruby and this would be no different.

David Koontz


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Charles Oliver Nutter-2]

Thomas E Enebo wrote:

> Yeah in that case it does seem to make sense...how about modifying
> things like so:
>
> 1. normal Java name
> 2. ruby name with underscores
> IF PROPERTY {
> 3. java property name (minus get/set/is)
> 4. java property name with ? if boolean
> 5. ruby property name (java prop name with underscores)
> 6. ruby property name with ? if boolean
> } ELSE {
> 7. java name with ? if boolean
> 8. ruby name with ? if boolean
> }

Yeah, I like this. I can't imagine people are going to want both the
property names, original names, AND question-marked original names. I
will also formalize the same set of rules in JavaClass's method-aliasing
logic.

>> I'm open for other ideas on the search issue too. But I'm skeptical as to
>> whether it's going to be a real hit. It will only ever happen once per
>> method per new interface impl class.
>
> Right now in Rails whenever we call a base method we do quite a number
> of hash misses and one hash hit.  So in many regards we can probably
> consider the cost by looking a ruby dispatch cache misses.

True, and I keep forgetting that this will be a continual perf hit for
interfaces implemented with method_missing. Grr. I guess I just need to
come up with some benchmarks to measure it.

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Charles Oliver Nutter-2]

David Koontz wrote:

> In all the time I've been doing JI work, I've never wanted to write a
> method name that wasn't the canonical Ruby style name.  So, for me and
> the people I've been working with, I don't think any intermediary name
> is really wanted or needed.  Ideally we would use the Ruby name in every
> case  and just pretend that setFoo on the Java class/interface is really
> just written as foo=.  So you would get my vote on making concrete
> classes work like interfaces with regards to overriding a method.  I do
> agree that this presents some pitfalls if you implement both a foo= and
> a setFoo method, although I would say you can do lots of stupid things
> in Ruby and this would be no different.

So here's the methods I see as being intermediate:

given a Java method name isMyFoo that returns boolean...

1 isMyFoo is obviously needed
2 is_my_foo is an intermediate or partial Ruby name
3 myFoo is an intermediate
4 my_foo is an intermediate
5 myFoo? is an intermediate
6 my_foo? is ideal Ruby

The boolean case is a little harder than a non-boolean case, since the
attr* methods don't create question-marked versions. So I think in the
spirit of supporting attr* methods, we need to keep 4 for sure in all
cases. The others...well it's a really tough call for me. It feels like
if we removed 3 we'd have to remove 4. And it seems like 2 is necessary
as an analog to 1. So only 3 and 4 are candidates to be removed as far
as I can see, and that isn't really saving us a whole lot, is it?

- Charlie

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email