[scala] overloading vs implicit resolution

[Vladimir Kirichenko-2]

object Test extends Application {
        def m(o: Object) = println("object!")

        def m(s: String) = println("string")

        case class A

        implicit def a2s(a:A):String = a.toString

        m(A())
}

results to: object!

So implicit conversion is not applied here. Is this supposed to be this way?

> scala -version
Scala code runner version 2.7.7.final -- Copyright 2002-2009, LAMP/EPFL
--
Best Regards,
Vladimir Kirichenko

[Jesper Nordenberg]

Vladimir Kirichenko wrote:

> object Test extends Application {
> def m(o: Object) = println("object!")
>
> def m(s: String) = println("string")
>
> case class A
>
> implicit def a2s(a:A):String = a.toString
>
> m(A())
> }
>
> results to: object!
>
> So implicit conversion is not applied here. Is this supposed to be this way?

Yes, there's no reason for the compiler to apply an implicit conversions
here.

/Jesper Nordenberg

[Vladimir Kirichenko-2]

> Yes, there's no reason for the compiler to apply an implicit conversions
> here.

May be availability of implicit conversion to more specific type is a
good enough reason for compiler to apply it? It causes real pain in
the ass with integration with java libraries with 'object' methods.


--
Best Regards,
Vladimir Kirichenko

[Jesper Nordenberg]

Vladimir Kirichenko wrote:
>> Yes, there's no reason for the compiler to apply an implicit conversions
>> here.
>
> May be availability of implicit conversion to more specific type is a
> good enough reason for compiler to apply it?

No, implicit conversions are only applied if the expression isn't valid
using the original type.

/Jesper Nordenberg

[Vladimir Kirichenko-2]

Jesper Nordenberg wrote:
> Vladimir Kirichenko wrote:
>>> Yes, there's no reason for the compiler to apply an implicit conversions
>>> here.
>>
>> May be availability of implicit conversion to more specific type is a
>> good enough reason for compiler to apply it?
>
> No, implicit conversions are only applied if the expression isn't valid
> using the original type.

I understand this. Question is why? One of the use of implicit
conversions is type adaptation, and this feature have no use if there is
Any/Object overload in the hierarchy. BTW it causes some "dynamic
language" problem - add this kind of overload to the base hierarchy -
and the existing code still "statically ok" but does not work anymore.
So predictability of implicit application becomes in question. Is there
any reason (other than "this is how it is now") to ignore implicit
conversions with more accurate types in favor of more common types?


--
Best Regards,
Vladimir Kirichenko

[Rex Kerr-2]

In general, you don't want to do an expensive implicit conversion from one class to another instead of a completely free use of a subclass in place of a superclass.  So allowing implicit conversions to work in this case is bad:
  class MyCollection extends BasicCollection { ... }
  class WeirdHighlyDerivedCollection extends BasicCollection { ... }
  implicit def mycol2weird(m:MyCollection):WeirdHighlyDerivedCollection { ... }
  def foo(c:BasicCollection) { ... }
  def foo(c:WeirdHighlyDerivedCollection) { ... }
  val a = new MyCollection
  foo(a)  // Do you really, really want to make the weird collection?

However, there is some incentive to make java.lang.Object a special case to make interoperability with Java code easier.

On the other hand, you can request the type that you want something to be, and the implicit conversion will do it if it can:
  class A
  class B extends A
  class C extends A
  implicit def b2c(b:B) = { println("Turning B into C"); new C }
  val b = new B
  object O {
    def foo(a:A) { println("A") }
    def foo(c:C) { println("C") }
  }
  O.foo(b)  // Prints "A"
  O.foo((b:C))  // Prints "Turning B into C" , "C"

So you don't actually need to remember what the implicit conversion function is called, just the non-Object type that you want.  (This is handy, if depressingly verbose, for getting the right integer conversion into System.out.printf, if you use it.)

  --Rex

On Thu, Oct 29, 2009 at 5:57 PM, Vladimir Kirichenko <vladimir.kirichenko@...> wrote:

Jesper Nordenberg wrote:
> Vladimir Kirichenko wrote:
>>> Yes, there's no reason for the compiler to apply an implicit conversions
>>> here.
>>
>> May be availability of implicit conversion to more specific type is a
>> good enough reason for compiler to apply it?
>
> No, implicit conversions are only applied if the expression isn't valid
> using the original type.

I understand this. Question is why? One of the use of implicit
conversions is type adaptation, and this feature have no use if there is
Any/Object overload in the hierarchy. BTW it causes some "dynamic
language" problem - add this kind of overload to the base hierarchy -
and the existing code still "statically ok" but does not work anymore.
So predictability of implicit application becomes in question. Is there
any reason (other than "this is how it is now") to ignore implicit
conversions with more accurate types in favor of more common types?


--
Best Regards,
Vladimir Kirichenko

[Vladimir Kirichenko-2]

Rex Kerr wrote:
> In general, you don't want to do an expensive implicit conversion from one
> class to another instead of a completely free use of a subclass in place of
> a superclass.  

Your examples both related to inheritance. I didn't mean inheritance in
the first place, I meant Object/non object case, when classes are
totally unrelated. This situation occurred in integration with java
library with two overloaded methods one with object parameter and the
second one with... let say parameter of type Matcher. Implicit
conversion suppose to convert more convenient scala object to that
"Matcher".

So inheritance does not matter here, the real problem is with the
methods with parameters of type java.lang.Object - it's more an
integration issue - there is no way to use implicit conversion in this
case...meanwhile compilation is ok. The explicit import of implicit
conversion just doesn't matter :( And this kind of behavior is not the
least surprise.

--
Best Regards,
Vladimir Kirichenko

[Paul Phillips-3]

On Fri, Oct 30, 2009 at 01:12:31AM +0200, Vladimir Kirichenko wrote:
> The explicit import of implicit conversion just doesn't matter :( And
> this kind of behavior is not the least surprise.

I consider what you propose to be way more surprising.

Here is the surprise function: if it type checks, don't do any implicit
search.  It type checks, so no search.  If it's surprising to you that
no implicit conversion takes place in this case, which clearly type
checks, then you are applying an inadequate mental model.

I don't even know how I'd go about forming an intuition about what was
going to happen if implicits might kick in and start converting things
even when I'm making a perfectly valid well-typed method call.  I guess
having no expectation whatsoever about what is going to happen next is
one way of never being surprised...

--
Paul Phillips      | We must respect the other fellow's religion, but only
In Theory          | in the sense and to the extent that we respect his
Empiricist         | theory that his wife is beautiful and his children smart.
ha! spill, pupil   |     -- H. L. Mencken

[Vladimir Kirichenko-2]

Paul Phillips wrote:

> On Fri, Oct 30, 2009 at 01:12:31AM +0200, Vladimir Kirichenko wrote:
>> The explicit import of implicit conversion just doesn't matter :( And
>> this kind of behavior is not the least surprise.
>
> I consider what you propose to be way more surprising.
>
> Here is the surprise function: if it type checks, don't do any implicit
> search.  It type checks, so no search.  If it's surprising to you that
> no implicit conversion takes place in this case, which clearly type
> checks, then you are applying an inadequate mental model.

Surprising moment is that my _explicit_ _import_ declaration is silently
ignored and ClassCastException is what I get in "perfectly valid
well-typed method call". There should be a least warning that there is
implicit conversion (that have more specific types) that is ambiguous
with real types "hey dude what did you mean importing this kind of stuff"?



--
Best Regards,
Vladimir Kirichenko

[Tony Morris-4]

It is impractical to use an implicit simply "because you can" when other
alternatives type-check. Better is to use an implicit "because you can't
(type check)."

What ClassCastException do you get for a perfect valid well-typed method
call?

Vladimir Kirichenko wrote:

> Paul Phillips wrote:
>  
>> On Fri, Oct 30, 2009 at 01:12:31AM +0200, Vladimir Kirichenko wrote:
>>    
>>> The explicit import of implicit conversion just doesn't matter :( And
>>> this kind of behavior is not the least surprise.
>>>      
>> I consider what you propose to be way more surprising.
>>
>> Here is the surprise function: if it type checks, don't do any implicit
>> search.  It type checks, so no search.  If it's surprising to you that
>> no implicit conversion takes place in this case, which clearly type
>> checks, then you are applying an inadequate mental model.
>>    
>
> Surprising moment is that my _explicit_ _import_ declaration is silently
> ignored and ClassCastException is what I get in "perfectly valid
> well-typed method call". There should be a least warning that there is
> implicit conversion (that have more specific types) that is ambiguous
> with real types "hey dude what did you mean importing this kind of stuff"?
>
>
>
>  

--
Tony Morris
http://tmorris.net/

[Jesper Nordenberg]

Vladimir Kirichenko wrote:
> Surprising moment is that my _explicit_ _import_ declaration is silently
> ignored and ClassCastException is what I get in "perfectly valid
> well-typed method call". There should be a least warning that there is
> implicit conversion (that have more specific types) that is ambiguous
> with real types "hey dude what did you mean importing this kind of stuff"?

There is no ambiguity. Having a warning that says that if an implicit
conversion was applied the call would have been made to another method
would be pretty darn annoying. The only reasonable warning the compiler
could give is that your import is unused, but that wouldn't help much in
general as most implicits are imported using wildcard syntax.

/Jesper Nordenberg

[Paul Phillips-3]

On Fri, Oct 30, 2009 at 01:32:22AM +0200, Vladimir Kirichenko wrote:
> Surprising moment is that my _explicit_ _import_ declaration is
> silently ignored and ClassCastException is what I get in "perfectly
> valid well-typed method call".

Wait, what? You did not share any code involving casts.  There weren't
any imports either, so that's a pretty mysterious rebuttal.  Here is the
code you sent, in case it's unavailable to you:

object Test extends Application {
        def m(o: Object) = println("object!")

        def m(s: String) = println("string")

        case class A

        implicit def a2s(a:A):String = a.toString

        m(A())
}                                                                              

> There should be a least warning that there is implicit conversion
> (that have more specific types) that is ambiguous with real types "hey
> dude what did you mean importing this kind of stuff"?

I'm sure your patch implementing this improbable heuristic will be given
due consideration.

--
Paul Phillips      | Simplicity and elegance are unpopular because
Future Perfect     | they require hard work and discipline to achieve
Empiricist         | and education to be appreciated.
slap pi uphill!    |     -- Dijkstra

[Vladimir Kirichenko-2]

Tony Morris wrote:
> It is impractical to use an implicit simply "because you can" when other
> alternatives type-check. Better is to use an implicit "because you can't
> (type check)."

Why is it impractical? That one who writes 'import
some.kid.of.Implicits._' means something, doesn't he?

> What ClassCastException do you get for a perfect valid well-typed method
> call?

Th real example related to overloading of method 'contains' of
implementation of java.util.Collection with more specific matching
parameters. The one in j.u.Collection has signature Object -> boolean.
The overloaded one accepts matching strategy type. In the scala code it
could be expressed better than with the use of plain java strategy
object. The idea was to write implicit conversion and have fun. The
reality bitten in the ass with silently not working code because of
basic Object -> boolean implementation.

--
Best Regards,
Vladimir Kirichenko

[Paul Phillips-3]

On Fri, Oct 30, 2009 at 01:50:59AM +0200, Vladimir Kirichenko wrote:
> The reality bitten in the ass with silently not working code because
> of basic Object -> boolean implementation.

"Behaving as specified" is a funny definition of not working.  Do you
blame the compiler for syntax errors? You chose a design which
fundamentally does not work.  Reality does bite, but that wasn't reality
on this occasion, it was his distant cousin self-infliction.

--
Paul Phillips      | It's not enough to bash in heads - you've got to
Protagonist        | bash in minds.
Empiricist         |     -- Capt Hammer
pull his pi pal!   |----------* http://www.improving.org/paulp/ *----------

[Tony Morris-4]

Vladimir Kirichenko wrote:

> Tony Morris wrote:
>  
>> It is impractical to use an implicit simply "because you can" when other
>> alternatives type-check. Better is to use an implicit "because you can't
>> (type check)."
>>    
>
> Why is it impractical? That one who writes 'import
> some.kid.of.Implicits._' means something, doesn't he?
>  

I'm hesitant to explain why it is impractical. Yes of course writing
imports means something, but this is a side issue to the one at hand.

>  
>> What ClassCastException do you get for a perfect valid well-typed method
>> call?
>>    
>
> Th real example related to overloading of method 'contains' of
> implementation of java.util.Collection with more specific matching
> parameters. The one in j.u.Collection has signature Object -> boolean.
> The overloaded one accepts matching strategy type. In the scala code it
> could be expressed better than with the use of plain java strategy
> object. The idea was to write implicit conversion and have fun. The
> reality bitten in the ass with silently not working code because of
> basic Object -> boolean implementation.
>
>  

I've not seen such an example, but I will conjecture that "not working"
is an incorrect description.

--
Tony Morris
http://tmorris.net/

[Vladimir Kirichenko-2]

Paul Phillips wrote:
> On Fri, Oct 30, 2009 at 01:50:59AM +0200, Vladimir Kirichenko wrote:
>> The reality bitten in the ass with silently not working code because
>> of basic Object -> boolean implementation.
>
> "Behaving as specified" is a funny definition of not working.

Question was "why" this is specified this way, and maybe there will be
profit specifying this other way?


> You chose a design which
> fundamentally does not work.  

I'm sorry for j.u.Collection interface and it's type parameters those
allows use of any kind of crap in it's remove, containsAll, retainAll,
contains, containsAll methods. I'll be more careful some other days:)




--
Best Regards,
Vladimir Kirichenko

[Dave Griffith]

Vladimir Kirichenko-2 wrote:

Why is it impractical? That one who writes 'import
some.kid.of.Implicits._' means something, doesn't he?

You would think so, but in the absence of IDEs that automate dead import removal, this turns out not to be the case.  The experience from pre-IDE Java was that production code quickly evolves to the point where most of the import statements in any given file are "dead", as code is added and deleted (or worse cut-and-pasted) but corresponding imports are not pruned.  

--Dave Griffith

[Vladimir Kirichenko-2]

The final one:

case class X

//its a very far away library that lies almost there where star wars
//suppose to happen

trait InAFarFarAwayLibrary {
        def foo( a: AnyRef ) = println( "oops!" )
}

//many different classes and objects located in different parts of the
//project and maybe in different libraries.

object A extends InAFarFarAwayLibrary {
        def foo( a: String ) = println( "yeah baby" )
}

object B {
        def foo( a: String ) = println( "yeah baby" )
}

object C {
        def foo( a: String ) = println( "yeah baby" )
}

class D {
        def foo( a: String ) = println( "yeah baby" )
}

//this is magical one - this one at bright side
object I {
        implicit def x2s( x: X ) = x.toString
}

//some usage of these classes that appeared at the same place - where
//the final battle happened

object Test extends Application {
        import I._ //haga, notice this one

        val x = X()
        A.foo( x )
        B.foo( x )
        C.foo( x )
        val d = new D
        d.foo( x )
        val z = new D with InAFarFarAwayLibrary
        z.foo( x )
}

================
oops!
yeah baby
yeah baby
yeah baby
oops!


if this is perfect way to go, and nothing can be improved here, so what
can I say.

--
Best Regards,
Vladimir Kirichenko

[Jesper Nordenberg]

Looks fine to me. Instead of complaining about the current behavior, why
don't you write down the rules on how you want overloading and implicit
conversions to work and exactly when the compiler should issue a
warning. Then we have something to discuss.

/Jesper Nordenberg

Vladimir Kirichenko wrote:

> The final one:
>
> case class X
>
> //its a very far away library that lies almost there where star wars
> //suppose to happen
>
> trait InAFarFarAwayLibrary {
> def foo( a: AnyRef ) = println( "oops!" )
> }
>
> //many different classes and objects located in different parts of the
> //project and maybe in different libraries.
>
> object A extends InAFarFarAwayLibrary {
> def foo( a: String ) = println( "yeah baby" )
> }
>
> object B {
> def foo( a: String ) = println( "yeah baby" )
> }
>
> object C {
> def foo( a: String ) = println( "yeah baby" )
> }
>
> class D {
> def foo( a: String ) = println( "yeah baby" )
> }
>
> //this is magical one - this one at bright side
> object I {
> implicit def x2s( x: X ) = x.toString
> }
>
> //some usage of these classes that appeared at the same place - where
> //the final battle happened
>
> object Test extends Application {
> import I._ //haga, notice this one
>
> val x = X()
> A.foo( x )
> B.foo( x )
> C.foo( x )
> val d = new D
> d.foo( x )
> val z = new D with InAFarFarAwayLibrary
> z.foo( x )
> }
>
> ================
> oops!
> yeah baby
> yeah baby
> yeah baby
> oops!
>
>
> if this is perfect way to go, and nothing can be improved here, so what
> can I say.
>

[Vladimir Kirichenko-2]

Jesper Nordenberg wrote:
> Looks fine to me. Instead of complaining about the current behavior, why
> don't you write down the rules on how you want overloading and implicit
> conversions to work and exactly when the compiler should issue a
> warning. Then we have something to discuss.

For compiler: implicit conversions in scope should be used in type
checks. Implicit conversion should be applied where it proposes
conversion to more specific type. Actually it's consistent with regular
resolution of the overloaded methods, just one addition: overloaded
method should be chosen with check of available implicit conversions.

For warning: flash a warning if there is an implicit conversion in scope
that could cause of different overloaded method with more specific type
to be chosen if it was applied. It's based on assumption that if someone
imports implicit conversion he expects it to be applied, and if it is
not applied in a presence of "potentially matching overloading" - it's a
potential bug in there.


As for me the proposal for compiler would be perfect if it will not
cause some greedy behavior in implicit checks that will make whole idea
unusable.

Warning could be usable in a presence of gazillion methods and imports
from different classes and libraries one of those have "dangerous"
AnyRef/j.l.Object overload that could cause code to be compiled but not
working as expected. There is no need to go far to find example -
java.util.* is a good one with a lots of j.l.Object parameters.

--
Best Regards,
Vladimir Kirichenko