[scala] The Language Designer's Dilemma

[Sean McDirmid]

In PL design, everyone seems to have different ideas about what the best, even perfect, design is. Fundamentalists can stick to a stone tablet of principles that they won't waver from even if they clearly won't work, while pragmatists can be as shifty as successful politicians on using whatever principle is convenient or popular. I think with experience, many designers drift toward the latter camp, especially if they want their languages to be used. And then there are those that remain fundamentalists because this is also an interesting and useful path.

I'm seeing a lot of debate recently about how decisions made in the Scala library are crazy, ill thought out, etc. However, the flip side of this coin is that the criticism doesn't consider the consequences of their "obvious" change. Example:

Should "cba" == "abc".reverse be true? Of course it should be! What was the Scala team smoking when they decided RichString.reverse should return anything other than String? Of course it should be changed to return String so equality works right! Oh wait...reverse is defined in Seq to return a Seq, and the LUB of String and Seq is...AnyRef. So if reverse in RichString returns a String, then either reverse has to be removed from Seq, or it has to be defined to return AnyRef, or some compiler magic has to make String a Seq (in much the same way that arrays are) introducing more boxing madness to Scala (and equality would still be broken!). Given these concerns, changing reverse in RichString to return a String is no longer obviously the right thing to do! So the Scala team wasn't so stoned when they decided to have reverse return RichString rather than String. When you actually sit down and design/code these things, you realize how many bazillion trade offs there are to make.

I've come to this conclusion: there are no good language/library design decisions, only bad design decisions. Whatever decision a language designer makes will necessarily be a bad one, because a trade off will be made that is obviously horrible to someone.  Some design decisions may be worse than others, but they are still bad. Accept that. And why would your bad design decision be any better or worse than someone else's? Self righteousness will only serve to aggravate others who don't hold your same principals. Anyone who suggest that they have found the perfect design solution is probably selling snake oil.

All languages have their own set of bad design decisions that define them. C++ is religiously backwards compatible with C and its pervious versions, and as a result is the butt of many PL jokes. Is Bjarne an idiot for designing such a bad language? Well, C++ is hugely successful because many people and projects still prefer from C++'s bad decisions vs. say Java's. Bjarne is probably a genius, even if I personally don't like programming C++. Haskell is more idealistic and pure, not as popular, but is well designed and serves a purpose. Likewise, Ruby will do things behind your back, implicitly coerce things, re-direct your method calls...there is so much magic going on! Is Ruby a bad language? Of course (because they all are), but lots of people feel like this magic and feel like it servers a purpose. C# and Java on the other hand are more verbose and try not to do anything implicitly. Another bad decision, but many people benefit from that also. I'm actually growing to like C# at Microsoft; e.g., extension methods are less magical than implicits, but get the job done. The C# team were obviously not on meth when they designed C#, no matter how many times the thought crossed my mind when I was learning how to use C#!

I like Scala's bad decisions more than the other languages'. In the library, a few of those bad decisions are mine (e.g., projections), and I've made many more bad decisions in the plugin (to quote comic shop guy: "the worst Scala plugin ever"). I've also disagreed with some of Martin's decisions (e.g., strong specification of protected that seems to reduce applicability, strictness in mutable collections where it seems to create a contradiction), but in the end his bad decisions often turn out to be less worse than mine (and of course, he out-ranks me :) ). Working with Martin was immensely educational and life changing.

I was at first weary of implicits, but see that they serve a purpose and shouldn't be discounted as evil, well, no, they are evil but that means you shouldn't use them: they sere a purpose; e.g., through implicits and other constructs can do stuff with Scala that people thought were only possible in dynamic languages. Scala has the opportunity to can draw ultra-pragmatic programmers from Java who might otherwise go to Ruby or Python. Scala is also principled with a powerful type system that can draw in fundamentalists from languages like Haskell who are looking for a language their bosses will actually accept. And of course, many people lie in between. Essentially, Martin has created a "big tent" language that will attract users from all over the place. Although a big tent good thing, it leads to some tension that I'm feeling already. There are already many debates about implicits, Java compatibility, breaking binary compatibility, etc...everyone has a strong opinion, referring to other opinions as crazy or in one case "immature." This can easily get blood boiling, especially in the more passionate among us. I guess it is bound to happen in any language community, but more so in Scala given its big tent focus. 

[Jesper Nordenberg]

Sean McDirmid wrote:
> Should "cba" == "abc".reverse be true? Of course it should be!

Agreed.

 > What was
> the Scala team smoking when they decided RichString.reverse should
> return anything other than String? Of course it should be changed to
> return String so equality works right! Oh wait...reverse is defined in
> Seq to return a Seq, and the LUB of String and Seq is...AnyRef. So if
> reverse in RichString returns a String, then either reverse has to be
> removed from Seq, or it has to be defined to return AnyRef, or some
> compiler magic has to make String a Seq (in much the same way that
> arrays are) introducing more boxing madness to Scala (and equality would
> still be broken!).

Actually not much compiler magic is needed, just make string literals be
of class scala.String (which would be a Seq) and provide implicit
conversions to and from java.lang.String.

> Essentially, Martin has created a "big tent" language
> that will attract users from all over the place. Although a big tent
> good thing, it leads to some tension that I'm feeling already. There are
> already many debates about implicits, Java compatibility, breaking
> binary compatibility, etc...everyone has a strong opinion, referring to
> other opinions as crazy or in one case "immature." This can easily get
> blood boiling, especially in the more passionate among us. I guess it is
> bound to happen in any language community, but more so in Scala given
> its big tent focus.

I don't think heated debates is a bad thing, it shows that there is a
lot of people who care about Scala. Scala is one of the best programming
languages around, and the Java interoperability is the killer feature.
The language is a bit too complex and there are some sub-optimal
constructs, but I understand why Martin put them in the language (most
of them are related to Java interoperability).

As for the standard library, it's not as important to get this right the
first time as the language, and with any non-trivial library I would say
it's almost impossible to get right the first time. A good module and
versioning system can help a lot here with regard to preserving or
breaking backwards compatibility. As long as the users knows when
backwards compatibility is broken AND have the option of either
migrating to the new version or continue using the old (still
maintained) version. This is the inevitable path of all libraries, you
might as well face it now. It would be unfortunate if the Scala library
turned into another JRE/JDK mess.

/Jesper Nordenberg

[Sean McDirmid]

Is it really that easy? Strings in Java are almost primitives, in spite of their java.lang.String exterior. We would still have issues with AnyRef strings coming in, equality wouldn't work where the user might expect it to, and it might be too slow. This is really related to the scala.Array problem: it breaks down in many cases where the compiler cannot detect where boxing is needed.

But if you think scala.String is the right way, you should explore it in more detail, see what the drawbacks are in addition to the benefits. If you can't find drawbacks, then you haven't been thorough enough :) (Never trust any analysis with too many + and not enough -).

Sean

On Sat, May 31, 2008 at 4:13 PM, Jesper Nordenberg <megagurka@...> wrote:

> What was

the Scala team smoking when they decided RichString.reverse should return anything other than String? Of course it should be changed to return String so equality works right! Oh wait...reverse is defined in Seq to return a Seq, and the LUB of String and Seq is...AnyRef. So if reverse in RichString returns a String, then either reverse has to be removed from Seq, or it has to be defined to return AnyRef, or some compiler magic has to make String a Seq (in much the same way that arrays are) introducing more boxing madness to Scala (and equality would still be broken!).

Actually not much compiler magic is needed, just make string literals be of class scala.String (which would be a Seq) and provide implicit conversions to and from java.lang.String.

[tmorris]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

"abc" == "cba".reverse may be true or false, with benefit/disadvantage
either way. How much effort does it take me, a user, to choose one from
the other? I think this is a more important question to answer.

I could solve it with a trait and implicit definitions, but then I'd be
calling the function as: ==("abc", "cba".reverse), which many would
argue is a syntactic cost. Whichever way you turn, you incur a problem
with inflexibility. I suggest that this is a more significant problem.

By the way, isn't this typical scala-debate material?

Tony Morris
http://tmorris.net/

Real-world problems are simply degenerate cases of pure mathematical
problems.



Sean McDirmid wrote:

> In PL design, everyone seems to have different ideas about what the best,
> even perfect, design is. Fundamentalists can stick to a stone tablet of
> principles that they won't waver from even if they clearly won't work, while
> pragmatists can be as shifty as successful politicians on using whatever
> principle is convenient or popular. I think with experience, many designers
> drift toward the latter camp, especially if they want their languages to be
> used. And then there are those that remain fundamentalists because this is
> also an interesting and useful path.
>
> I'm seeing a lot of debate recently about how decisions made in the Scala
> library are crazy, ill thought out, etc. However, the flip side of this coin
> is that the criticism doesn't consider the consequences of their "obvious"
> change. Example:
>
> Should "cba" == "abc".reverse be true? Of course it should be! What was the
> Scala team smoking when they decided RichString.reverse should return
> anything other than String? Of course it should be changed to return String
> so equality works right! Oh wait...reverse is defined in Seq to return a
> Seq, and the LUB of String and Seq is...AnyRef. So if reverse in RichString
> returns a String, then either reverse has to be removed from Seq, or it has
> to be defined to return AnyRef, or some compiler magic has to make String a
> Seq (in much the same way that arrays are) introducing more boxing madness
> to Scala (and equality would still be broken!). Given these concerns,
> changing reverse in RichString to return a String is no longer obviously the
> right thing to do! So the Scala team wasn't so stoned when they decided to
> have reverse return RichString rather than String. When you actually sit
> down and design/code these things, you realize how many bazillion trade offs
> there are to make.
>
> I've come to this conclusion: there are no good language/library design
> decisions, only bad design decisions. Whatever decision a language designer
> makes will necessarily be a bad one, because a trade off will be made that
> is obviously horrible to someone.  Some design decisions may be worse than
> others, but they are still bad. Accept that. And why would your bad design
> decision be any better or worse than someone else's? Self righteousness will
> only serve to aggravate others who don't hold your same principals. Anyone
> who suggest that they have found the perfect design solution is probably
> selling snake oil.
>
> All languages have their own set of bad design decisions that define them.
> C++ is religiously backwards compatible with C and its pervious versions,
> and as a result is the butt of many PL jokes. Is Bjarne an idiot for
> designing such a bad language? Well, C++ is hugely successful because many
> people and projects still prefer from C++'s bad decisions vs. say Java's.
> Bjarne is probably a genius, even if I personally don't like programming
> C++. Haskell is more idealistic and pure, not as popular, but is well
> designed and serves a purpose. Likewise, Ruby will do things behind your
> back, implicitly coerce things, re-direct your method calls...there is so
> much magic going on! Is Ruby a bad language? Of course (because they all
> are), but lots of people feel like this magic and feel like it servers a
> purpose. C# and Java on the other hand are more verbose and try not to do
> anything implicitly. Another bad decision, but many people benefit from that
> also. I'm actually growing to like C# at Microsoft; e.g., extension methods
> are less magical than implicits, but get the job done. The C# team were
> obviously not on meth when they designed C#, no matter how many times the
> thought crossed my mind when I was learning how to use C#!
>
> I like Scala's bad decisions more than the other languages'. In the library,
> a few of those bad decisions are mine (e.g., projections), and I've made
> many more bad decisions in the plugin (to quote comic shop guy: "the worst
> Scala plugin ever"). I've also disagreed with some of Martin's decisions
> (e.g., strong specification of protected that seems to reduce applicability,
> strictness in mutable collections where it seems to create a contradiction),
> but in the end his bad decisions often turn out to be less worse than mine
> (and of course, he out-ranks me :) ). Working with Martin was immensely
> educational and life changing.
>
> I was at first weary of implicits, but see that they serve a purpose and
> shouldn't be discounted as evil, well, no, they are evil but that means you
> shouldn't use them: they sere a purpose; e.g., through implicits and other
> constructs can do stuff with Scala that people thought were only possible in
> dynamic languages. Scala has the opportunity to can draw ultra-pragmatic
> programmers from Java who might otherwise go to Ruby or Python. Scala is
> also principled with a powerful type system that can draw in fundamentalists
> from languages like Haskell who are looking for a language their bosses will
> actually accept. And of course, many people lie in between. Essentially,
> Martin has created a "big tent" language that will attract users from all
> over the place. Although a big tent good thing, it leads to some tension
> that I'm feeling already. There are already many debates about implicits,
> Java compatibility, breaking binary compatibility, etc...everyone has a
> strong opinion, referring to other opinions as crazy or in one case
> "immature." This can easily get blood boiling, especially in the more
> passionate among us. I guess it is bound to happen in any language
> community, but more so in Scala given its big tent focus.
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIQQzbmnpgrYe6r60RAjQ1AKCHvWte3BvWFV83RC0wLWdUCN6oAACfTemm
PurOOtwMtcsTm2dsK9fcJgU=
=Vp2l
-----END PGP SIGNATURE-----

[Jesper Nordenberg]

The drawback is that a java.lang.String wouldn't be equal to a
scala.String even if the contents is the same, which given Scalas design
goals might not be acceptable.

/Jesper Nordenberg

Sean McDirmid wrote:

> Is it really that easy? Strings in Java are almost primitives, in spite
> of their java.lang.String exterior. We would still have issues with
> AnyRef strings coming in, equality wouldn't work where the user might
> expect it to, and it might be too slow. This is really related to the
> scala.Array problem: it breaks down in many cases where the compiler
> cannot detect where boxing is needed.
>
> But if you think scala.String is the right way, you should explore it in
> more detail, see what the drawbacks are in addition to the benefits. If
> you can't find drawbacks, then you haven't been thorough enough :)
> (Never trust any analysis with too many + and not enough -).
>
> Sean
>
> On Sat, May 31, 2008 at 4:13 PM, Jesper Nordenberg <megagurka@...
> <mailto:megagurka@...>> wrote:
>
>
>      > What was
>
>         the Scala team smoking when they decided RichString.reverse
>         should return anything other than String? Of course it should be
>         changed to return String so equality works right! Oh
>         wait...reverse is defined in Seq to return a Seq, and the LUB of
>         String and Seq is...AnyRef. So if reverse in RichString returns
>         a String, then either reverse has to be removed from Seq, or it
>         has to be defined to return AnyRef, or some compiler magic has
>         to make String a Seq (in much the same way that arrays are)
>         introducing more boxing madness to Scala (and equality would
>         still be broken!).
>
>
>     Actually not much compiler magic is needed, just make string
>     literals be of class scala.String (which would be a Seq) and provide
>     implicit conversions to and from java.lang.String.
>

[Martin Odersky]

I largely agree with Sean. When it comes to the standard library it
seems there can be only bad decisions. However (slow!) progress is
possible. In particular, regarding

"abc" == "cba".reverse

we will be able to achieve this and still keep Seq-like interface for
Strings, using the ideas for a higher-kinded addition to the standard
libraries that Adriaan and I are exploring. When we designed the
standard library, there were no higher-kinded types, so this option
was not available.

Cheers

 -- Martin

[James Iry-2]

Well said! Hear hear!

On Fri, May 30, 2008 at 11:31 PM, Sean McDirmid <sean.mcdirmid@...> wrote:

In PL design, everyone seems to have different ideas about what the best, even perfect, design is.