[scala] Risks of using Scala in a large application?

[Ashley Aitken]

Howdy All,

Scala seems to me, from my quick introduction, what Java may be like  
in 5+ years.

So using Scala can give one a competitive advantage.  However, there  
is often risk associated with such a decision.  As a result, I've just  
popped in to ask this question, which is mostly from a business risk  
angle, but this relates to technical issues regarding Java and Scala.

What are the risks of using Scala in a large Enterprise application?  
Mostly I mean in relationship to compatibility with Java, interaction  
with Java libraries (will Scala limit this) and tools, Java  
interaction with Scala classes etc.

The low risk situation would be that Java and Scala are completely  
interchangeable, if I write something now in Scala it won't limit me  
and I could replace it with Java in the future if need be (e.g. if/
when Java catches up and/or Scala doesn't catch on) without any trouble.

Of course there are other risk: finding developers that are competent  
enough in Scala, IDE tooling for Scala, longevity of Scala, and any  
discussion of these would also be appreciated.

I hope this sort of question is ok for this mailing list.

Cheers,
Ashley.

--
Ashley Aitken
04 1226-8159 (Mb)
08 9368-5505 (Ph)
Perth, Western Australia
Skype/iChat: MrHatken (GMT + 8hrs!)

[Sean McDirmid]

The new Scala IDE plugin is a large code base written in Scala that
interacts pretty well in a Java-centric Eclipse ecosystem. As far as
interoperability goes, there are some tool issues (inter-language
refactoring), but things are very comfortable at the runtime/code
level.

The biggest risks I see are:

(1) Scala-centric libraries are more difficult to use in Java because
they heavily rely on Scala features (closures, traits, type
parameters). They especially become more functional, which is
inconvenient (though not impossible) to deal with in Java. Java
libraries require some amount of wrapping before they become
convenient to use in Scala (see scala.collection.jcl for an example).

As for (1), I believe that whether or not Scala catches on, Java won't
catch up and there always be something out there that is kind of
functional on the chase to replace it (e.g., I don't think you will
migrate Scala code to Java, but Scala to whatever static Ruby/Python,
UberJava++, and so on). Java is dying.

(2) Once you program in Scala, its hard to go back to less
programmer-friendly languages such as Java or C#. If you have to code
in Java for a living, it can be painful knowing there are better
options available that you can't use in your day job. Psychologically,
I believe this is a real issue, but it depends on how passionate you
are about programming (some people are probably content to program in
anything they are told to).





On Wed, May 7, 2008 at 12:33 PM, Ashley Aitken <mrhatken@...> wrote:

>
>  Howdy All,
>
>  Scala seems to me, from my quick introduction, what Java may be like in 5+
> years.
>
>  So using Scala can give one a competitive advantage.  However, there is
> often risk associated with such a decision.  As a result, I've just popped
> in to ask this question, which is mostly from a business risk angle, but
> this relates to technical issues regarding Java and Scala.
>
>  What are the risks of using Scala in a large Enterprise application?
> Mostly I mean in relationship to compatibility with Java, interaction with
> Java libraries (will Scala limit this) and tools, Java interaction with
> Scala classes etc.
>
>  The low risk situation would be that Java and Scala are completely
> interchangeable, if I write something now in Scala it won't limit me and I
> could replace it with Java in the future if need be (e.g. if/when Java
> catches up and/or Scala doesn't catch on) without any trouble.
>
>  Of course there are other risk: finding developers that are competent
> enough in Scala, IDE tooling for Scala, longevity of Scala, and any
> discussion of these would also be appreciated.
>
>  I hope this sort of question is ok for this mailing list.
>
>  Cheers,
>  Ashley.
>
>  --
>  Ashley Aitken
>  04 1226-8159 (Mb)
>  08 9368-5505 (Ph)
>  Perth, Western Australia
>  Skype/iChat: MrHatken (GMT + 8hrs!)
>
>
>

[Dave Griffith]

Scala seems to me, from my quick introduction, what Java may be like  
in 5+ years.

I'd guess more like 10 years.  Java evolution has gotten very slow, compared to the level of gap between Java and Scala.

The low risk situation would be that Java and Scala are completely  
interchangeable, if I write something now in Scala it won't limit me  
and I could replace it with Java in the future if need be (e.g. if/
when Java catches up and/or Scala doesn't catch on) without any trouble.

From the point of view of technical risk, this low-risk situation is pretty close to the truth.  Moreover, the Scala team seems to have a high level of commitment to keeping it that way.


Of course there are other risk: finding developers that are competent  
enough in Scala, IDE tooling for Scala, longevity of Scala, and any  
discussion of these would also be appreciated.


While recruiting and tooling risks are certainly important, a larger issue is that the evolution of Scala is driven by a pretty small team, and their orientation is academic rather than commercial.  While this certainly has great benefits in terms of both language quality and rate of evolution, it does raise some risks with regard to the predictability of that evolution.  With a language like Java, you can make some pretty good guesses as to what it will look like 5 years from now.  With Scala, it's tough to guess what it will look like next Christmas (other than that it will be great).  This aggravates other risks, as developers become reluctant to train toward a moving target, and tool makers become skittish of the dev costs of supporting one.  This risk isn't a killer, but may explain some conservatism you may encounter about moving to Scala.

[James Iry-2]

On Tue, May 6, 2008 at 9:33 PM, Ashley Aitken <mrhatken@...> wrote:

 

Scala seems to me, from my quick introduction, what Java may be like in 5+ years.

Language design is as much about what you don't put in as what you do.  Scala ditched or changed several of the shakier concepts in Java.  For instance, do you really need both if/else and the ternary (?:) operator;  can't one construct do both?  Isn't return just syntactic noise in most places?  Hasn't "switch" with fall-through been a known source of bugs since the days of C?  

I don't see Java fixing these things.  In other words, no matter how much is added to it, Java will probably never be Scala.

So using Scala can give one a competitive advantage.  However, there is often risk associated with such a decision.  

There are substantial productivity gains to be made, though, and that can help offset risks.  E.g., applications often get replaced about every 5 years anyway - so what could you gain in 5 years with double productivity?  If for some reason you have to replace after only 3 years, won't you still have come out ahead?
 

What are the risks of using Scala in a large Enterprise application?  Mostly I mean in relationship to compatibility with Java, interaction with Java libraries (will Scala limit this) and tools, Java interaction with Scala classes etc.

Scala can "consume" any normal Java code, extend a Java class, etc.  There are some challenges in some cases where libraries use byte code enhancement.  In those few cases, people have been writing a few parts of their code base in Java.

The converse takes more effort.  Java can consume Scala code as long as it more or less sticks to the Java-like subset of Scala.

 

The low risk situation would be that Java and Scala are completely interchangeable, if I write something now in Scala it won't limit me and I could replace it with Java in the future if need be (e.g. if/when Java catches up and/or Scala doesn't catch on) without any trouble.

In a theoretical sense neither language will create a hard limit, of course.  Anything you can accomplish in one you can accomplish in the other.  In practice the interaction with 3rd party libraries is a bit different and you just can't afford to reinvent all those wheels.  The good news is that Java's slow change rate and the vast size of the existing code base for Java mean that you shouldn't have to worry about Java dependencies disappearing for a good long time.  Since you can consume pretty much all of that from Scala right now, you're set for awhile.
 

Of course there are other risk: finding developers that are competent enough in Scala, IDE tooling for Scala, longevity of Scala, and any discussion of these would also be appreciated.

For now just assume you won't find Scala developers and that you will have to train.  Some back of the envelope estimates indicate, to me at least, that the ROI is very good on this investment - e.g. you might have to keep a developer for only 4 months for training time to pay off.  But that's very back of the envelope.  While there's much anecdotal evidence on the increase in productivity with Scala there's been no attempt to do a real study.  And how long it takes to retrain a Java programmer in Scala is anybody's guess.

IDEs and other tooling are making huge progress.  But remember, the reports of productivity increase have come with tools as they exist now.  As for longevity of Scala - well the bits aren't going anywhere.  Even if your team turns into the only one on the planet that uses Scala, you'll be able to keep going for a long time while you figure out if, when, and how to replace it.

[Martin Odersky]

>  While recruiting and tooling risks are certainly important, a larger issue
>  is that the evolution of Scala is driven by a pretty small team, and their
>  orientation is academic rather than commercial.  While this certainly has
>  great benefits in terms of both language quality and rate of evolution, it
>  does raise some risks with regard to the predictability of that evolution.
>  With a language like Java, you can make some pretty good guesses as to what
>  it will look like 5 years from now.  With Scala, it's tough to guess what it
>  will look like next Christmas (other than that it will be great).

This was true 2 years ago, but it's no longer true now. Scala has been
evolving rapidly until about one year ago. Since then, any changes
have been incremental. I can guarantee that Scala next Christmas will
be pretty much the same as what Scala is now. In particular, I don't
see any backwards incompatible changes. For the longer term, future
evolution will be carefully managed. Scala has come out of the
academic world for good; it's too late to put the Jini back in the
bottle. So if there would be major breaking changes they would be
announced long in advance, and a transition path would be mapped out,
sort of what happens now with Python 3K vs Python 2.6.

Cheers

 -- Martin

[Bruce Stephens]

Ashley Aitken <mrhatken@...> writes:

[...]

> Of course there are other risk: finding developers that are competent
> enough in Scala,

That seems bogus for most situations.  Maybe for a startup with no
money you'd require that kind of specific knowledge (so people could
start right away).  Or a new project in some company where the
developers have worked extensively with Java but with nothing much
else.

Surely in most projects you've got a fair amount of stuff that a new
developer's going to have to learn, and scala doesn't seem
particularly onerous.

If you have a policy of employing people who have a really limited
range of experience (so not including anything outside C/C++/Java)
then there's some risk, but surely anyone with some familiarity with
one of F#, Lisp, Perl, Python, Ruby, etc., (even C#) isn't going to
find scala *that* tricky to pick up?  And surely those are the people
you're looking for anyway?

[...]

[bearfeeder]

On Tue, May 6, 2008 at 9:33 PM, Ashley Aitken <mrhatken@...> wrote:

Howdy All,

Scala seems to me, from my quick introduction, what Java may be like in 5+ years.

So using Scala can give one a competitive advantage.  However, there is often risk associated with such a decision.  As a result, I've just popped in to ask this question, which is mostly from a business risk angle, but this relates to technical issues regarding Java and Scala.

What are the risks of using Scala in a large Enterprise application?  Mostly I mean in relationship to compatibility with Java, interaction with Java libraries (will Scala limit this) and tools, Java interaction with Scala classes etc.

The low risk situation would be that Java and Scala are completely interchangeable, if I write something now in Scala it won't limit me and I could replace it with Java in the future if need be (e.g. if/when Java catches up and/or Scala doesn't catch on) without any trouble.

I blogged about some of these issues: http://blog.lostlake.org/index.php?/archives/73-For-all-you-know,-its-just-another-Java-library.html
 

Of course there are other risk: finding developers that are competent enough in Scala, IDE tooling for Scala, longevity of Scala, and any discussion of these would also be appreciated.

Finding developers is the single biggest risk factor.  There's a class of "Java Guys" who are staunchly anti-Scala.  In general, I would hire clear poliglots... people who thrive on using many different languages (personally, I like Ruby/Python developers who have had some Java in their background).  In my experience, these are the kind of people who thrive on Scala as Scala blends a bunch of different language concepts elegantly into a unified whole.  This is not the case with many of the Java bolt-ons (Generics, etc.)

I would not worry about the tooling issue.  Much of the existing tooling works very well: debuggers, build systems, deployment systems, etc.  Sitting on top of the JVM infrastructure is a very nice place to sit.

The IDE issue depends on your team.  I tend to hire people who are more productive in text editors than IDEs and have their personal choices (emacs, vi, etc.)  Going with an IDE, one also hitches ones horse to a the configuration baggage that the given IDE brings (i.e., mixing IntelliJ, Eclipse, and NetBeans in a single project is non-trivial.)

You will also likely see some announcements this week (in particular Saturday at the Scala lift off http://scalaliftoff.com ) about commitments different vendors have to supporting the Scala community.  The IDE vendors seem to have woken up to Scala in the last few months.  Vendors like ZeroTurnaround has put a fair amount of testing into JavaRebel to make sure it works as well with Scala as it does with Java.  The list is a growing one.

Thanks,

David

PS -- I started using Scala in November 2006.  The evolution of the language has primarily been one of addition and there have been very few places where things have broken and when they have, it's been at compile time and once I've changed my code to the new syntax, all has been fine.  Martin has publicly committed to avoid breaking changes to the language syntax in the future.  Between lift and the other code bases I maintain, I've got about 40,000 LoC of Scala code across various projects.  To date, I have been extremely happy (blown away actually) at the quality of Scala and the level of support, both on this list and from the Scala team, that I've received.  The Scala team, measured against any commercial ISV, ranks in the top percentile in terms of code quality and responsiveness of any ISV I've dealt with in my 30 years of professional software development.
 

I hope this sort of question is ok for this mailing list.

Cheers,
Ashley.

--
Ashley Aitken
04 1226-8159 (Mb)
08 9368-5505 (Ph)
Perth, Western Australia
Skype/iChat: MrHatken (GMT + 8hrs!)

--
Scala lift off unconference, May 10th 2008, San Francisco http://scalaliftoff.com
lift, the secure, simple, powerful web framework http://liftweb.net
Collaborative Task Management http://much4.us

[Christos KK Loverdos]

Hi David,

There's a class of "Java Guys" who are staunchly anti-Scala.

!!!!

I tend to hire people who are more productive in text editors than IDEs and have their personal choices (emacs, vi, etc.)

What are you reminding me...!!!!
I once was in a job interview (6 years ago?) and was asked what IDE I was using for my coding. I then answered, ViM mostly, with my own plugins that made it behave like an IDE (integrated javadocs, compilation, file/project exploration, code outlining and that kind of stuff). At the name of ViM (which I had to explain it was like Vi), there was a sigh and I think I saw a minus sign drawn in a tiny check box ... :-)
 

Christos

--
__~O
-\ <, Christos KK Loverdos
(*)/ (*) http://ckkloverdos.com

[Ricky Clarkson]

I hope you rejected their offer.

On Wed, May 7, 2008 at 1:16 PM, Christos KK Loverdos <loverdos@...> wrote:

Hi David,

There's a class of "Java Guys" who are staunchly anti-Scala.

!!!!

I tend to hire people who are more productive in text editors than IDEs and have their personal choices (emacs, vi, etc.)

What are you reminding me...!!!!
I once was in a job interview (6 years ago?) and was asked what IDE I was using for my coding. I then answered, ViM mostly, with my own plugins that made it behave like an IDE (integrated javadocs, compilation, file/project exploration, code outlining and that kind of stuff). At the name of ViM (which I had to explain it was like Vi), there was a sigh and I think I saw a minus sign drawn in a tiny check box ... :-)
 

Christos

--
__~O
-\ <, Christos KK Loverdos
(*)/ (*) http://ckkloverdos.com

[Christos KK Loverdos]

Actually I got lucky and they turned me down first: "We regret to inform you that....but we are keeping your CV for future oppourtunities...".

The funny thing is they wanted someone "experienced" to build a new internal R&D team.  I damn could write an IDE, but was not experienced enough since I did not use one.

Today that "to-be" team has grown into a full department. They are my clients, all right :-) Life has its turns.

Cheers
Christos

On Wed, May 7, 2008 at 3:30 PM, Ricky Clarkson <ricky.clarkson@...> wrote:

I hope you rejected their offer.

On Wed, May 7, 2008 at 1:16 PM, Christos KK Loverdos <loverdos@...> wrote:

Hi David,

There's a class of "Java Guys" who are staunchly anti-Scala.

!!!!

I tend to hire people who are more productive in text editors than IDEs and have their personal choices (emacs, vi, etc.)

What are you reminding me...!!!!
I once was in a job interview (6 years ago?) and was asked what IDE I was using for my coding. I then answered, ViM mostly, with my own plugins that made it behave like an IDE (integrated javadocs, compilation, file/project exploration, code outlining and that kind of stuff). At the name of ViM (which I had to explain it was like Vi), there was a sigh and I think I saw a minus sign drawn in a tiny check box ... :-)
 

Christos

--
__~O
-\ <, Christos KK Loverdos
(*)/ (*) http://ckkloverdos.com

--
__~O
-\ <, Christos KK Loverdos
(*)/ (*) http://ckkloverdos.com

[Ricky Clarkson]

Fantastic story, thanks for sharing that.  And you said it in 1% of the words Steve Yegge would have, without losing anything.  Truly a Scala programmer. :)

On Wed, May 7, 2008 at 1:50 PM, Christos KK Loverdos <loverdos@...> wrote:

Actually I got lucky and they turned me down first: "We regret to inform you that....but we are keeping your CV for future oppourtunities...".

The funny thing is they wanted someone "experienced" to build a new internal R&D team.  I damn could write an IDE, but was not experienced enough since I did not use one.

Today that "to-be" team has grown into a full department. They are my clients, all right :-) Life has its turns.

Cheers
Christos

On Wed, May 7, 2008 at 3:30 PM, Ricky Clarkson <ricky.clarkson@...> wrote:

I hope you rejected their offer.

On Wed, May 7, 2008 at 1:16 PM, Christos KK Loverdos <loverdos@...> wrote:

Hi David,

There's a class of "Java Guys" who are staunchly anti-Scala.

!!!!

I tend to hire people who are more productive in text editors than IDEs and have their personal choices (emacs, vi, etc.)

What are you reminding me...!!!!
I once was in a job interview (6 years ago?) and was asked what IDE I was using for my coding. I then answered, ViM mostly, with my own plugins that made it behave like an IDE (integrated javadocs, compilation, file/project exploration, code outlining and that kind of stuff). At the name of ViM (which I had to explain it was like Vi), there was a sigh and I think I saw a minus sign drawn in a tiny check box ... :-)
 

Christos

--
__~O
-\ <, Christos KK Loverdos
(*)/ (*) http://ckkloverdos.com

--
__~O
-\ <, Christos KK Loverdos
(*)/ (*) http://ckkloverdos.com

[Sean McDirmid]

It is an issue here in China where the students don't get enough
programming language experience (C++ and Java, maybe C#). Some of the
students are more aggressive at learning than others, they are the
better programs and can learn more than the base language. iPhone is
really helping here, as many of the better hackers are learning
Cocoa/Objective C to build applications for it.

At the low end though, Scala would definitely be a problem. If you
have a good high-end programming team, I don't think Scala would be a
problem, they would probably be happy to use it (having already been
exposed to languages better than C#/Java). If you are outsourcing to
Vietnam, you might want to be more careful (then again, its only a
matter of time before the computer science education systems in these
countries pick up).

Sean

On Wed, May 7, 2008 at 4:45 PM, Bruce Stephens
<bruce@...> wrote:

>  That seems bogus for most situations.  Maybe for a startup with no
>  money you'd require that kind of specific knowledge (so people could
>  start right away).  Or a new project in some company where the
>  developers have worked extensively with Java but with nothing much
>  else.
>
>  Surely in most projects you've got a fair amount of stuff that a new
>  developer's going to have to learn, and scala doesn't seem
>  particularly onerous.
>
>  If you have a policy of employing people who have a really limited
>  range of experience (so not including anything outside C/C++/Java)
>  then there's some risk, but surely anyone with some familiarity with
>  one of F#, Lisp, Perl, Python, Ruby, etc., (even C#) isn't going to
>  find scala *that* tricky to pick up?  And surely those are the people
>  you're looking for anyway?
>
>  [...]
>
>

[Amir Michail]

On Wed, May 7, 2008 at 1:58 AM, martin odersky <martin.odersky@...> wrote:

> >  While recruiting and tooling risks are certainly important, a larger issue
>  >  is that the evolution of Scala is driven by a pretty small team, and their
>  >  orientation is academic rather than commercial.  While this certainly has
>  >  great benefits in terms of both language quality and rate of evolution, it
>  >  does raise some risks with regard to the predictability of that evolution.
>  >  With a language like Java, you can make some pretty good guesses as to what
>  >  it will look like 5 years from now.  With Scala, it's tough to guess what it
>  >  will look like next Christmas (other than that it will be great).
>
>  This was true 2 years ago, but it's no longer true now. Scala has been
>  evolving rapidly until about one year ago. Since then, any changes
>  have been incremental. I can guarantee that Scala next Christmas will
>  be pretty much the same as what Scala is now. In particular, I don't
>  see any backwards incompatible changes. For the longer term, future
>  evolution will be carefully managed. Scala has come out of the
>  academic world for good; it's too late to put the Jini back in the

If Scala has come out of the academic world, then how will its
development be funded?

Although the language itself may not change much, one might still want
to port it to various platforms, expand its libraries, enhance its
tools/IDEs, etc.

Amir

>  bottle. So if there would be major breaking changes they would be
>  announced long in advance, and a transition path would be mapped out,
>  sort of what happens now with Python 3K vs Python 2.6.
>
>  Cheers
>
>   -- Martin
>

[James Iry-2]

On Wed, May 7, 2008 at 4:54 AM, David Pollak <feeder.of.the.bears@...> wrote:

 There's a class of "Java Guys" who are staunchly anti-Scala. 

Most of that crowd is staunchly anti-everything-that's-not-Java. :-) 

Someone, I think Brooks in Mythical Man Month, talked about studies that showed a strong correlation between the number of languages known and a programmer's productivity even in a single language.

It's not clear whether knowing more languages makes you more productive or whether strong programmers are attracted to learning languages.  Either way, even pure Java shops are probably better served hiring polyglots.

[Ashley Aitken]

On 08/05/2008, at 12:10 AM, James Iry wrote:

> On Wed, May 7, 2008 at 4:54 AM, David Pollak <feeder.of.the.bears@...
> > wrote:
>
>  There's a class of "Java Guys" who are staunchly anti-Scala.
>
> Most of that crowd is staunchly anti-everything-that's-not-Java. :-)

I was thinking that any (most?) anti-Scala sentiment would probably be  
more along the lines of 'its too powerful (or complex) for the  
"average" developer'.

BTW, thanks to everyone who has contributed to this thread, it has  
been very informative.

Cheers,
Ashley.

--
Ashley Aitken
04 1226-8159 (Mb)
08 9368-5505 (Ph)
Perth, Western Australia
Skype/iChat: MrHatken (GMT + 8hrs!)

[Dave Griffith]

Martin Odersky wrote:

 Scala has been
evolving rapidly until about one year ago. Since then, any changes
have been incremental. I can guarantee that Scala next Christmas will
be pretty much the same as what Scala is now.

That's very good to hear.  The next steps along those lines need to be publishing of roadmaps, publicly readable issue-tracking systems (don't know, and (eventually, eventually) thinking about submission to appropriate standardization forums (JCR?).   Those would go a long way to alleviating fears of evolution risks.

[Paul LaCrosse]

Dave Griffith wrote:

That's very good to hear.  The next steps along those lines need to be publishing of roadmaps, publicly readable issue-tracking systems (don't know, and (eventually, eventually) thinking about submission to appropriate standardization forums (JCR?).   Those would go a long way to alleviating fears of evolution risks.

What's wrong with the bug-tracking system they already have up?

https://lampsvn.epfl.ch/trac/scala/query

Even though I'm a member of the expert group of one of the JSRs (in the JCP), even I don't think that it would be an appropriate place for Scala (the current arrangement seems to be working out just fine).