[scala] Collections performance

[Christos KK Loverdos]

I just remembered that I tried once to figure out what is going on in the OSS licenses landscape and my little findings are here:

http://blog.ckkloverdos.com/2007/10/18/on-open-source-licenses-ckkl-core/

BR
Christos.

On Wed, Jul 30, 2008 at 4:20 PM, Christos KK Loverdos <loverdos@...> wrote:

This is totally untrue. It is vaguely sortof kindof true if you think
applications begin and end with web applications which don't get
distributed to end users, but they don't. If you're distributing the
application, distributing a library licensed under the GPL along with
it is going to force you to distribute the source of the application
too.

You may be thinking of the LGPL, which is a very different license.

Yes, I was trying to speak generally for the OSS landscape with a particular license in mind, which is unsound by definition. :-)

Well, the case with GPL is what constitutes a derivative work: Does my program, which just links to a GPL library, even dynamically, constitute a derivative work of that GPL library?
It seems that this issue has not been legally settled for good up to now, but If one wants to be careful, then one should be better off avoiding the use of GPL code in commercial cases we discuss here....

--
__~O
-\ <, Christos KK Loverdos
(*)/ (*) http://ckkloverdos.com

--
__~O
-\ <, Christos KK Loverdos
(*)/ (*) http://ckkloverdos.com

[David MacIver]

On Wed, Jul 30, 2008 at 1:46 PM, David Pollak <dpp@...> wrote:

>
>
> Sean McDirmid wrote:
>>
>> I'm not sure about the licensing implications of that. The source code
>> for the Java collections is GPLed, and that would definitely count as
>> a derivative work.
>
> Just put it out under the same license.
>
> Copyright violations are code theft.  It's wrong, not matter what the
> purpose is.  Further, if parts of Scala are licensed under the GPL, lift's

I'm not sure I see how that's a copyright violation. I'd been under
the impression it was perfectly legitimate to modify GPLed code or
produce derivative works as long as the results were also GPLed. But I
could be wrong - I don't tend to use the GPL for anything if I can
possibly help it.

[Erik Engbrecht]

No, you don't have it wrong.  The GPL is viral, and consequently a lot of commercial entities avoid GPL'd code.  Ergo, if Scala and lift were infected with the GPL virus, both would be much less attractive to the commercial world.

On 7/30/08, David MacIver <david.maciver@...> wrote:

On Wed, Jul 30, 2008 at 1:46 PM, David Pollak <dpp@...> wrote:
>
>
> Sean McDirmid wrote:
>>
>> I'm not sure about the licensing implications of that. The source code
>> for the Java collections is GPLed, and that would definitely count as
>> a derivative work.
>
> Just put it out under the same license.
>
> Copyright violations are code theft.  It's wrong, not matter what the
> purpose is.  Further, if parts of Scala are licensed under the GPL, lift's

I'm not sure I see how that's a copyright violation. I'd been under
the impression it was perfectly legitimate to modify GPLed code or
produce derivative works as long as the results were also GPLed. But I
could be wrong - I don't tend to use the GPL for anything if I can
possibly help it.

--
http://erikengbrecht.blogspot.com/

[David MacIver]

I knew that bit was right. The bit I was unclear on was the copyright issue.

On Wed, Jul 30, 2008 at 3:26 PM, Erik Engbrecht
<erik.engbrecht@...> wrote:

> No, you don't have it wrong.  The GPL is viral, and consequently a lot of
> commercial entities avoid GPL'd code.  Ergo, if Scala and lift were infected
> with the GPL virus, both would be much less attractive to the commercial
> world.
>
> On 7/30/08, David MacIver <david.maciver@...> wrote:
>>
>> On Wed, Jul 30, 2008 at 1:46 PM, David Pollak <dpp@...> wrote:
>> >
>> >
>> > Sean McDirmid wrote:
>> >>
>> >> I'm not sure about the licensing implications of that. The source code
>> >> for the Java collections is GPLed, and that would definitely count as
>> >> a derivative work.
>> >
>> > Just put it out under the same license.
>> >
>> > Copyright violations are code theft.  It's wrong, not matter what the
>> > purpose is.  Further, if parts of Scala are licensed under the GPL,
>> > lift's
>>
>> I'm not sure I see how that's a copyright violation. I'd been under
>> the impression it was perfectly legitimate to modify GPLed code or
>> produce derivative works as long as the results were also GPLed. But I
>> could be wrong - I don't tend to use the GPL for anything if I can
>> possibly help it.
>
>
>
> --
> http://erikengbrecht.blogspot.com/

[Miles Sabin]

On Wed, Jul 30, 2008 at 3:26 PM, Erik Engbrecht
<erik.engbrecht@...> wrote:
> No, you don't have it wrong.  The GPL is viral, and consequently a lot of
> commercial entities avoid GPL'd code.  Ergo, if Scala and lift were infected
> with the GPL virus, both would be much less attractive to the commercial
> world.

GPL virus?

Reasonable people can disagree about the relative merits of the GPL
vs. more BSD-like licenses, so can we please avoid emotive language
...

My view is that since both the Scala compiler and libraries and the
Eclipse plugin are under BSD-style licenses, then it would make sense
for projects like scala[xz] which aim to feed into them to do the
same, otherwise there'll be issues with relicensing later.

Cheers,


Miles

[Sean McDirmid]

I wasn't talking about stealing code. Just copy it, label it as a derived work, and make sure the appropriate license is used. GPL definitely allows for that, there is nothing wrong with creating a derived work as long as the original work is attributed and the derived work is released under GPL. Isn't that what GPL is for? Also, you can link your applications to a GPL library and still have the application be commercial, you just can't modify/create a derivative of the library that is closed source.

OK, I see I've entered into a license hell discussion, but better to have it now then after we've actually ported code :)

On Wed, Jul 30, 2008 at 8:46 PM, David Pollak <dpp@...> wrote:

Sean McDirmid wrote:

I'm not sure about the licensing implications of that. The source code
for the Java collections is GPLed, and that would definitely count as
a derivative work.

Just put it out under the same license.

Copyright violations are code theft.  It's wrong, not matter what the purpose is.  Further, if parts of Scala are licensed under the GPL, lift's license becomes incompatible.  And it means lots of corporations can't use it. 

If RMS wants to sue us, we can burn that bridge when we get to it.

RMS is not the copyright holder, Sun is and they take their IP very, very seriously. 

We aren't selling Scala or being commercial in anyway, so I don't see any liability there.

The GPL imposes significant burdens on the licensee.  It's not about being open, it's about being free.  The GPL means that all other software that's linked to it must also be free.

We just make sure our stuff is as open as the stuff we are copying.

Once again, copyright violation is theft.  It's no different than stealing a car.  Being part of the FOSS community means respecting that.

David

[Sean McDirmid]

But copying code or creating derived work in a way that follows the rules of the license is neither theft nor a copyright violation. Just read the GPL...you won't be infected just by reading it, I promise.

On Wed, Jul 30, 2008 at 9:20 PM, David Pollak <dpp@...> wrote:

Viktor Klang wrote:

We just make sure our stuff is as open as the stuff we are copying.

Once again, copyright violation is theft.  It's no different than stealing a car.  Being part of the FOSS community means respecting that.

With all respect, copyright violation is copyright violation, theft is theft.
Both are illegal, but don't compare apples to nuts.

Copying an MP3 from a friend may be copyright violation, but it may not be theft.

In this case, taking someone else's code (Property) and putting it in your code in violation of license terms (without their permission), is theft.  See http://en.wikipedia.org/wiki/Theft

[Geoffrey Alan Washburn-4]

Sean McDirmid wrote:
> Isn't that what GPL is for? Also, you can
> link your applications to a GPL library and still have the application
> be commercial, you just can't modify/create a derivative of the library
> that is closed source.

No you can't.  If you link to GPL code your code must be GPLed. Which
doesn't mean you can't sell you program, but that is another discussion.
  You're probably thinking of LGPL.

[Sean McDirmid]

The issue is important if we wanted to port other libraries directly to Scala, as these would be derived works and could have restricted licenses accordingly. E.g., if we want to port JCL directly to Scala, then we'd have to put the resulting code under the GPL, because that is what the GPL requires. If we say that scalay can only contain code under the current BSD license, then we have to be careful not to port any code whose derived works cannot be licensed with BSD.

On Wed, Jul 30, 2008 at 10:56 PM, Miles Sabin <miles@...> wrote:

On Wed, Jul 30, 2008 at 3:26 PM, Erik Engbrecht
<erik.engbrecht@...> wrote:

> No, you don't have it wrong.  The GPL is viral, and consequently a lot of
> commercial entities avoid GPL'd code.  Ergo, if Scala and lift were infected
> with the GPL virus, both would be much less attractive to the commercial
> world.

GPL virus?

Reasonable people can disagree about the relative merits of the GPL
vs. more BSD-like licenses, so can we please avoid emotive language
...

My view is that since both the Scala compiler and libraries and the
Eclipse plugin are under BSD-style licenses, then it would make sense
for projects like scala[xz] which aim to feed into them to do the
same, otherwise there'll be issues with relicensing later.

Cheers,


Miles

[Miles Sabin]

On Wed, Jul 30, 2008 at 4:21 PM, Sean McDirmid <sean.mcdirmid@...> wrote:
> The issue is important if we wanted to port other libraries directly to
> Scala, as these would be derived works and could have restricted licenses
> accordingly.

Sure, so we have to be careful and choose licenses which are
compatible with the existing ones used in the Scala libraries.

> E.g., if we want to port JCL directly to Scala, then we'd have
> to put the resulting code under the GPL, because that is what the GPL
> requires.

If the JCL is published under the full GPL rather than the LGPL or the
GPL+classpath exception then we just can't use it if we want to fold
the result into the Scala standard library. I don't know if it is or
not ... could someone check?

> If we say that scalay can only contain code under the current BSD
> license, then we have to be careful not to port any code whose derived works
> cannot be licensed with BSD.

Yeah, well, we have to be careful :-)

Cheers,


Miles

--
Miles Sabin
tel: [phone]
mobile: [phone]
skype: milessabin

[Josh Suereth]

You can still sell you program in you link to GPL'd code.  That is of course, if the licensing allows for commercial licensing.  There's a lot of GPL products out their that do this, and seems to be a good business idea.   You make the open source community happy AND you can sell your product to companies.  Win-Win.  That being said, I'm not sure if paying for Scala would make more people want to use it (as it must be good if they're charging) or less (as most people want everything + the kitchen sink for free).   The real issue here is how to we get a fast collections API, not arguing of licensing details.  Let's assume we all have enough sense not to commit illegal activites.  Let's not argue what stealing is.  Let's just respect the authors of code, and figure out how/where we can get a fast collections API.

My $.02 are that we should try to obtain/create code that's compatable with the existing Scala license so there aren't any issues.  And don't borrow code without discussing it with the authors.

On Wed, Jul 30, 2008 at 11:19 AM, Geoffrey Alan Washburn <geoffrey.washburn@...> wrote:

Sean McDirmid wrote:

Isn't that what GPL is for? Also, you can link your applications to a GPL library and still have the application be commercial, you just can't modify/create a derivative of the library that is closed source.

No you can't.  If you link to GPL code your code must be GPLed. Which doesn't mean you can't sell you program, but that is another discussion.  You're probably thinking of LGPL.

[Florian Hars-3]

David MacIver schrieb:
> Scalax, Scalaz, Scalay. Sooner or later we're going to run out of letters!

Well, I don't think so:

$ cat foo.scala
package šçăłąƻ

object Φøơ {
  def main(argv: Array[String]) {
    println("I saw some grass growing through the pavement today.");
  }
}
$ scalac foo.scala
hars@abel:~$ scala  šçăłąƻ.Φøơ
I saw some grass growing through the pavement today.
$

(Anyone got a font that actually contains cuneiform glyphs?)

- Florian
--
But our moral language is fragmented; our contemporaries reject the Kantian
hunch that choosing those things most admirable and plausible as ends in
themselves is the best practice; autonomous sources of the good are everywhere
brown and broken. Thus we have PHP. http://lambda-the-ultimate.org/node/1463

[Erik Engbrecht]

WARNING:  Non-lawyer discussing legal issues

 

You can't link to a GPL library from a commercial application that you distribute unless you distribute the application under the GPL.

 

You can link to a GPL library from a commercial application that you do not distribute but make available over the web.

 

An interesting wrinkle is the question of "porting."  If the intent were to cut & paste the Java into an IDE and then keep on hacking at it until it would compile as Scala, then I'd say it would be a derived work.  But underlying data structures and algorithms aren't copywrited, and neither are the various optimization techniques.  They can't be.  That's not what copywrite is for, at least in the US.  They may be patented, but I doubt it.  You probably could find much of the underlying concepts in academic literature.

 

So...in my opinion it's more like using the Java libraries for inspiration rather than creating a derived work.  I'd compare it to creating a new song that borrows elements like beat patterns, themes, etc from other songs.  Of course there have been lawsuits about stuff like that, but I think generally it is OK.

 

On 7/30/08, Sean McDirmid <sean.mcdirmid@...> wrote:

I wasn't talking about stealing code. Just copy it, label it as a derived work, and make sure the appropriate license is used. GPL definitely allows for that, there is nothing wrong with creating a derived work as long as the original work is attributed and the derived work is released under GPL. Isn't that what GPL is for? Also, you can link your applications to a GPL library and still have the application be commercial, you just can't modify/create a derivative of the library that is closed source.

OK, I see I've entered into a license hell discussion, but better to have it now then after we've actually ported code :)

On Wed, Jul 30, 2008 at 8:46 PM, David Pollak <dpp@...> wrote:

Sean McDirmid wrote:

I'm not sure about the licensing implications of that. The source code
for the Java collections is GPLed, and that would definitely count as
a derivative work.

Just put it out under the same license.

Copyright violations are code theft.  It's wrong, not matter what the purpose is.  Further, if parts of Scala are licensed under the GPL, lift's license becomes incompatible.  And it means lots of corporations can't use it. 

If RMS wants to sue us, we can burn that bridge when we get to it.

RMS is not the copyright holder, Sun is and they take their IP very, very seriously. 

We aren't selling Scala or being commercial in anyway, so I don't see any liability there.

The GPL imposes significant burdens on the licensee.  It's not about being open, it's about being free.  The GPL means that all other software that's linked to it must also be free.

We just make sure our stuff is as open as the stuff we are copying.
 

Once again, copyright violation is theft.  It's no different than stealing a car.  Being part of the FOSS community means respecting that.

David

 

 

--
http://erikengbrecht.blogspot.com/

[Sean McDirmid]

Don't forget chinese characters:

package scala三

class XXX {
  def foo = "你好"
}

On Wed, Jul 30, 2008 at 11:37 PM, Florian Hars <hars@...> wrote:

David MacIver schrieb:

> Scalax, Scalaz, Scalay. Sooner or later we're going to run out of letters!

Well, I don't think so:

$ cat foo.scala
package šçăłąƻ

object Φøơ {
 def main(argv: Array[String]) {
   println("I saw some grass growing through the pavement today.");
 }
}
$ scalac foo.scala
hars@abel:~$ scala  šçăłąƻ.Φøơ
I saw some grass growing through the pavement today.
$

(Anyone got a font that actually contains cuneiform glyphs?)

- Florian
--
But our moral language is fragmented; our contemporaries reject the Kantian
hunch that choosing those things most admirable and plausible as ends in
themselves is the best practice; autonomous sources of the good are everywhere
brown and broken. Thus we have PHP. http://lambda-the-ultimate.org/node/1463

[Patrick Wright]

My 0.02: There are, that I know of, at least 3 complete
implementations of the JCL: OpenJDK, Apache Harmony, and GNU
Classpath. Apache Harmony would have the license that's closest to the
BSD license that Scala uses. IMO, it's probably better for Scala as a
project to avoid simply including rewritten code from another
3rd-party library in the core.

There are also other implementations of the collections in Java, some
implementing the standard JCL interfaces, see
http://java-source.net/open-source/collection-libraries. Javolution
also has implementations of Set, List and Map (at least);
http://javolution.org/.

I agree with Erik that whoever takes these optimizations on can and
should take inspiration from the implementations listed above. I also
think it doesn't make a lot of sense to try and simply port one of
them to Scala; or at least, I'm not sure I would trust such a port.
Collections were meant to be written by hand :). Either way, there are
implementations that can serve as a guideline here (plus accompanying
tests in some cases).


Regards
Patrick

[James Iry-2]

Sean,

Scala itself might not be commercial but many of its users do hope to use it for commercial purposes.  And the issues go far beyond being sued.  If a company gains notice, it doesn't want its marketing message drowned out in accusations of licensing violations.  If somebody bets a company on Scala they want to know that when it comes time to sell the company the code base can stand up under the intense spotlight of due diligence.

As irritating and arcane as licensing is, it's important to understand the implications of various choices and make wise decisions.  And it's important not just to the Scala team but to Scala's users.

On Wed, Jul 30, 2008 at 4:11 AM, Sean McDirmid <sean.mcdirmid@...> wrote:

If RMS wants to sue us, we can burn that bridge when we get to it. We aren't selling Scala or being commercial in anyway, so I don't see any liability there. We just make sure our stuff is as open as the stuff we are copying.

[Sean McDirmid]

My original comments were a bit harsh and mis-informed. Basically, we have to avoid deriving from any GPL code, which primarily means Sun's implementation of the JCL (most other OSS collection projects that I know of use more liberal BSD-like licenses). Other comments:

(a) I don't speak for the Scala team, actually, I don't even work on the library anymore (someone else is maintaining the JCL wrappers now). As far as the core Scala project is concerned, I'm a developer who contributes to the plugin.

(b) I was recommending a Google code project maybe called scalay that would avoid the SCA and allow for more open contributions of possibly derived works under approved licenses. We could then shove a lot of collections (with unit tests) into the library, possibly most of them being derived vs. clean room works (properly attributed of course). Anything that goes through the EPFL must still be properly vetted and would be safer. scalay would be more open and edgy and you might want to avoid it for business purposes (or maybe not, I'd have no quams about using it in my work).

(c) deriving/copying is perfectly legal as long as you do it within the license. You won't go to jail (or lose your business) for looking at GPL code and then having capitalistic thoughts about it. But the GPL is seems rather toxic, best not to go near it.

(d) in communist china, the code licenses you.

On Thu, Jul 31, 2008 at 12:46 AM, James Iry <jamesiry@...> wrote:

Sean,

Scala itself might not be commercial but many of its users do hope to use it for commercial purposes.  And the issues go far beyond being sued.  If a company gains notice, it doesn't want its marketing message drowned out in accusations of licensing violations.  If somebody bets a company on Scala they want to know that when it comes time to sell the company the code base can stand up under the intense spotlight of due diligence.

As irritating and arcane as licensing is, it's important to understand the implications of various choices and make wise decisions.  And it's important not just to the Scala team but to Scala's users.

On Wed, Jul 30, 2008 at 4:11 AM, Sean McDirmid <sean.mcdirmid@...> wrote:

If RMS wants to sue us, we can burn that bridge when we get to it. We aren't selling Scala or being commercial in anyway, so I don't see any liability there. We just make sure our stuff is as open as the stuff we are copying.

[Miles Sabin]

I wrote:
> If the JCL is published under the full GPL rather than the LGPL or the
> GPL+classpath exception then we just can't use it if we want to fold
> the result into the Scala standard library. I don't know if it is or
> not ... could someone check?

OK, I just have done.

The OpenJDK JCL is published under the GPL *with* the classpath
exception[1]. Which is obvious really, because that code is pretty
much what the classpath exception is for. As such I think it would be
possible to include fairly lightly modified versions (eg. a quick and
dirty conversion to Scala) in the Scala standard library without any
real difficulties (ie. the issues with the full GPL raised by DP and
others don't arise with the classpath exception).

The code would have to continue to be published under the
GPL+classpath exception and it's up to the EPFL if they want to
complicate their licensing story with a mix of licenses in the core
libraries.

Oh, and for the record ... IANAL ;-)

[1] http://en.wikipedia.org/wiki/Classpath_exception

Cheers,


Miles

[Erik Engbrecht]

-1 to complicating the license situation, but no objections to using GPL+Classpath code for research purposes.

On 7/30/08, Miles Sabin <miles@...> wrote:

I wrote:
> If the JCL is published under the full GPL rather than the LGPL or the
> GPL+classpath exception then we just can't use it if we want to fold
> the result into the Scala standard library. I don't know if it is or
> not ... could someone check?

OK, I just have done.

The OpenJDK JCL is published under the GPL *with* the classpath
exception[1]. Which is obvious really, because that code is pretty
much what the classpath exception is for. As such I think it would be
possible to include fairly lightly modified versions (eg. a quick and
dirty conversion to Scala) in the Scala standard library without any
real difficulties (ie. the issues with the full GPL raised by DP and
others don't arise with the classpath exception).

The code would have to continue to be published under the
GPL+classpath exception and it's up to the EPFL if they want to
complicate their licensing story with a mix of licenses in the core
libraries.

Oh, and for the record ... IANAL ;-)

[1] http://en.wikipedia.org/wiki/Classpath_exception

Cheers,


Miles

--
http://erikengbrecht.blogspot.com/

[Arrgh]

First, in all that follows, remember: IANAL and TINLA (I am not a lawyer
and this is not legal advice):

Christos KK Loverdos wrote:
> Well, the case with GPL is what constitutes a derivative work: Does my
> program, which just links to a GPL library, even dynamically,
> constitute a derivative work of that GPL library?
While the number of court cases involving closed-source software that
merely links against GPL libraries may be quite small, the very
existence of the LGPL should give you a pretty clear idea of the viral
*intent* of the GPL. :)

Don't forget that it's very specifically *derivation followed by
distribution,* not modification, that's important to the GPL.  You are
always free to *modify* GPL'd software, and to use it internally,
modified or not, for any purpose that suits you.  It's once you
*distribute* software derived from GPL'd software that your code must
also be made available under the GPL.

There may be a grey area in cases in which there's an SPI-type boundary
(e.g. JDBC) between commercial applications and GPL libraries, however
it's a grey area that would be very expensive to clear up.  Nobody wants
to spend that money, or even to be *observed* trying to litigate their
way to clarity in this area for their product.

My rules of thumb are:

1. Obviously, if any distribution of your product includes a GPL'd
library, and your product links to the library, *whether statically or
dynamically*, it's a derivative work and your product is now GPL'd.

2. If any distribution of your product includes a GPL'd library, and
your product links to that library indirectly, e.g. across an SPI
boundary, you're in the grey area.  You don't want to be there.

3. If you *always* distribute your product without any GPL'd libraries, and:
  * the "client" side of the SPI is redistributable under license terms
that are satisfactory to you, and;
  * your product will not work until the end user chooses, licenses and
installs their own copy of an implementation of the SPI;
  * your product will work substantially correctly with more than one
SPI implementation other than a special GPL'd one;
...you're *probably* safe.

As someone else has pointed out in this thread, the GPL has caused the
emergence, likely unintended, of a brand new software business model:
dual-licensed (GPL and commercial) libraries with a single copyright
holder.  Qt and MySQL are interesting examples of this.  Derived FOSS
that's GPL-compatible can go nuts; commercial users must buy a license.

-0xe1a

P.S. I hope Sun re-licenses the MySQL JDBC driver under the LGPL someday.