Groovy syntax /API enhancement

[mingfai]

hi,

after programmed Groovy for a while, I've got a wish list for syntax/API improvement.

1. find() (or findAll()) to take a "limit" integer to decide how many records shall be returned.
   e.g. [1,2,3,4].find(2){it>1} == [2,3]
   
   
2. support !in or not in
   e.g. 3 !in [2,3,4] == false
   Remarks: we have to use !(3 in [2,3,4]) now, !3 in [2,3,4] doesn't work.
   
   
3. support << to put data to map.
   e.g. ([key1:"value1"] << [key2:"value2"])  ==  [key1:"value1",key2:"value2"]
   Remarks: this is just more elegant but it's bad idea as it encourages people to create a new map at the right hand side.
   
   
4. support everyWithIndex, anyWithIndex, collectWithIndex, sumWithIndex etc. it's even better if the original every, any has an implicit variable/syntax for retrieving the index
   e.g. [10,20].collectWithIndex{ it,i -> it*(i+1) } = [ 10, 40 ]
   
   I actually prefer all methods for collections to support the index parameter by default, without a need to use a different WithIndex method.
   
   
5. In any collection closure, e.g. each, every, any etc., support it.next() and it.prev()/previous() and it.sibing(int) to retrieve the next item, return null if not existed (but no nullpointerexception)
   e.g. [1,2,3,2].sort().find{ it == it.silbling(-1) } == 2

Trust me, I'm not such a demanding man when I wrote Java! :-)

Anyone second my suggestions? any more ideas?

Regards,
mingfai

[Barzilai Spinak-2]

Mingfai wrote:

> [...]
> support everyWithIndex, anyWithIndex, collectWithIndex, sumWithIndex
> etc. it's even better if the original every, any has an implicit
> variable/syntax for retrieving the index
>
>    1. e.g. [10,20].collectWithIndex{ it,i -> it*(i+1) } = [ 10, 40 ]
>
>       I actually prefer all methods for collections to support the
>       index parameter by default, without a need to use a different
>       WithIndex method.
>

The problem here is that you are assuming that those methods (every,
any, collect, etc) are *iterating* over the collection with some
predetermined order, in sequence. Nothing in the semantics says it
should be so. They could be even be doing it in parallel.

>    1. In any collection closure, e.g. each, every, any etc., support
>       it.next() and it.prev()/previous() and it.sibing(int) to
>       retrieve the next item, return null if not existed (but no
>       nullpointerexception)
>       e.g. [1,2,3,2].sort().find{ it == it.silbling(-1) } == 2
>
>
I have no idea how you think this could be implemented. You are calling
a method (sibling) on the contained element (which could "belong" to
many different collections and have references from many places in the
code). The method should be called on the container, or delegate of the
closure... but I think it's too much trouble.


BarZ

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jochen Theodorou]

Mingfai schrieb:
> hi,
>
> after programmed Groovy for a while, I've got a wish list for syntax/API
> improvement.
>
>    1. find() (or findAll()) to take a "limit" integer to decide how many
>       records shall be returned.
>       e.g. [1,2,3,4].find(2){it>1} == [2,3]

you could work on a sublist.

>    2. support !in or not in
>       e.g. 3 !in [2,3,4] == false
>       Remarks: we have to use !(3 in [2,3,4]) now, !3 in [2,3,4] doesn't
>       work.

(3 !in in [2,3,4]) is equal to (!(3 in [2,3,4]))

>    3. support << to put data to map.
>       e.g. ([key1:"value1"] << [key2:"value2"])  ==
>       [key1:"value1",key2:"value2"]
>       Remarks: this is just more elegant but it's bad idea as it
>       encourages people to create a new map at the right hand side.

true... not to forget that you can do: map[key2] = value2

>    4. support everyWithIndex, anyWithIndex, collectWithIndex,
>       sumWithIndex etc. it's even better if the original every, any has
>       an implicit variable/syntax for retrieving the index
>       e.g. [10,20].collectWithIndex{ it,i -> it*(i+1) } = [ 10, 40 ]

I am against all these withIndex methods... they are polluting the API
so much. I would like to have a more general solution, but I can't think
of any yet

[...]
>    5. In any collection closure, e.g. each, every, any etc., support
>       it.next() and it.prev()/previous() and it.sibing(int) to retrieve
>       the next item, return null if not existed (but no
>       nullpointerexception)
>       e.g. [1,2,3,2].sort().find{ it == it.silbling(-1) } == 2

"it" is no iterator, "it" is the actual value. Adding these methods to
the value for example by using categories may hide an existing method

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[mingfai]

thx BarZ and Jochen for your comments.

I made those suggestions completely from the users (API users) point of view without considering how (difficult) to implement or how optimize is the performance. I didn't make up those suggestions from my mind but from my Groovy programming code, I feel if Groovy supports those syntax, my code could be cleaner.

for 1..3, they certainly can be done in another way but it's not as simple/elegant as my suggested way.

• It doesn't look like a big issue if it's just "!(x in ['x'])", but when the line is longer and more complex and when use together with other parenthesis, skipping one pair of parenthesis means a lot in code clarity. It's just like in SQL, we can add a NOT to NOT LIKE or NOT IN.
  
  
• find(x) vs sublist, they are different. say i have a 10,000 item list, my find return 5,000, and I want to get the first 100. if i use sublist, the code is longer, as well as I need to construct a list of 5000, and then sublist to a list of 100. It seems to me the using an enhanced find could potentially be more optimized by Groovy. (initially, it could be just a wrapping of sublist, but there are chance to optimize)
  
  
• [:] << [k,'v'] , i actually suspect it is possible for the compiler to optimize it than create a new map then join the map. It's just a more consistent usage of left shift. For a list, stream, stringbuffer we can use << to add data, why not a map? I expect the compiler could transparently translate it to x.put( k, v). I am using map[ k ] = value, and I feel map << [ key, value ] looks more clean.
  

for 4..5, I did assume the collection is in sequence and is iteratable. It seems to me Groovy collections are usually in sequence by default. e.g. [] is ArrayList and [:].getClass() is LinkedHashMap, so in the majority of case, collection have order and have index. e.g. [] as Set is used, then the index will just be unavailable or always return -1.

using it.next() is just a stupid idea. The key point is there should be an issue way to get a sibling. Say, if i want to check if a list has duplicated value, I may use:
  boolean isDuplicated = mylist.sort().any{ it == mylist[mylist.indexOf(it)+1]   }

When I wrote the above line, I think Groovy is suppose to provide a more intelligent way such as :
  boolean isDuplicated = mylist.sort().any{ it == nextValue()  }
(as Groovy is much more intelligent than Java,  I got an expectation that it has to be extremely intelligent! :-) )

My example was not meant to explore how to check duplicated value, but there are many cases that we would want to do collection closure activity that involves siblings. I also wonder the compiler could optimize the performance as it knows I'm calling nextValue(), it can read the current and next value and there is no need to do a looking and retrieval again. I have no knowledge of how a compiler works, and perhaps such optimization possibility is just a myth.

If we can assume the majority of collection are in sequence, then I imagine it should be possible to dynamically add a method or variable to the Closure to get the previous or next value. (Ok, I admit it.next() is stupid! :-) ) For the minority of case such as Set, the prevValue() , nextValue() will just return null. I don't really know how the language is implemented. At least, it should be possible to assign a parameter for prev and next just like the XXXWithIndex can use one more parameter for the index.

regards,
mingfai

On Mon, Apr 21, 2008 at 7:20 AM, Jochen Theodorou <blackdrag@...> wrote:

Mingfai schrieb:

hi,

after programmed Groovy for a while, I've got a wish list for syntax/API improvement.

  1. find() (or findAll()) to take a "limit" integer to decide how many

     records shall be returned.
     e.g. [1,2,3,4].find(2){it>1} == [2,3]

you could work on a sublist.

  2. support !in or not in

     e.g. 3 !in [2,3,4] == false
     Remarks: we have to use !(3 in [2,3,4]) now, !3 in [2,3,4] doesn't
     work.

(3 !in in [2,3,4]) is equal to (!(3 in [2,3,4]))

  3. support << to put data to map.

     e.g. ([key1:"value1"] << [key2:"value2"])  ==      [key1:"value1",key2:"value2"]
     Remarks: this is just more elegant but it's bad idea as it
     encourages people to create a new map at the right hand side.

true... not to forget that you can do: map[key2] = value2

  4. support everyWithIndex, anyWithIndex, collectWithIndex,

     sumWithIndex etc. it's even better if the original every, any has
     an implicit variable/syntax for retrieving the index
     e.g. [10,20].collectWithIndex{ it,i -> it*(i+1) } = [ 10, 40 ]

I am against all these withIndex methods... they are polluting the API so much. I would like to have a more general solution, but I can't think of any yet

[...]

  5. In any collection closure, e.g. each, every, any etc., support

     it.next() and it.prev()/previous() and it.sibing(int) to retrieve
     the next item, return null if not existed (but no
     nullpointerexception)
     e.g. [1,2,3,2].sort().find{ it == it.silbling(-1) } == 2

"it" is no iterator, "it" is the actual value. Adding these methods to the value for example by using categories may hide an existing method

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email

[Jochen Theodorou]

Mingfai schrieb:

> thx BarZ and Jochen for your comments.
>
> I made those suggestions completely from the users (API users) point of
> view without considering how (difficult) to implement or how optimize is
> the performance. I didn't make up those suggestions from my mind but
> from my Groovy programming code, I feel if Groovy supports those syntax,
> my code could be cleaner.
>
> for 1..3, they certainly can be done in another way but it's not as
> simple/elegant as my suggested way.
>
>     * It doesn't look like a big issue if it's just "!(x in ['x'])", but
>       when the line is longer and more complex and when use together
>       with other parenthesis, skipping one pair of parenthesis means a
>       lot in code clarity. It's just like in SQL, we can add a NOT to
>       NOT LIKE or NOT IN.

sure, that is true, you have the same for example for instanceof. But
"!in" looks irritating to me, "not in" would be much better, but I am
not too sure we can put that in our grammar.

>     * find(x) vs sublist, they are different. say i have a 10,000 item
>       list, my find return 5,000,  and I want to get the first 100. if i
>       use sublist, the code is longer, as well as I need to construct a
>       list of 5000, and then sublist to a list of 100.

ok, there is a misunderstanding... you don't want to iterate the whole
thing and break after for example 100 matches where found. normally this
is something for "break", which does not work in Groovy 1.x.. hmm

>     * [:] << [k,'v'] , i actually suspect it is possible for the
>       compiler to optimize it than create a new map then join the map.

this kind of optimizations are quite troublesome in a dynamic language.
Especially if an operator like << can change its meaning during runtime

> for 4..5, I did assume the collection is in sequence and is iteratable.
> It seems to me Groovy collections are usually in sequence by default.
> e.g. [] is ArrayList and [:].getClass() is LinkedHashMap, so in the
> majority of case, collection have order and have index. e.g. [] as Set
> is used, then the index will just be unavailable or always return -1.
>
> using it.next() is just a stupid idea. The key point is there should be
> an issue way to get a sibling. Say, if i want to check if a list has
> duplicated value, I may use:
>   boolean isDuplicated = mylist.sort().any{ it ==
> *mylist[mylist.indexOf(it)+1]*   }
>
> When I wrote the above line, I think Groovy is suppose to provide a more
> intelligent way such as :
>   boolean isDuplicated = mylist.sort().any{ it == *nextValue()*  }
> (as Groovy is much more intelligent than Java,  I got an expectation
> that it has to be extremely intelligent! :-) )

Well... there are some problems with your example... first, sort will
modify the list itself, next, you could do:

isDuplicated = mylist.size()==mylist.unique().size()

to realize your idea, Groovy would have to give some kind of Iterator
object and there lies a basic problem, because we use the normal
Iterator provided by the list. There is no peeking. If you compare to
the previous element, then you can use inject:

mylist.inject(null) { value, item -> isDuplicated |= item==value; item }

of course this solution has several downsides.... isDuplicated is set
inside the closure, which means you see the place it is set not very
good. And an additional the initial element given to inject has to be
something that is not in the list, which may require an additional
line... hmm... there might be another way too:

isDuplicated = mylist.inject([false,null]) { value, item ->
  [ value[0] |= item==value[1], item ]
}[0]

well, of course that is not as nice as it could be with some kind of
iterator that allows direct access... and of course this solution is a
bit different too ;)

> My example was not meant to explore how to check duplicated value, but
> there are many cases that we would want to do collection closure
> activity that involves siblings.

can you give more examples?

> I also wonder the compiler could
> optimize the performance as it knows I'm calling nextValue(),

you should forget about the compiler knowing something and improving
performance. This isn't a macro and it is no static language

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Tom Nichols]

On Sun, Apr 20, 2008 at 7:20 PM, Jochen Theodorou <blackdrag@...> wrote:

> >   4. support everyWithIndex, anyWithIndex, collectWithIndex,
> >
> >      sumWithIndex etc. it's even better if the original every, any has
> >      an implicit variable/syntax for retrieving the index
> >      e.g. [10,20].collectWithIndex{ it,i -> it*(i+1) } = [ 10, 40 ]
> >
>
>  I am against all these withIndex methods... they are polluting the API so
> much. I would like to have a more general solution, but I can't think of any
> yet

Can't you just modify the each/any/collect methods to test how many
parameters the closure takes?  If the closure only takes one param
assume it is the item.  If it takes two params, assume (item, index).
No?

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[glaforge]

On Mon, Apr 21, 2008 at 3:32 PM, Tom Nichols <tmnichols@...> wrote:
> [...]
>  >  I am against all these withIndex methods... they are polluting the API so
>  > much. I would like to have a more general solution, but I can't think of any
>  > yet
>
>  Can't you just modify the each/any/collect methods to test how many
>  parameters the closure takes?  If the closure only takes one param
>  assume it is the item.  If it takes two params, assume (item, index).
>  No?

That's what we're already doing for some of the methods like each().
But it'd be nice to expand this approach to all methods for which it may apply.

--
Guillaume Laforge
Groovy Project Manager
G2One, Inc. Vice-President Technology
http://www.g2one.com

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jochen Theodorou]

Tom Nichols schrieb:

> On Sun, Apr 20, 2008 at 7:20 PM, Jochen Theodorou <blackdrag@...> wrote:
>
>>>   4. support everyWithIndex, anyWithIndex, collectWithIndex,
>>>
>>>      sumWithIndex etc. it's even better if the original every, any has
>>>      an implicit variable/syntax for retrieving the index
>>>      e.g. [10,20].collectWithIndex{ it,i -> it*(i+1) } = [ 10, 40 ]
>>>
>>  I am against all these withIndex methods... they are polluting the API so
>> much. I would like to have a more general solution, but I can't think of any
>> yet
>
> Can't you just modify the each/any/collect methods to test how many
> parameters the closure takes?

well, if you look at Map, then yo have a problem for "each", because one
argument means an iteration using the entry, 2 arguments means key and
value. There is no place for a each-version with two arguments meaning
the entry and index.  I am thinking currently about an index generating
closure, that calls your work closure with an additional parameter:

def index = 0
def indexGen = { closure, Object[] par ->
   closure.call(*par, index)
   index++
}

['a','b','c'].each indexGen.curry {item,index ->
                   println "$index: $item"}

with the ouput:
"""
0: a
1: b
2: c
"""

of course this is ugly on first sight... but indexGen is a generic piece
of code that could work with any closure... or be used as a method on
the closure:

['a','b','c'].each {item,index -> println "$index: $item"}.indexed

where indexed is a property on the closure that will return a new
closure containing the index. then it would work on *any* closure as
part of the closure, having less methods, but not loosing the
functionality. As that it can then be uased for each, any, find,
findAll, inject, collect ....

the only thing that keeps me from removing all withIndex methods from
DGM (and some others as well) and adding this to Closure is that I am
not fully satisfied with the syntax. Also in a GPath expression it might
be a bit confusing... so I am still searching for the "perfect" solution.

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email