XMLParser: attribute gets truncated when updated

[Gael Marziou]

Hello,

I'm very new to Groovy so I suppose that I misunderstood something.

I wrote a little test to update an attribute of an XML node and it does
not work: new value gets truncated to the size of the previous value.

def input = '''<tag id="12" />'''
def tag = new XmlParser().parseText(input)
tag.attribute("id").value = "ABCD"
// id contains "AB" it has been truncated to original size
assert tag.attribute("id").size() == 4 : tag.attribute("id")

I suppose that the error is on line 3 but I don't understand it.

Thanks,

Gael

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jim White]

Holy smokes Batman!

I've always know that Groovy's disregard for private scope is a VERY BAD
THING.

But I've never bothered to work up some cases to demonstrate that fact.

The error on line 3 is that you're getting the private field 'value' of
a java.lang.String object.  That field happens to be an array of char
and when you assign a new value to it then the value of the string
changes.  Of course that doesn't have any effect on the length attribute
of the string.

This violation of private permits untold havoc and means all classes,
not just java.lang.String, are deprived of assurance that their API
contracts are enforced.  But such a succinct case is quite compelling.

def s = '12'
s.value = 'abcd'
println s
==>
ab

As for how to change the id attribute of a groovy.util.Node, I'm not
sure, although there are some docs on it.

Jim

Gael Marziou wrote:

> Hello,
>
> I'm very new to Groovy so I suppose that I misunderstood something.
>
> I wrote a little test to update an attribute of an XML node and it does
> not work: new value gets truncated to the size of the previous value.
>
> def input = '''<tag id="12" />'''
> def tag = new XmlParser().parseText(input)
> tag.attribute("id").value = "ABCD"
> // id contains "AB" it has been truncated to original size
> assert tag.attribute("id").size() == 4 : tag.attribute("id")
>
> I suppose that the error is on line 3 but I don't understand it.
>
> Thanks,
>
> Gael

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Mike Dillon-4]

begin Jim White quotation:
> This violation of private permits untold havoc and means all classes,
> not just java.lang.String, are deprived of assurance that their API
> contracts are enforced.  But such a succinct case is quite compelling.

I brought this up a while ago:

    http://www.nabble.com/Access-modifiers-in-Groovy-to13099409.html

-md

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jim White]

Mike Dillon wrote:

> begin Jim White quotation:
>
>>This violation of private permits untold havoc and means all classes,
>>not just java.lang.String, are deprived of assurance that their API
>>contracts are enforced.  But such a succinct case is quite compelling.
>
> I brought this up a while ago:
>
>     http://www.nabble.com/Access-modifiers-in-Groovy-to13099409.html

Indeed, although the case is more compelling with 'ABC' than '[ "A",
"B", "C" ] as char[]'.

I also noted this problem in an earlier (but different) topic:

http://www.nabble.com/Re%3A-Why-does-Groovy-do-object-conversion--p15587047.html

And I should repeat here what I said there, please vote for this issue:

http://jira.codehaus.org/browse/GROOVY-1875

I'm confused now though, because at that time I got the impression that
only private methods were exposed, and I thought I had made a test for
myself that showed fields were protected.  I think though I must have
made a mistake there somewhere.

Ah, I see.  I was misled by the fact that private fields do not show up
in the property list of a class and mistakenly assumed that you couldn't
set a property that isn't in the list.  I'll update the issue.

As I say there, it is time to close the gaping hole and should do it for
1.6.  If it is too big an incompatiblity, and some users can't deal with
it right away (this being so close to time for 1.6RC1), then we can have
a switch to enable the old behavior for them.

Jim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Scott Hickey-3]

I assume you understand that private access is possible in Java using reflection.
 
Scott

----- Original Message ----
From: Jim White <jim@...>
To: user@...
Sent: Friday, April 18, 2008 10:22:22 PM
Subject: Re: [groovy-user] Yikes! String is mutable! (was Re: [groovy-user] XMLParser: attribute gets truncated when updated)

Mike Dillon wrote:

> begin Jim White quotation:
>
>>This violation of private permits untold havoc and means all classes,
>>not just java.lang.String, are deprived of assurance that their API
>>contracts are enforced.  But such a succinct case is quite compelling.
>
> I brought this up a while ago:
>
>     http://www.nabble.com/Access-modifiers-in-Groovy-to13099409.html

Indeed, although the case is more compelling with 'ABC' than '[ "A",
"B", "C" ] as char[]'.

I also noted this problem in an earlier (but different) topic:

http://www.nabble.com/Re%3A-Why-does-Groovy-do-object-conversion--p15587047.html

And I should repeat here what I said there, please vote for this issue:

http://jira.codehaus.org/browse/GROOVY-1875

I'm confused now though, because at that time I got the impression that
only private methods were exposed, and I thought I had made a test for
myself that showed fields were protected.  I think though I must have
made a mistake there somewhere.

Ah, I see.  I was misled by the fact that private fields do not show up
in the property list of a class and mistakenly assumed that you couldn't
set a property that isn't in the list.  I'll update the issue.

As I say there, it is time to close the gaping hole and should do it for
1.6.  If it is too big an incompatiblity, and some users can't deal with
it right away (this being so close to time for 1.6RC1), then we can have
a switch to enable the old behavior for them.

Jim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email







---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jim White]

Scott Hickey wrote:

> I assume you understand that private access is possible in Java using reflection.
>  
> Scott

Yes, of course.  But it is under the control of the security manager.
That is how access to private stuff should remain.

Groovy doing it by default and with no way to prevent it is seriously
wrong.  I assume you saw the totally ordinary and in no way contrived
case that started this thread?

Jim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Gael Marziou]

Thanks for the somehow frightening explanations, it helped me to find a
workaround.

I changed the buggy line :

tag.attribute("id").value = "ABCD"

to this:

tag.attributes().id = "ABCD"

Is there a better way?

Thanks,

Gael

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jim White]

Gael Marziou wrote:

> Thanks for the somehow frightening explanations, it helped me to find a
> workaround.
>
> I changed the buggy line :
>
> tag.attribute("id").value = "ABCD"
>
> to this:
>
> tag.attributes().id = "ABCD"
>
> Is there a better way?

That looks good to me.  You're getting the map of attributes and setting
the element 'id' to a new value.  It is equivalent to this (less
pleasing) statement:

tag.attributes()['id'] = "ABCD"

Ah, and you've jogged my memory (I don't use this Groovy XML/DOM jazz
much).  There are GPath expressions.  So you can also do this:

tag.'@id' = "ABCD"

or even:

tag.@id = "ABCD"

Although the wiki says that is for when using XmlSlurper, and so I'm not
sure when you can't use it.  There are some differences when using W3C
DOM although they can be adjusted for by using a category.

http://groovy.codehaus.org/GPath

http://groovy.codehaus.org/Processing+XML

More details in the GinA (Groovy in Action) book, which I recommend.

Jim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Gael Marziou]

Thanks Jim, I really like this one, it works fine and is so concise:

tag.@id = "ABCD"

The only strange thing is that the compiler rejects this syntax in a
"switch" while it accepts it in an "if " statement.

Gael.


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jochen Theodorou]

Jim White schrieb:
[...]
> As I say there, it is time to close the gaping hole and should do it for
> 1.6.  If it is too big an incompatiblity, and some users can't deal with
> it right away (this being so close to time for 1.6RC1), then we can have
> a switch to enable the old behavior for them.

The problem is that the current MOP is not good enough to resolve this
kind of situation. Let me show you an example:


class X {
   private foo
   def bar() {
     return {this.@foo}
   }
   def setFoo(x){foo=x}
}

def x = new X()
x.setFoo(1)
assert x.bar().call() == 1

i this case it is quite clear, that this.@foo means the private field,
we can resolve it statically and last theoretically we could solve this
case... Now let us change the closure a little:

class X {
   private foo
   def bar() {
     return {foo}
   }
   def setFoo(x){foo=x}
}

def x = new X()
x.setFoo(1)
assert x.bar().call() == 1

it is more or less the same code, but now foo is a property in the first
place. Meaning that the lookup will be done using getProperty in
Closure. This method does not transport the caller class, so we can not
verify that the closure has access to foo.

The whole thing is totally blowing up when you do:

class X {
   private foo
   def bar() {
     return {foo}
   }
   def setFoo(x){foo=x}
}
class Y extends X{}

def x = new Y()
x.setFoo(1)
assert x.bar().call() == 1

even with the current information stored in the MOP and not going
through getPropterty, we still have the problem that the caller class is
actually the Closure and not X. But since the private field foo is now
only available in X and not in Y this example will fail even in current
Groovy. And what counts for getProperty partially counts for
invokeMethod too, but the problem is not as big there, as invokeMethod
is usually only called if the normal method call fails. Of course that
also means that Interceptable successfully deletes any access
information during the method call.

Not to mention that transporting the type information and checking
access rights is bad for the runtime performance

bye blackdrag

--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jim White]

Yes, Java has the same issues with private members and inner classes.

While dealing with all of the finer points of that will have to wait for
<http://jira.codehaus.org/browse/GROOVY-69>, we can take some of their
tricks such as providing package scope access to private members when
needed.

Which in the current compiler probably means always, so rather than
creating hidden methods as Java does, I'd say just have private compiled
as package.

Groovy already doesn't respect private, so there is no loss by compiling
them to package, and there is great gain in restoring the meaning of
private.

And Java code is not going to suddenly run into improper access issues
because it does respect package scope, and for code in the same package
the access situation would be effectively the same as now.

Jim

Jochen Theodorou wrote:

> Jim White schrieb:
> [...]
>
>> As I say there, it is time to close the gaping hole and should do it
>> for 1.6.  If it is too big an incompatiblity, and some users can't
>> deal with it right away (this being so close to time for 1.6RC1), then
>> we can have a switch to enable the old behavior for them.
>
>
> The problem is that the current MOP is not good enough to resolve this
> kind of situation. Let me show you an example:
>
>
> class X {
>   private foo
>   def bar() {
>     return {this.@foo}
>   }
>   def setFoo(x){foo=x}
> }
>
> def x = new X()
> x.setFoo(1)
> assert x.bar().call() == 1
>
> i this case it is quite clear, that this.@foo means the private field,
> we can resolve it statically and last theoretically we could solve this
> case... Now let us change the closure a little:
>
> class X {
>   private foo
>   def bar() {
>     return {foo}
>   }
>   def setFoo(x){foo=x}
> }
>
> def x = new X()
> x.setFoo(1)
> assert x.bar().call() == 1
>
> it is more or less the same code, but now foo is a property in the first
> place. Meaning that the lookup will be done using getProperty in
> Closure. This method does not transport the caller class, so we can not
> verify that the closure has access to foo.
>
> The whole thing is totally blowing up when you do:
>
> class X {
>   private foo
>   def bar() {
>     return {foo}
>   }
>   def setFoo(x){foo=x}
> }
> class Y extends X{}
>
> def x = new Y()
> x.setFoo(1)
> assert x.bar().call() == 1
>
> even with the current information stored in the MOP and not going
> through getPropterty, we still have the problem that the caller class is
> actually the Closure and not X. But since the private field foo is now
> only available in X and not in Y this example will fail even in current
> Groovy. And what counts for getProperty partially counts for
> invokeMethod too, but the problem is not as big there, as invokeMethod
> is usually only called if the normal method call fails. Of course that
> also means that Interceptable successfully deletes any access
> information during the method call.
>
> Not to mention that transporting the type information and checking
> access rights is bad for the runtime performance
>
> bye blackdrag
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jochen Theodorou]

Jim White schrieb:

> Yes, Java has the same issues with private members and inner classes.
>
> While dealing with all of the finer points of that will have to wait for
> <http://jira.codehaus.org/browse/GROOVY-69>, we can take some of their
> tricks such as providing package scope access to private members when
> needed.
>
> Which in the current compiler probably means always, so rather than
> creating hidden methods as Java does, I'd say just have private compiled
> as package.
>
> Groovy already doesn't respect private, so there is no loss by compiling
> them to package, and there is great gain in restoring the meaning of
> private.

private currently does not mean the method is hidden if called on an
instance of a class that declares that method. True, but if you
subclass, then the method is hidden. compiling to package will also mean
the method is hidden in subclasses, but what do we gain from this? Isn't
the method still available in a subclass I call from another class in
the same package? Also the current access is not really package scoped.
In fact package scoping is a big problem for us, because checking
package strings will slow the method call down pretty much.

> And Java code is not going to suddenly run into improper access issues
> because it does respect package scope, and for code in the same package
> the access situation would be effectively the same as now.

why should Java code call private methods all of a sudden?


  bye blackdrag


--
Jochen "blackdrag" Theodorou
The Groovy Project Tech Lead (http://groovy.codehaus.org)
http://blackdragsview.blogspot.com/
http://www.g2one.com/

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Scott Hickey-3]

I'm just curious to know if your String example in your original post is something that happened in the course of coding on a project by yourself or someone one your team, or are you trying to be proactive for possible pitfalls? I know you said it wasn't contrived but I am trying to understand the context in which someone tried to assign something to value on a String. In three years of working with a project team that has spans a wide range of Java experience, I've seem some unexpected things but I haven't seen this personally.

I know this is a emotionally charged topic and I'm not for clarification to fan the flames. I have an interest in tracking hypothetical problems or pitfalls with Groovy versus ones actually encountered in the course of project work.


Scott

----- Original Message ----
From: Jim White <jim@...>
To: user@...
Sent: Saturday, April 19, 2008 12:17:14 AM
Subject: Re: [groovy-user] Yikes! String is mutable! (was Re: [groovy-user] XMLParser: attribute gets truncated when updated)

Scott Hickey wrote:

> I assume you understand that private access is possible in Java using reflection.
>  
> Scott

Yes, of course.  But it is under the control of the security manager.
That is how access to private stuff should remain.

Groovy doing it by default and with no way to prevent it is seriously
wrong.  I assume you saw the totally ordinary and in no way contrived
case that started this thread?

Jim


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email






---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jim White]

The original post was the first in the thread I replied to.

http://markmail.org/message/qppcm7ksqe4akceu

The summary being you get a result from a function and think it has a
property named 'value', you want to change it so you do.  The oringal
poster thinking 'attribute' returned some Node object or the like.

Bad news if the value happens to be java.lang.String or anything else
that has a property named 'value' that is private.  Obviously the error
can occur on either side of the function call.  And with Groovy being
dynamically typed, there is no way to prevent this from happening.

As I said to start with, this is clearly a bad situation but hadn't
bothered to search for compelling examples.  But here it is just tossed
over the transom.

For myself, I just don't write code that is bugged, so it isn't a
problem... ;-)

Jim

Scott Hickey wrote:

> I'm just curious to know if your String example in your original post
> is something that happened in the course of coding on a project by
> yourself or someone one your team, or are you trying to be proactive
> for possible pitfalls? I know you said it wasn't contrived but I am
> trying to understand the context in which someone tried to assign
> something to value on a String. In three years of working with a
> project team that has spans a wide range of Java experience, I've
> seem some unexpected things but I haven't seen this personally.
>
> I know this is a emotionally charged topic and I'm not for
> clarification to fan the flames. I have an interest in tracking
> hypothetical problems or pitfalls with Groovy versus ones actually
> encountered in the course of project work.
>
>
> Scott
>
> ----- Original Message ---- From: Jim White <jim@...> To:
> user@... Sent: Saturday, April 19, 2008 12:17:14 AM
> Subject: Re: [groovy-user] Yikes! String is mutable! (was Re:
> [groovy-user] XMLParser: attribute gets truncated when updated)
>
> Scott Hickey wrote:
>
>
>> I assume you understand that private access is possible in Java
>> using reflection.
>>
>> Scott
>
>
> Yes, of course.  But it is under the control of the security manager.
>  That is how access to private stuff should remain.
>
> Groovy doing it by default and with no way to prevent it is seriously
>  wrong.  I assume you saw the totally ordinary and in no way
> contrived case that started this thread?
>
> Jim

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email