|
View:
New views
20 Messages
—
Rating Filter:
Alert me
|
| < Prev | 1 - 2 - 3 - 4 - 5 - 6 | Next > |
|
|
Re: Groovy runs code in static initializers during compileJochen Theodorou wrote:
> Guillaume Laforge schrieb: >> Interesting. >> Perhaps we could reuse ASM for super-fast crawling in class files to >> find all the relevant information we need. > > then we need two modes, compilation from files and compilation from > unknown sources. If compiled from files the compiler can expect to find > a .class file for each class and will use getRessource to find that > file. In the other case the compiler will use loadClass, as it does > already... I think we can control that through the compiler > configuration. The default for groovyc would be to compile from files > and the default for compiling during runtime would be to load the class > if a .class file was not found. What do you think? I'm still not clear why you want to use loadClass. Are you compiling based on remote resources or something? What classes do you need to find with loadClass that you couldn't find with getResource? - Charlie --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileCharles Oliver Nutter schrieb:
> Jochen Theodorou wrote: >> Guillaume Laforge schrieb: >>> Interesting. >>> Perhaps we could reuse ASM for super-fast crawling in class files to >>> find all the relevant information we need. >> >> then we need two modes, compilation from files and compilation from >> unknown sources. If compiled from files the compiler can expect to >> find a .class file for each class and will use getRessource to find >> that file. In the other case the compiler will use loadClass, as it >> does already... I think we can control that through the compiler >> configuration. The default for groovyc would be to compile from files >> and the default for compiling during runtime would be to load the >> class if a .class file was not found. What do you think? > > I'm still not clear why you want to use loadClass. Are you compiling > based on remote resources or something? it is one ability to do so, yes. Remote resources and databases or other "sources" that have no real class representation. And yes, I know at last one user who used that. > What classes do you need to find > with loadClass that you couldn't find with getResource? well if there is no file oriented binary representation, then getResource will not be of much help. bye blackdrag -- Jochen "blackdrag" Theodorou Groovy Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 5/2/07, Jochen Theodorou <blackdrag@...> wrote:
> [...] > then we need two modes, compilation from files and compilation from > unknown sources. If compiled from files the compiler can expect to find > a .class file for each class and will use getRessource to find that > file. In the other case the compiler will use loadClass, as it does > already... I think we can control that through the compiler > configuration. The default for groovyc would be to compile from files > and the default for compiling during runtime would be to load the class > if a .class file was not found. What do you think? Yeah, right, if we don't find a .class file locally or in a jar file, we need to load the class. Does the other loadClass signature calls the static initializer as well as the normal method or not? -- Guillaume Laforge Groovy Project Manager http://glaforge.free.fr/blog/groovy --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileQuoting Jochen Theodorou <blackdrag@...>:
> then we need two modes, compilation from files and compilation from > unknown sources. If compiled from files the compiler can expect to find > a .class file for each class and will use getRessource to find that > file. In the other case the compiler will use loadClass, as it does > already... I think we can control that through the compiler > configuration. The default for groovyc would be to compile from files > and the default for compiling during runtime would be to load the class > if a .class file was not found. What do you think? That doesn't work; malicious code could still execute code while the compiler runs. The compiler must *always* read the .class file by some other means than the classloader. Otherwise, it will be impossible to contain security risks (for example, in online apps where groovy code is generated at runtime). Also, I don't like the idea of mode switches if there is another way to do it. Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://www.pdark.de/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileQuoting Jochen Theodorou <blackdrag@...>:
>> I'm still not clear why you want to use loadClass. Are you >> compiling based on remote resources or something? > > it is one ability to do so, yes. Remote resources and databases or > other "sources" that have no real class representation. And yes, I know > at last one user who used that. > well if there is no file oriented binary representation, then > getResource will not be of much help. :-/ If the class is already visible in the classloader (ie. the compilation doesn't trigger *additional* classloading which can't be controlled), I guess the security risks of this approach are okay. I'll have to think some more about this. To solve my issues here, I'd offer to patch ResolveVisitor to allow to read the last modified date from the class file without using reflection. I've chosen this class because a) it's relatively simple to test, b) there is little impact on other things. Would you accept such a patch? Can someone please send me the Eclipse code formatter settings to use with Groovy? Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://www.pdark.de/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileAaron Digulla schrieb:
> Quoting Jochen Theodorou <blackdrag@...>: > >> then we need two modes, compilation from files and compilation from >> unknown sources. If compiled from files the compiler can expect to find >> a .class file for each class and will use getRessource to find that >> file. In the other case the compiler will use loadClass, as it does >> already... I think we can control that through the compiler >> configuration. The default for groovyc would be to compile from files >> and the default for compiling during runtime would be to load the class >> if a .class file was not found. What do you think? > > That doesn't work; malicious code could still execute code while the > compiler runs. The compiler must *always* read the .class file by some > other means than the classloader. > > Otherwise, it will be impossible to contain security risks (for example, > in online apps where groovy code is generated at runtime). > > Also, I don't like the idea of mode switches if there is another way to > do it. But there is no other way. For example I could at runitme define a class in Groovy or with something else and then subclass from this class in Groovy. Now this would not compile if I need to compile against the class file. There is simply no class file! the only way to get that class is using loadclass. And bevpr you say that does not happen.. it can happen when you use the console or evaluate. Depending on class files means that Groovy will lose abilities. So there is no other way, not in general. bye blackdrag -- Jochen "blackdrag" Theodorou Groovy Tech Lead (http://groovy.codehaus.org) http://blackdragsview.blogspot.com/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileQuoting Guillaume Laforge <glaforge@...>:
> Does the other loadClass signature calls the static initializer as > well as the normal method or not? When you look at the stacktraces of the bug, you can see that loadClass() doesn't matter since Groovy isn't calling it directly. So even if the other method would solve the problem (which I doubt or the Eclipse guys would have used that instead of inventing the wheel again), we can't make Java Reflection use it. Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://www.pdark.de/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 3 May 2007, at 11:49, Jochen Theodorou wrote: > Aaron Digulla schrieb: >> That doesn't work; malicious code could still execute code while >> the compiler runs. The compiler must *always* read the .class file >> by some other means than the classloader... > > But there is no other way. For example I could at runitme define a > class in Groovy or with something else and then subclass from this > class in Groovy. Now this would not compile if I need to compile > against the class file. There is simply no class file! the only way > to get that class is using loadclass. ... Maybe I'm missing the point, but I think there's some confusion here. I've taken a careful look at the Java Language Specification (see Chapter 12), and the JVM makes a clear distinction between loading, linking, and initializing a class Loading a class with the class loader is not the same as initializing the class, and should not cause any code to be executed - according to section 12.4.1 of the JLS: > Initialization of a class consists of executing its static > initializers and the initializers for static fields declared in the > class. > > [...] > > A class or interface type T will be initialized immediately before > the first occurrence of any one of the following: > T is a class and an instance of T is created. > T is a class and a static method declared by T is invoked. > A static field declared by T is assigned. > A static field declared by T is used and the field is not a > constant variable (§4.12.4). > T is a top-level class, and an assert statement (§14.10) lexically > nested within T is executed. > Invocation of certain reflective methods in class Class and in > package java.lang.reflect also causes class or interface > initialization. A class or interface will not be initialized under > any other circumstance. Now, clearly there is a need for the Groovy compiler to get hold of a Class object at compile-time without causing the class to be initialized, but that should be perfectly possible unless the Groovy compiler needs to do one of the above. I don't see why the compiler would need to do any the things listed that involve the type T directly, but initialization can also occur for the following reason: > Invocation of certain reflective methods in class Class and in > package java.lang.reflect also causes class or interface > initialization. Unfortunately, the JLS doesn't say which methods, but looking at the relevant documentation, the obvious method is Class.forName(String) - however, as others have already pointed out on this thread, if you don't want the class to be initialized, you can call Class.forName (String, boolean, ClassLoader) instead: > forName > > public static Class<?> forName(String name, > boolean initialize, > ClassLoader loader) > throws ClassNotFoundException > > Returns the Class object associated with the class or interface > with the given string name, using the given class loader. [...] The > class is initialized only if the initialize parameter is true and > if it has not been initialized earlier. missing the point, but according to my reading of the Java Language Specification and the relevant API documentation, a correctly implemented class loader does not cause classes to be initialized when you call loadClass, and you can get hold of an uninitialized version of a class using the extended version of forName. So why is the Groovy compiler causing classes to be initialized at compile-time, and why can't it use Class.forName to avoid this problem from occurring? Robert --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 3 May 2007, at 11:56, Aaron Digulla wrote: > Quoting Guillaume Laforge <glaforge@...>: > >> Does the other loadClass signature calls the static initializer as >> well as the normal method or not? > > When you look at the stacktraces of the bug, you can see that > loadClass() doesn't matter since Groovy isn't calling it directly. More confusion I'm afraid - the two versions of loadClass are to do with whether the class gets linked, not whether it gets initialized. The flag that controls whether a class gets initialized is on the Class.forName method. > So even if the other method would solve the problem (which I doubt > or the Eclipse guys would have used that instead of inventing the > wheel again), we can't make Java Reflection use it. I don't know why the Eclipse people didn't use Class.forName, but I think it would have solved the problem they described in the message you posted - the compiler needs to use a different class loader for the classes it is compiling, but that should be sufficient to keep the two names spaces apart in my opinion, unless I'm missing something and the problem is more subtle... Robert --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileQuoting Robert Stroud <R.J.Stroud@...>:
> So why is the Groovy compiler causing classes to be initialized at > compile-time, and why can't it use Class.forName to avoid this problem > from occurring? One place is the ResolveVisitor which tries to find out the last modification from the class. There, it has to initialize the class because the "constant" is initialized in the static initializer of the class. Without that, the constant is 0. I've attached a small piece of demo code how to use ASM to resolve the constant's value without actually loading the class to GROOVY-1863. There are a few issues with the code, though: Groovy creates *3* long constants for the lastModification time. I've arbitrarily chosen the first one. Also, my code depends on the ordering of the bytecode. If someone can insert code to fill a long constant before the lmod-code, it will fail. Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://www.pdark.de/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileQuoting Robert Stroud <R.J.Stroud@...>:
>> When you look at the stacktraces of the bug, you can see that >> loadClass() doesn't matter since Groovy isn't calling it directly. > > More confusion I'm afraid - the two versions of loadClass are to do > with whether the class gets linked, not whether it gets initialized. > The flag that controls whether a class gets initialized is on the > Class.forName method. Some facts just to get everyone on the same level: a) Class.forName ("...") runs code in the static init part b) Class.forName ("...", false, Thread.currentThread().getContextClassLoader()) does not run code in the static init part c) Reflection will init the class if it isn't already So the main problem here is that the Groovy compiler is depending on reflection to examine class files instead of doing it's own thing like any other Java compiler does. Ripping reflection out of groovyc is possible but probably more than a "minor change". Looking at the code I had to write to make ResolveVisitor.getTimeStamp() work again, there will be a couple of "ripple effects" (where a change in place X causes N changes in a lot of other places). Other places I found in the same file: findStaticField() forces the class to init itself. Then, we have ClassNode which is full of reflection code. MethodNode. FieldNode. Probably anything which extends AnnotatedNode. This looks like ripping the backbone out of groovyc. And since this is a major security hole, the question at hand isn't "Do we want to do this?" but "Do we have to do this for 1.1?" Mr. Laforge, your shot. In the meantime, I'll let my subconscious work on some ideas. Maybe it's possible to move the static init code into a static method which is just called. That might allow to separate safe from unsafe stuff and partially init the class so we can still use reflection. A second idea is to write a reflection-like API which uses the bytecode or, when that isn't available, falls back to the real class, so everyone can still use a single API which does the switching automatically depending on whether it can find a class file or not. Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://www.pdark.de/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 3 May 2007, at 13:40, Aaron Digulla wrote: > Quoting Robert Stroud <R.J.Stroud@...>: > >> So why is the Groovy compiler causing classes to be initialized at >> compile-time, and why can't it use Class.forName to avoid this >> problem >> from occurring? > > One place is the ResolveVisitor which tries to find out the last > modification from the class. There, it has to initialize the class > because the "constant" is initialized in the static initializer of > the class. Without that, the constant is 0. Thanks - however, why can't the compiler check the modification times on the actual files? I imagine that is what the Java compiler must do in order to determine whether a source file needs to be re-compiled, otherwise it would suffer from the same problem. > I've attached a small piece of demo code how to use ASM to resolve > the constant's value without actually loading the class to > GROOVY-1863.... Is that ClassExaminer.groovy? I was just looking at it, but I was puzzled by the fact that it didn't contain any static initializers. What has happened to original example that is supposed to illustrate the bug? Robert --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileQuoting Robert Stroud <R.J.Stroud@...>:
>>> So why is the Groovy compiler causing classes to be initialized at >>> compile-time, and why can't it use Class.forName to avoid this problem >>> from occurring? >> >> One place is the ResolveVisitor which tries to find out the last >> modification from the class. There, it has to initialize the class >> because the "constant" is initialized in the static initializer of >> the class. Without that, the constant is 0. > > Thanks - however, why can't the compiler check the modification times > on the actual files? Because, as someone else here has pointed out, there isn't always a file. When you generate code on the fly, you just pass a string to the groovy compiler. The resulting class(es) are not backed by files but they still have a compilation time. >> I've attached a small piece of demo code how to use ASM to resolve >> the constant's value without actually loading the class to >> GROOVY-1863.... > > Is that ClassExaminer.groovy? Yes. > I was just looking at it, but I was > puzzled by the fact that it didn't contain any static initializers. > What has happened to original example that is supposed to illustrate > the bug? An admin must have deleted it. Anyone? Should I attach it again or is there a problem with it? Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://www.pdark.de/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 3 May 2007, at 14:14, Aaron Digulla wrote: > Quoting Robert Stroud <R.J.Stroud@...>: > >>>> So why is the Groovy compiler causing classes to be initialized at >>>> compile-time, and why can't it use Class.forName to avoid this >>>> problem >>>> from occurring? >>> >>> One place is the ResolveVisitor which tries to find out the last >>> modification from the class. There, it has to initialize the >>> class because the "constant" is initialized in the static >>> initializer of the class. Without that, the constant is 0. >> >> Thanks - however, why can't the compiler check the modification times >> on the actual files? > > Because, as someone else here has pointed out, there isn't always a > file. When you generate code on the fly, you just pass a string to > the groovy compiler. The resulting class(es) are not backed by > files but they still have a compilation time. OK - but in that case, you presumably know that the class is up to date, and there's nothing to be re-compiled. If you only called isSourceNewer for classes that had source code on disk, then you'd be able to compare modification times without poking around in the guts of the Class file. Could that be achieved with a bit of refactoring? Thanks. Robert --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 3 May 2007, at 14:14, Aaron Digulla wrote: >> What has happened to original example that is supposed to illustrate >> the bug? > > An admin must have deleted it. Anyone? Should I attach it again or > is there a problem with it? Please could you put it back to avoid confusion. Thanks, Robert --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 3 May 2007, at 13:59, Aaron Digulla wrote:
No - not all reflective operations will cause the class to be initialised, but some certainly will...
at java.lang.reflect.Field.get(Field.java:358) at org.codehaus.groovy.control.ResolveVisitor.getTimeStamp(ResolveVisitor.java:275) at org.codehaus.groovy.control.ResolveVisitor.isSourceNewer(ResolveVisitor.java:298) at org.codehaus.groovy.control.ResolveVisitor.resolveToScript(ResolveVisitor.java:327) But if you only call "isSourceNewer" when you know that a source file exists, then there ought to be a class file as well, so you should be able to check the file modification times directly. |
|
|
Re: Groovy runs code in static initializers during compileJochen Theodorou wrote:
> But there is no other way. For example I could at runitme define a class > in Groovy or with something else and then subclass from this class in > Groovy. Now this would not compile if I need to compile against the > class file. There is simply no class file! the only way to get that > class is using loadclass. And bevpr you say that does not happen.. it > can happen when you use the console or evaluate. > > Depending on class files means that Groovy will lose abilities. So there > is no other way, not in general. Perhaps I'm being naive here, but why do you have to load classes that are referenced at all? You still do dynamic dispatch on them, yes? What would be lost if you saved the class referencing until runtime? - Charlie --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 3 May 2007, at 14:52, Charles Oliver Nutter wrote: > Jochen Theodorou wrote: >> But there is no other way. For example I could at runitme define a >> class in Groovy or with something else and then subclass from this >> class in Groovy. Now this would not compile if I need to compile >> against the class file. There is simply no class file! the only >> way to get that class is using loadclass. And bevpr you say that >> does not happen.. it can happen when you use the console or evaluate. >> Depending on class files means that Groovy will lose abilities. So >> there is no other way, not in general. > > Perhaps I'm being naive here, but why do you have to load classes > that are referenced at all? You still do dynamic dispatch on them, > yes? What would be lost if you saved the class referencing until > runtime? Well, quite. This very "feature" causes problems that I detailed a few weeks ago, where Grails for has problems with compiled code in src/ that tries to reference domain classes etc. Groovyc will compile these referenced classes to disk even though we didn't ask it to... so we have to ... ugh... delete the class files after running groovy for classes we know are part of the grails-app/ tree. This is because we inject code and groovy compile time for these classes normally, which groovyc is not aware of and if it creates .class files for us, Grails can't replace them at load time with the "instrumented" versions. groovyc SomeFile.groovy ...should -only- produce SomeFile.class, SomeFile$xxxx.class and any other classes that are contained in that source file, period. I believe this isn't currently the case. Marc --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileOn 3 May 2007, at 14:14, Aaron Digulla wrote: > Quoting Robert Stroud <R.J.Stroud@...>: > >> why can't the compiler check the modification times >> on the actual files? > > Because, as someone else here has pointed out, there isn't always a > file. When you generate code on the fly, you just pass a string to > the groovy compiler. The resulting class(es) are not backed by > files but they still have a compilation time. But it looks to me as though the relevant code only compares the modification times if the source file exists - if the code has been generated on the fly, then there won't be a source file, so there will be nothing to compare. The problem is that isSourceNewer takes a URL and a Class as an argument rather than two URLs - hence, it seems to me that the easiest solution to this particular problem would be to add a getClassFile method to GroovyResourceLoader, and then compare the modification times on the two URLs, with appropriate code to deal with a situation where one or other of the files doesn't exist. In other words, I would rewrite the relevant parts of resolveToScript to be something like this: URL sourceURL = null; URL compiledURL = null; try { sourceURL = gcl.getResourceLoader().loadGroovySource(name); if ( sourceURL != null ) { compiledURL = gcl.getResourceLoader().loadGroovyClass(name); } } catch (MalformedURLException e) { // fall through and let both URLs be null } if (sourceURL !=null) { if (type.isResolved()) { // if the file is not newer we don't want to recompile if (!isSourceNewer(sourceURL, compiledURL)) return true; cachedClasses.remove(type.getName()); type.setRedirect(null); } ... } The code for loadGroovyClass would be more or less identical to the code for localGroovySource - both could call a version of getSourceFile that took the appropriate file extension as an argument. Of course, there may be other reasons why the Groovy compiler needs to initialize classes during compilation, but I don't see why it's necessary in this particular situation. Robert --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
|
|
Re: Groovy runs code in static initializers during compileQuoting Robert Stroud <R.J.Stroud@...>:
>> Because, as someone else here has pointed out, there isn't always a >> file. When you generate code on the fly, you just pass a string to >> the groovy compiler. The resulting class(es) are not backed by >> files but they still have a compilation time. > > OK - but in that case, you presumably know that the class is up to > date, and there's nothing to be re-compiled. If you only called > isSourceNewer for classes that had source code on disk, then you'd be > able to compare modification times without poking around in the guts of > the Class file. The same code in GroovyClassLoader, too. blackdrag wrote that code, he should know why groovy needs this information. As far as I can see, this code it used to calculate whether a class needs to be recompiled. Regards, -- Aaron "Optimizer" Digulla a.k.a. Philmann Dark "It's not the universe that's limited, it's our imagination. Follow me and I'll show you something beyond the limits." http://www.pdark.de/ --------------------------------------------------------------------- To unsubscribe from this list please visit: http://xircles.codehaus.org/manage_email |
| < Prev | 1 - 2 - 3 - 4 - 5 - 6 | Next > |
| Free embeddable forum powered by Nabble | Forum Help |