[scala] So close... yet so far
|
View:
New views
10 Messages
—
Rating Filter:
Alert me
|
 |
[scala] So close... yet so far
by Kevin Wright-4
::
Rate this Message:
I'm having great success with my compiler plugin, and now proudly
announce that I can generate synthetic methods. They work fantastically and I can happily call them from other scala classes and get the expected behaviour. Just one tiny niggly problem... If I compile a class "Foo" with synthetic methods in, I can then stick it on the classpath for compiling "Bar", which works just fantastically. If I try to compile both Foo and Bar at the same time though, Bar can't see my generated methods :( Before I spend any more time on this, does anyone know if it's at least theoretically possible to make it work for a plugin that executes immediately after typer? |
|
|
Re: [scala] So close... yet so far
by Lukas Rytz
::
Rate this Message:
Kevin,
you would need to run the namer on "Foo" before running the typer on "Bar". This basically means that you'd need to add your synthetics before the namer, which however might not be possible for you (there's no type information on trees yet..). The best mechanism to add a synthetic definition early is to use Namer.enterSyntheticSym. I don't know if it's possible to use it from a plugin. And again, you don't have type information yet when you do it so early. That's actually the reason the BeanSetter is not handled by "enterSyntheticSym", the parameter type for the bean setter is still unknown (given a field definition "val property = 10"), and we can't build the corresponding tree. So in this case, the compiler only generates the Symbol for the bean setter, and the trees are created later (Typer.addGetterSetter) when types are known. Lukas
On Wed, Nov 4, 2009 at 23:58, Kevin Wright <kev.lee.wright@...> wrote: I'm having great success with my compiler plugin, and now proudly |
|
|
Re: [scala] So close... yet so far
by Kevin Wright-4
::
Rate this Message:
Ouch!
I have seen lots of stuff around namer for handling such synthetics as @beanproperty, case classes and companion objects. So I was starting to wonder a bit... One of my first thoughts was to run the plugin between namer and typer, but that's not possible given that typer runsRightAfter namer. Guess I could always run directly after parser, but at that level it's pretty hardcore / low level and little more than a code generator (for example, I can't do anything tricky using the interface of a trait independently of the implementation) Is it possible to do a limited early-run of namer/typer within my phase if I run directly after parser? Alternatively, would it be possible to postpone Bar or to force it to be re-processed through typer after synthetics have been added to Foo (I accept that this would cause a performance hit) The only other way is to have a more generalised extension point for synthetics, I can see that this would also help clean up existing code in the compiler. On Thu, Nov 5, 2009 at 6:46 AM, Lukas Rytz <lukas.rytz@...> wrote: > Kevin, > > you would need to run the namer on "Foo" before running the typer on > "Bar". This basically means that you'd need to add your synthetics before > the namer, which however might not be possible for you (there's no type > information on trees yet..). > > The best mechanism to add a synthetic definition early is to use > Namer.enterSyntheticSym. I don't know if it's possible to use it from > a plugin. > > And again, you don't have type information yet when you do it so early. > That's actually the reason the BeanSetter is not handled by > "enterSyntheticSym", > the parameter type for the bean setter is still unknown (given a field > definition > "val property = 10"), and we can't build the corresponding tree. > So in this case, the compiler only generates the Symbol for the bean setter, > and the trees are created later (Typer.addGetterSetter) when types are > known. > > > Lukas > > On Wed, Nov 4, 2009 at 23:58, Kevin Wright <kev.lee.wright@...> > wrote: >> >> I'm having great success with my compiler plugin, and now proudly >> announce that I can generate synthetic methods. >> They work fantastically and I can happily call them from other scala >> classes and get the expected behaviour. >> >> Just one tiny niggly problem... >> >> If I compile a class "Foo" with synthetic methods in, I can then stick >> it on the classpath for compiling "Bar", which works just >> fantastically. >> If I try to compile both Foo and Bar at the same time though, Bar >> can't see my generated methods :( >> >> Before I spend any more time on this, does anyone know if it's at >> least theoretically possible to make it work for a plugin that >> executes immediately after typer? > > |
|
|
Re: [scala] So close... yet so far
by Lukas Rytz
::
Rate this Message:
On Thu, Nov 5, 2009 at 08:03, Kevin Wright <kev.lee.wright@...> wrote: Ouch! Do you think removing this constraint would help? You should try that out. Guess I could always run directly after parser, but at that level it's Maybe, have a look at "compileLate" in global, it does something like that. Alternatively, would it be I don't think postponing is possible..
For early synthetics (definitions which have to be visible to other code) we use "enterSyntheticSym". For late synthetics (where no new members are added, only overrides, e.g. case class "toString") we use addSyntheticMethods (in typechecker/SyntheticMethods.scala). It's easy to do both in compiler plugins. The only hard case is when you actually need type information for constructing your trees, but want to add new synthetics which should be visible. It's logical in some sense that this is hard, because you want to add something before type checking, but you need information from after. It would be very useful if there was a standard way to achieve this (enter symbols early, but add trees only later). Lukas
|
|
|
Re: [scala] So close... yet so far
by Chris Twiner
::
Rate this Message:
Having a generalised synthetic solution would be very cool and revive my plugin but I guess that would be 3.0 territory if at all possible. On Nov 5, 2009 8:04 AM, "Kevin Wright" <kev.lee.wright@...> wrote: |
|
|
Re: [scala] So close... yet so far
by Kevin Wright-4
::
Rate this Message:
On Thu, Nov 5, 2009 at 7:43 AM, Lukas Rytz <lukas.rytz@...> wrote:
> > > On Thu, Nov 5, 2009 at 08:03, Kevin Wright <kev.lee.wright@...> > wrote: >> >> Ouch! >> >> I have seen lots of stuff around namer for handling such synthetics as >> @beanproperty, case classes and companion objects. So I was starting >> to wonder a bit... >> >> One of my first thoughts was to run the plugin between namer and >> typer, but that's not possible given that typer runsRightAfter namer. > > Do you think removing this constraint would help? You should try > that out. I've raised this issue before. If my understanding is correct, this constraint had to be added as the two phases don't strictly execute one after the other, instead control passes back and forth between namer and typer. My compiler-fu isn't yet good enough to understand exactly what's going on here, but it definitely looks ripe for further investigation. >> >> Guess I could always run directly after parser, but at that level it's >> pretty hardcore / low level and little more than a code generator (for >> example, I can't do anything tricky using the interface of a trait >> independently of the implementation) >> >> Is it possible to do a limited early-run of namer/typer within my >> phase if I run directly after parser? > > Maybe, have a look at "compileLate" in global, it does something like > that. This could well be the solution I need. I'd be very interested to see what performance penalty is in having to run the compiler a second time over classes that use my synthesised methods. My biggest fear here is that it could be non-trivial to identify where this needs to happen. >> >> Alternatively, would it be >> possible to postpone Bar or to force it to be re-processed through >> typer after synthetics have been added to Foo (I accept that this >> would cause a performance hit) > > I don't think postponing is possible.. Fair point... If Bar is named/typed by the compiler before Foo then re-processing is the only real possibility. >> >> The only other way is to have a more generalised extension point for >> synthetics, I can see that this would also help clean up existing code >> in the compiler. > > For early synthetics (definitions which have to be visible to other code) > we use "enterSyntheticSym". For late synthetics (where no new members > are added, only overrides, e.g. case class "toString") we use > addSyntheticMethods (in typechecker/SyntheticMethods.scala). My understanding is that enterSyntheticSym is a method defined in typer. So I'd either need to extend typer (not possible in a plugin to my knowledge), or redo typing in my phase and then trigger a cascade reprocessing of all classes that might then use the synthetics, and all the classes that use those classes, and so on... > It's easy to do both in compiler plugins. The only hard case is when you > actually need type information for constructing your trees, but want to add > new synthetics which should be visible. It's logical in some sense that this > is hard, because you want to add something before type checking, but you > need information from after. Guess what... :) > It would be very useful if there was a standard way to achieve this (enter > symbols early, but add trees only later). Well, I'm definitely not adverse to comitting some code in nsc to help out, especially if it's going to prove useful to others. I think that a post-typer phase would probably be the approach to take here (let's call it generateSynthetics). It would definitely help simplify the existing handling of case class and beanproperties, and I'd also add the ability to append extra interfaces to a class in this phase. The alternative would be a mechanism to extend the behaviour of the existing namer & typer phases, but I do appreciate that this would be a pretty major extension to the existing plugin mechanism. I also know that typing is one of the trickiest areas of the compiler, so if there's a fundamental reason why either of these approaches wouldn't work then I'd be really grateful to anyone who points that out *before* I put in the necessary time and effort. > Lukas > > > > >> >> >> On Thu, Nov 5, 2009 at 6:46 AM, Lukas Rytz <lukas.rytz@...> wrote: >> > Kevin, >> > >> > you would need to run the namer on "Foo" before running the typer on >> > "Bar". This basically means that you'd need to add your synthetics >> > before >> > the namer, which however might not be possible for you (there's no type >> > information on trees yet..). >> > >> > The best mechanism to add a synthetic definition early is to use >> > Namer.enterSyntheticSym. I don't know if it's possible to use it from >> > a plugin. >> > >> > And again, you don't have type information yet when you do it so early. >> > That's actually the reason the BeanSetter is not handled by >> > "enterSyntheticSym", >> > the parameter type for the bean setter is still unknown (given a field >> > definition >> > "val property = 10"), and we can't build the corresponding tree. >> > So in this case, the compiler only generates the Symbol for the bean >> > setter, >> > and the trees are created later (Typer.addGetterSetter) when types are >> > known. >> > >> > >> > Lukas >> > >> > On Wed, Nov 4, 2009 at 23:58, Kevin Wright >> > <kev.lee.wright@...> >> > wrote: >> >> >> >> I'm having great success with my compiler plugin, and now proudly >> >> announce that I can generate synthetic methods. >> >> They work fantastically and I can happily call them from other scala >> >> classes and get the expected behaviour. >> >> >> >> Just one tiny niggly problem... >> >> >> >> If I compile a class "Foo" with synthetic methods in, I can then stick >> >> it on the classpath for compiling "Bar", which works just >> >> fantastically. >> >> If I try to compile both Foo and Bar at the same time though, Bar >> >> can't see my generated methods :( >> >> >> >> Before I spend any more time on this, does anyone know if it's at >> >> least theoretically possible to make it work for a plugin that >> >> executes immediately after typer? >> > >> > > > |
|
|
Re: [scala] So close... yet so far
by Lukas Rytz
::
Rate this Message:
On Thu, Nov 5, 2009 at 11:07, Kevin Wright <kev.lee.wright@...> wrote:
I think that's not entirely true. There are clearly two phases which traverse the trees. Somebody please correct the errors in the following: Namer traversal: creates symbols for top-level definitions and assigns them a lazy type. When being completed, these lazy types use facilities from the typer (convert type signature trees into types, compute return types, etc). Typer traversal: type-checks all bodies. Creates symbols where necessary (which probably uses facilities from the namer). I think if one knows what's going on and is careful enough, running phases between namer and typer should be ok. My compiler-fu isn't yet good enough to understand Wouldn't the first traversal of "Bar" yield error messages (synthetics in Foo are not found!)?
No, enterSyntheticSym is defined in the namer. So I'd either need to extend typer (not possible in a plugin You'd still need to create the symbols before the typer, right? It would definitely help simplify the |
|
|
Re: [scala] So close... yet so far
by Kevin Wright-4
::
Rate this Message:
On Thu, Nov 5, 2009 at 1:54 PM, Lukas Rytz <lukas.rytz@...> wrote:
> > > On Thu, Nov 5, 2009 at 11:07, Kevin Wright <kev.lee.wright@...> > wrote: >> >> On Thu, Nov 5, 2009 at 7:43 AM, Lukas Rytz <lukas.rytz@...> wrote: >> > >> > >> > On Thu, Nov 5, 2009 at 08:03, Kevin Wright >> > <kev.lee.wright@...> >> > wrote: >> >> >> >> Ouch! >> >> >> >> I have seen lots of stuff around namer for handling such synthetics as >> >> @beanproperty, case classes and companion objects. So I was starting >> >> to wonder a bit... >> >> >> >> One of my first thoughts was to run the plugin between namer and >> >> typer, but that's not possible given that typer runsRightAfter namer. >> > >> > Do you think removing this constraint would help? You should try >> > that out. >> >> I've raised this issue before. If my understanding is correct, this >> constraint had to be added as the two phases don't strictly execute >> one after the other, instead control passes back and forth between >> namer and typer. > > I think that's not entirely true. There are clearly two phases which > traverse > the trees. > > Somebody please correct the errors in the following: > > Namer traversal: creates symbols for top-level definitions and assigns them > a lazy > type. When being completed, these lazy types use facilities from the typer > (convert type signature trees into types, compute return types, etc). > > Typer traversal: type-checks all bodies. Creates symbols where necessary > (which > probably uses facilities from the namer). > > I think if one knows what's going on and is careful enough, running phases > between namer and typer should be ok. > Looking at the way @BeanProperty works, it definitely seems to be the right thing that synthetics should be handled by both namer and typer, so being able to run a phase between the may well be the answer. But you're right about having to "be careful", I suspect that this is the runsRightAfter contract was used here. I think that any changes here would represent something needing an SIP. > >> >> My compiler-fu isn't yet good enough to understand >> exactly what's going on here, but it definitely looks ripe for further >> investigation. >> >> >> >> >> Guess I could always run directly after parser, but at that level it's >> >> pretty hardcore / low level and little more than a code generator (for >> >> example, I can't do anything tricky using the interface of a trait >> >> independently of the implementation) >> >> >> >> Is it possible to do a limited early-run of namer/typer within my >> >> phase if I run directly after parser? >> > >> > Maybe, have a look at "compileLate" in global, it does something like >> > that. >> >> This could well be the solution I need. I'd be very interested to see >> what performance penalty is in having to run the compiler a second >> time over classes that use my synthesised methods. My biggest fear >> here is that it could be non-trivial to identify where this needs to >> happen. >> >> >> >> >> Alternatively, would it be >> >> possible to postpone Bar or to force it to be re-processed through >> >> typer after synthetics have been added to Foo (I accept that this >> >> would cause a performance hit) >> > >> > I don't think postponing is possible.. >> >> Fair point... If Bar is named/typed by the compiler before Foo then >> re-processing is the only real possibility. > > Wouldn't the first traversal of "Bar" yield error messages (synthetics in > Foo are not found!)? Error reporting doesn't get postponed until compilation is finished? That could throw a spanner in the works... >> >> >> >> >> The only other way is to have a more generalised extension point for >> >> synthetics, I can see that this would also help clean up existing code >> >> in the compiler. >> > >> > For early synthetics (definitions which have to be visible to other >> > code) >> > we use "enterSyntheticSym". For late synthetics (where no new members >> > are added, only overrides, e.g. case class "toString") we use >> > addSyntheticMethods (in typechecker/SyntheticMethods.scala). >> >> My understanding is that enterSyntheticSym is a method defined in >> typer. > > No, enterSyntheticSym is defined in the namer. My bad :( The problem/solution still remain unchanged though... >> >> So I'd either need to extend typer (not possible in a plugin >> to my knowledge), or redo typing in my phase and then trigger a >> cascade reprocessing of all classes that might then use the >> synthetics, and all the classes that use those classes, and so on... >> >> > It's easy to do both in compiler plugins. The only hard case is when you >> > actually need type information for constructing your trees, but want to >> > add >> > new synthetics which should be visible. It's logical in some sense that >> > this >> > is hard, because you want to add something before type checking, but you >> > need information from after. >> >> Guess what... :) >> >> > It would be very useful if there was a standard way to achieve this >> > (enter >> > symbols early, but add trees only later). >> >> >> Well, I'm definitely not adverse to comitting some code in nsc to help >> out, especially if it's going to prove useful to others. I think that >> a post-typer phase would probably be the approach to take here (let's >> call it generateSynthetics). > > You'd still need to create the symbols before the typer, right? Ideally, there'd be a way to do them as part of the typer. In that regard, I would consider generateSynthetics to be typer part II > >> >> It would definitely help simplify the >> existing handling of case class and beanproperties, and I'd also add >> the ability to append extra interfaces to a class in this phase. The >> alternative would be a mechanism to extend the behaviour of the >> existing namer & typer phases, but I do appreciate that this would be >> a pretty major extension to the existing plugin mechanism. >> >> I also know that typing is one of the trickiest areas of the compiler, >> so if there's a fundamental reason why either of these approaches >> wouldn't work then I'd be really grateful to anyone who points that >> out *before* I put in the necessary time and effort. >> >> > Lukas >> > >> > >> > >> > >> >> >> >> >> >> On Thu, Nov 5, 2009 at 6:46 AM, Lukas Rytz <lukas.rytz@...> wrote: >> >> > Kevin, >> >> > >> >> > you would need to run the namer on "Foo" before running the typer on >> >> > "Bar". This basically means that you'd need to add your synthetics >> >> > before >> >> > the namer, which however might not be possible for you (there's no >> >> > type >> >> > information on trees yet..). >> >> > >> >> > The best mechanism to add a synthetic definition early is to use >> >> > Namer.enterSyntheticSym. I don't know if it's possible to use it from >> >> > a plugin. >> >> > >> >> > And again, you don't have type information yet when you do it so >> >> > early. >> >> > That's actually the reason the BeanSetter is not handled by >> >> > "enterSyntheticSym", >> >> > the parameter type for the bean setter is still unknown (given a >> >> > field >> >> > definition >> >> > "val property = 10"), and we can't build the corresponding tree. >> >> > So in this case, the compiler only generates the Symbol for the bean >> >> > setter, >> >> > and the trees are created later (Typer.addGetterSetter) when types >> >> > are >> >> > known. >> >> > >> >> > >> >> > Lukas >> >> > >> >> > On Wed, Nov 4, 2009 at 23:58, Kevin Wright >> >> > <kev.lee.wright@...> >> >> > wrote: >> >> >> >> >> >> I'm having great success with my compiler plugin, and now proudly >> >> >> announce that I can generate synthetic methods. >> >> >> They work fantastically and I can happily call them from other scala >> >> >> classes and get the expected behaviour. >> >> >> >> >> >> Just one tiny niggly problem... >> >> >> >> >> >> If I compile a class "Foo" with synthetic methods in, I can then >> >> >> stick >> >> >> it on the classpath for compiling "Bar", which works just >> >> >> fantastically. >> >> >> If I try to compile both Foo and Bar at the same time though, Bar >> >> >> can't see my generated methods :( >> >> >> >> >> >> Before I spend any more time on this, does anyone know if it's at >> >> >> least theoretically possible to make it work for a plugin that >> >> >> executes immediately after typer? >> >> > >> >> > >> > >> > > > |
|
|
Re: [scala] So close... yet so far
by Lukas Rytz
::
Rate this Message:
I don't know, maybe not. After all it only affects plugins. But that's for others to decide :)
Nope.. :( That could throw a spanner in the works...
Doesn't work: given the example ("Bar" uses synthetics from "Foo"), when the typer traverses "Bar" the synthetic symbols need to be here already. Otherwise an error will be issued. |
|
|
Re: [scala] So close... yet so far
by Kevin Wright-4
::
Rate this Message:
Okay, I'm bumping up to scala-internals as this one's getting a bit
deep now, and certainly involves FAR too much uncertainty for my liking :) Is there anyone actively working on compiler internals with a good understanding of the issues involved here? Better still, somone who's already considered this and has a clear picture as to how it might already be possible, or how the compiler needs to evolve such that synthetics of this variety can be cleanly implemented. On Thu, Nov 5, 2009 at 3:07 PM, Lukas Rytz <lukas.rytz@...> wrote: > >> >> >> One of my first thoughts was to run the plugin between namer and >> >> >> typer, but that's not possible given that typer runsRightAfter >> >> >> namer. >> >> > >> >> > Do you think removing this constraint would help? You should try >> >> > that out. >> >> >> >> I've raised this issue before. If my understanding is correct, this >> >> constraint had to be added as the two phases don't strictly execute >> >> one after the other, instead control passes back and forth between >> >> namer and typer. >> > >> > I think that's not entirely true. There are clearly two phases which >> > traverse >> > the trees. >> > >> > Somebody please correct the errors in the following: >> > >> > Namer traversal: creates symbols for top-level definitions and assigns >> > them >> > a lazy >> > type. When being completed, these lazy types use facilities from the >> > typer >> > (convert type signature trees into types, compute return types, etc). >> > >> > Typer traversal: type-checks all bodies. Creates symbols where necessary >> > (which >> > probably uses facilities from the namer). >> > >> > I think if one knows what's going on and is careful enough, running >> > phases >> > between namer and typer should be ok. >> > >> >> Looking at the way @BeanProperty works, it definitely seems to be the >> right thing that synthetics should be handled by both namer and typer, >> so being able to run a phase between the may well be the answer. But >> you're right about having to "be careful", I suspect that this is the >> runsRightAfter contract was used here. I think that any changes here >> would represent something needing an SIP. > > I don't know, maybe not. After all it only affects plugins. But that's for > others > to decide :) > > >> >> >> Alternatively, would it be >> >> >> possible to postpone Bar or to force it to be re-processed through >> >> >> typer after synthetics have been added to Foo (I accept that this >> >> >> would cause a performance hit) >> >> > >> >> > I don't think postponing is possible.. >> >> >> >> Fair point... If Bar is named/typed by the compiler before Foo then >> >> re-processing is the only real possibility. >> > >> > Wouldn't the first traversal of "Bar" yield error messages (synthetics >> > in >> > Foo are not found!)? >> >> >> Error reporting doesn't get postponed until compilation is finished? > > Nope.. :( > >> >> That could throw a spanner in the works... >> > > >> >> >> > It's easy to do both in compiler plugins. The only hard case is when >> >> > you >> >> > actually need type information for constructing your trees, but want >> >> > to >> >> > add >> >> > new synthetics which should be visible. It's logical in some sense >> >> > that >> >> > this >> >> > is hard, because you want to add something before type checking, but >> >> > you >> >> > need information from after. >> >> >> >> Guess what... :) >> >> >> >> > It would be very useful if there was a standard way to achieve this >> >> > (enter >> >> > symbols early, but add trees only later). >> >> >> >> >> >> Well, I'm definitely not adverse to comitting some code in nsc to help >> >> out, especially if it's going to prove useful to others. I think that >> >> a post-typer phase would probably be the approach to take here (let's >> >> call it generateSynthetics). >> > >> > You'd still need to create the symbols before the typer, right? >> >> Ideally, there'd be a way to do them as part of the typer. >> In that regard, I would consider generateSynthetics to be typer part II > > Doesn't work: given the example ("Bar" uses synthetics from "Foo"), when the > typer traverses "Bar" the synthetic symbols need to be here already. > Otherwise > an error will be issued. > > > |
| Free embeddable forum powered by Nabble | Forum Help |
