Fornax-Platform
Forum

Annoying regeneration problem

View: New views
8 Messages — Rating Filter:   Alert me  

Annoying regeneration problem

by PaloT :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

If I should be honest, what annoy me most now on Sculptor is
regeneration of one-time generated files (main/java main/resources).
Before I had idea with timestamps but this is difficult to handle. Now
I found maybe better solution. After generation we should checksum all
generated files in main/java and main/resources. Than before
generation we should check all sums and if file have same checksum we
should remove it (file wasn't change and we should regenerate it).

I don't know enough details about oAW if I can write some hook method
which can do this for every generated file or I have to write Maven
plugin, which should be part of clean task. Can somebody give me any
hint about oAW or Maven where should I look?

I'm willing to write anything what can help to solve this problem.

TNX

Pavel

PS: Patrik, do we have already some incident about this?

------------------------------------------------------------------------------
_______________________________________________
Fornax-developer mailing list
Fornax-developer@...
https://lists.sourceforge.net/lists/listinfo/fornax-developer

Re: Annoying regeneration problem

by Karsten Thoms :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

A good starting point is to provide a specialized Outlet  
implementation. The implementation class to use must be configured for  
the Maven Mojo to do this.

You should read this article from Heiko about the Generation Gap  
pattern. He suggests to introduce a conditional generated folder,  
where only files are generated if they were not moved to a manually  
maintained folder.
http://www.1160pm.net/2009/04/23/generation-gap-pattern/

~Karsten

Am 02.07.2009 um 13:04 schrieb Pavel Tavoda:

> If I should be honest, what annoy me most now on Sculptor is
> regeneration of one-time generated files (main/java main/resources).
> Before I had idea with timestamps but this is difficult to handle. Now
> I found maybe better solution. After generation we should checksum all
> generated files in main/java and main/resources. Than before
> generation we should check all sums and if file have same checksum we
> should remove it (file wasn't change and we should regenerate it).
>
> I don't know enough details about oAW if I can write some hook method
> which can do this for every generated file or I have to write Maven
> plugin, which should be part of clean task. Can somebody give me any
> hint about oAW or Maven where should I look?
>
> I'm willing to write anything what can help to solve this problem.
>
> TNX
>
> Pavel
>
> PS: Patrik, do we have already some incident about this?
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Fornax-developer mailing list
> Fornax-developer@...
> https://lists.sourceforge.net/lists/listinfo/fornax-developer


------------------------------------------------------------------------------
_______________________________________________
Fornax-developer mailing list
Fornax-developer@...
https://lists.sourceforge.net/lists/listinfo/fornax-developer

Re: Annoying regeneration problem

by PaloT :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Karsten TNX for hint but I found it more complicated than what we
need. I developed Maven plugin which care about removing untouched
files.

Plugin have 3 actions: clean, before-generate, after-generate

I have no idea how to test it, than I developed script 'test.sh' which
is included in attached zip file. Here is also way how to start it and
watch what's going on.

Many questions:
* Patrik, can we make it as one of subprojects (I can change name,
artifactId, ..., I did my best to guess right names)?

* Can somebody help me integrate it right way to sculptor POM? I don't
know how to configure POM.XML that right action (clean,
before-generate, after-generate) is started at right time in build
process.

* Who is willing to test it in more realistic environment?

Have a nice weekend

Pavel


On Fri, Jul 3, 2009 at 11:50 AM, Karsten Thoms<karsten.thoms@...> wrote:

> A good starting point is to provide a specialized Outlet
> implementation. The implementation class to use must be configured for
> the Maven Mojo to do this.
>
> You should read this article from Heiko about the Generation Gap
> pattern. He suggests to introduce a conditional generated folder,
> where only files are generated if they were not moved to a manually
> maintained folder.
> http://www.1160pm.net/2009/04/23/generation-gap-pattern/
>
> ~Karsten
>
> Am 02.07.2009 um 13:04 schrieb Pavel Tavoda:
>
>> If I should be honest, what annoy me most now on Sculptor is
>> regeneration of one-time generated files (main/java main/resources).
>> Before I had idea with timestamps but this is difficult to handle. Now
>> I found maybe better solution. After generation we should checksum all
>> generated files in main/java and main/resources. Than before
>> generation we should check all sums and if file have same checksum we
>> should remove it (file wasn't change and we should regenerate it).
>>
>> I don't know enough details about oAW if I can write some hook method
>> which can do this for every generated file or I have to write Maven
>> plugin, which should be part of clean task. Can somebody give me any
>> hint about oAW or Maven where should I look?
>>
>> I'm willing to write anything what can help to solve this problem.
>>
>> TNX
>>
>> Pavel
>>
>> PS: Patrik, do we have already some incident about this?
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Fornax-developer mailing list
>> Fornax-developer@...
>> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Fornax-developer mailing list
> Fornax-developer@...
> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>


------------------------------------------------------------------------------

_______________________________________________
Fornax-developer mailing list
Fornax-developer@...
https://lists.sourceforge.net/lists/listinfo/fornax-developer

fornax-cartridges-sculptor-maven-plugin-checksum.zip (43K) Download Attachment

Re: Annoying regeneration problem

by PaloT :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Looks like attachment get not through. Here is again with password -
sculptor. And it's ZIP file (stupid Gmail).

Pavel

On Fri, Jul 3, 2009 at 6:35 PM, Pavel Tavoda<pavel.tavoda@...> wrote:

> Karsten TNX for hint but I found it more complicated than what we
> need. I developed Maven plugin which care about removing untouched
> files.
>
> Plugin have 3 actions: clean, before-generate, after-generate
>
> I have no idea how to test it, than I developed script 'test.sh' which
> is included in attached zip file. Here is also way how to start it and
> watch what's going on.
>
> Many questions:
> * Patrik, can we make it as one of subprojects (I can change name,
> artifactId, ..., I did my best to guess right names)?
>
> * Can somebody help me integrate it right way to sculptor POM? I don't
> know how to configure POM.XML that right action (clean,
> before-generate, after-generate) is started at right time in build
> process.
>
> * Who is willing to test it in more realistic environment?
>
> Have a nice weekend
>
> Pavel
>
>
> On Fri, Jul 3, 2009 at 11:50 AM, Karsten Thoms<karsten.thoms@...> wrote:
>> A good starting point is to provide a specialized Outlet
>> implementation. The implementation class to use must be configured for
>> the Maven Mojo to do this.
>>
>> You should read this article from Heiko about the Generation Gap
>> pattern. He suggests to introduce a conditional generated folder,
>> where only files are generated if they were not moved to a manually
>> maintained folder.
>> http://www.1160pm.net/2009/04/23/generation-gap-pattern/
>>
>> ~Karsten
>>
>> Am 02.07.2009 um 13:04 schrieb Pavel Tavoda:
>>
>>> If I should be honest, what annoy me most now on Sculptor is
>>> regeneration of one-time generated files (main/java main/resources).
>>> Before I had idea with timestamps but this is difficult to handle. Now
>>> I found maybe better solution. After generation we should checksum all
>>> generated files in main/java and main/resources. Than before
>>> generation we should check all sums and if file have same checksum we
>>> should remove it (file wasn't change and we should regenerate it).
>>>
>>> I don't know enough details about oAW if I can write some hook method
>>> which can do this for every generated file or I have to write Maven
>>> plugin, which should be part of clean task. Can somebody give me any
>>> hint about oAW or Maven where should I look?
>>>
>>> I'm willing to write anything what can help to solve this problem.
>>>
>>> TNX
>>>
>>> Pavel
>>>
>>> PS: Patrik, do we have already some incident about this?
>>>
>>> ------------------------------------------------------------------------------
>>> _______________________________________________
>>> Fornax-developer mailing list
>>> Fornax-developer@...
>>> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>>
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Fornax-developer mailing list
>> Fornax-developer@...
>> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>>
>


------------------------------------------------------------------------------

_______________________________________________
Fornax-developer mailing list
Fornax-developer@...
https://lists.sourceforge.net/lists/listinfo/fornax-developer

fornax-cartridges-sculptor-maven-plugin-checksum-zip.arch (44K) Download Attachment

Re: Annoying regeneration problem

by Andreas Källberg-2 :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,
I can help test it out, but it will take some time since we are going on holiday in a few days.

I'm not sure how this should work. Is the checksum going to be calculated and persisted in the file tree?
If so, I think it should be persisted in the target directory so it just can be cleaned out in a single sweep and doesn't pollute the source directories.
And I think the process could also be even simpler. We could make a it two step rocket since we always do a clean before we do install so I think it would be enough that the cleaning checks the checksum if the files should be removed (and removes them if so), and the install calculates the checksums again after generation.
About the maven integration:
You need to execute the plugin together with clean lifecycle and with the default lifecycle.
In the clean lifecycle you need to execute it in the 'pre-clean' phase (otherwise the checksum files will be gone).
In the default lifecycle you need to execute it after the 'generate-sources' phase. Perhaps the 'process-sources' will be good.

By this you keep the cleaning in the cleaning lifecycle. The downside is that the files wont be cleaned out if only doing 'mvn install', but as I said, we always do 'mvn clean install' anyway, so...
Have a look at how the fornax-oaw-m2-plugin is configured in pom's.
And, here are all about the lifecycle and phases:
http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

regards.../Andreas

On Fri, Jul 3, 2009 at 6:35 PM, Pavel Tavoda <pavel.tavoda@...> wrote:
Karsten TNX for hint but I found it more complicated than what we
need. I developed Maven plugin which care about removing untouched
files.

Plugin have 3 actions: clean, before-generate, after-generate

I have no idea how to test it, than I developed script 'test.sh' which
is included in attached zip file. Here is also way how to start it and
watch what's going on.

Many questions:
* Patrik, can we make it as one of subprojects (I can change name,
artifactId, ..., I did my best to guess right names)?

* Can somebody help me integrate it right way to sculptor POM? I don't
know how to configure POM.XML that right action (clean,
before-generate, after-generate) is started at right time in build
process.

* Who is willing to test it in more realistic environment?

Have a nice weekend

Pavel


On Fri, Jul 3, 2009 at 11:50 AM, Karsten Thoms<karsten.thoms@...> wrote:
> A good starting point is to provide a specialized Outlet
> implementation. The implementation class to use must be configured for
> the Maven Mojo to do this.
>
> You should read this article from Heiko about the Generation Gap
> pattern. He suggests to introduce a conditional generated folder,
> where only files are generated if they were not moved to a manually
> maintained folder.
> http://www.1160pm.net/2009/04/23/generation-gap-pattern/
>
> ~Karsten
>
> Am 02.07.2009 um 13:04 schrieb Pavel Tavoda:
>
>> If I should be honest, what annoy me most now on Sculptor is
>> regeneration of one-time generated files (main/java main/resources).
>> Before I had idea with timestamps but this is difficult to handle. Now
>> I found maybe better solution. After generation we should checksum all
>> generated files in main/java and main/resources. Than before
>> generation we should check all sums and if file have same checksum we
>> should remove it (file wasn't change and we should regenerate it).
>>
>> I don't know enough details about oAW if I can write some hook method
>> which can do this for every generated file or I have to write Maven
>> plugin, which should be part of clean task. Can somebody give me any
>> hint about oAW or Maven where should I look?
>>
>> I'm willing to write anything what can help to solve this problem.
>>
>> TNX
>>
>> Pavel
>>
>> PS: Patrik, do we have already some incident about this?
>>
>> ------------------------------------------------------------------------------
>> _______________________________________________
>> Fornax-developer mailing list
>> Fornax-developer@...
>> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>
>
> ------------------------------------------------------------------------------
> _______________________________________________
> Fornax-developer mailing list
> Fornax-developer@...
> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>

------------------------------------------------------------------------------

_______________________________________________
Fornax-developer mailing list
Fornax-developer@...
https://lists.sourceforge.net/lists/listinfo/fornax-developer



------------------------------------------------------------------------------

_______________________________________________
Fornax-developer mailing list
Fornax-developer@...
https://lists.sourceforge.net/lists/listinfo/fornax-developer

Re: Annoying regeneration problem

by PaloT :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

New version:
* Simplified file management
* Removed some logical race conditions

Thanks Andreas for interest. How does it work:
* By default only /src/main/java and /src/main/resources are checked
(you can override via checksumDirs variable, look to test.sh)
* For now I have two central files 'checksum.txt' and
'ignore-checksum.txt'. One store checksums for files in format
checksum:file and second is only list of files which are already
considered changed and they are ignored on checksumming and removing.
* action 'clean' is only going through files in checksum.txt and
checking if checksum changed. If yes than checksum is removed and file
is appended to ignore-checksum.txt file
* action 'before-install' is doing 'clean' at begining and than it
looks for file called '.beforeInstall'. If file doesn't exist (normal
status) it's going through all files which still exist after clean
procedure and is appending them to ignore-checksum.txt (this are new
files created by user, as result of normal dev cycle). Than file
'.beforeInstall' is created.
* action 'before-install' is started but file '.beforeInstall' exist
(recovery mode) all files which are not in ignore-checksum.txt are
DELETED. This apply in situations when maven stop at time of
generation (user break, generation error, ...) and new checksum file
was not created
* action 'generate' just generate new checksums to 'checksum.txt' file
and removing '.beforeInstall' file.

Summarize, here is only important rule:
NEVER DO MANUAL CHANGE OR ADD FILES AFTER UNSUCCESSFUL GENERATION

Attached file is ZIP and password is 'sculptor'.

Pavel

2009/7/4 Andreas Källberg <andreas.kallberg@...>:

> Hi,
> I can help test it out, but it will take some time since we are going on
> holiday in a few days.
>
> I'm not sure how this should work. Is the checksum going to be calculated
> and persisted in the file tree?
> If so, I think it should be persisted in the target directory so it just can
> be cleaned out in a single sweep and doesn't pollute the source directories.
> And I think the process could also be even simpler. We could make a it two
> step rocket since we always do a clean before we do install so I think it
> would be enough that the cleaning checks the checksum if the files should be
> removed (and removes them if so), and the install calculates the checksums
> again after generation.
> About the maven integration:
> You need to execute the plugin together with clean lifecycle and with the
> default lifecycle.
> In the clean lifecycle you need to execute it in the 'pre-clean' phase
> (otherwise the checksum files will be gone).
> In the default lifecycle you need to execute it after the 'generate-sources'
> phase. Perhaps the 'process-sources' will be good.
>
> By this you keep the cleaning in the cleaning lifecycle. The downside is
> that the files wont be cleaned out if only doing 'mvn install', but as I
> said, we always do 'mvn clean install' anyway, so...
> Have a look at how the fornax-oaw-m2-plugin is configured in pom's.
> And, here are all about the lifecycle and phases:
> http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
>
> regards.../Andreas
>
> On Fri, Jul 3, 2009 at 6:35 PM, Pavel Tavoda <pavel.tavoda@...> wrote:
>>
>> Karsten TNX for hint but I found it more complicated than what we
>> need. I developed Maven plugin which care about removing untouched
>> files.
>>
>> Plugin have 3 actions: clean, before-generate, after-generate
>>
>> I have no idea how to test it, than I developed script 'test.sh' which
>> is included in attached zip file. Here is also way how to start it and
>> watch what's going on.
>>
>> Many questions:
>> * Patrik, can we make it as one of subprojects (I can change name,
>> artifactId, ..., I did my best to guess right names)?
>>
>> * Can somebody help me integrate it right way to sculptor POM? I don't
>> know how to configure POM.XML that right action (clean,
>> before-generate, after-generate) is started at right time in build
>> process.
>>
>> * Who is willing to test it in more realistic environment?
>>
>> Have a nice weekend
>>
>> Pavel
>>
>>
>> On Fri, Jul 3, 2009 at 11:50 AM, Karsten Thoms<karsten.thoms@...>
>> wrote:
>> > A good starting point is to provide a specialized Outlet
>> > implementation. The implementation class to use must be configured for
>> > the Maven Mojo to do this.
>> >
>> > You should read this article from Heiko about the Generation Gap
>> > pattern. He suggests to introduce a conditional generated folder,
>> > where only files are generated if they were not moved to a manually
>> > maintained folder.
>> > http://www.1160pm.net/2009/04/23/generation-gap-pattern/
>> >
>> > ~Karsten
>> >
>> > Am 02.07.2009 um 13:04 schrieb Pavel Tavoda:
>> >
>> >> If I should be honest, what annoy me most now on Sculptor is
>> >> regeneration of one-time generated files (main/java main/resources).
>> >> Before I had idea with timestamps but this is difficult to handle. Now
>> >> I found maybe better solution. After generation we should checksum all
>> >> generated files in main/java and main/resources. Than before
>> >> generation we should check all sums and if file have same checksum we
>> >> should remove it (file wasn't change and we should regenerate it).
>> >>
>> >> I don't know enough details about oAW if I can write some hook method
>> >> which can do this for every generated file or I have to write Maven
>> >> plugin, which should be part of clean task. Can somebody give me any
>> >> hint about oAW or Maven where should I look?
>> >>
>> >> I'm willing to write anything what can help to solve this problem.
>> >>
>> >> TNX
>> >>
>> >> Pavel
>> >>
>> >> PS: Patrik, do we have already some incident about this?
>> >>
>> >>
>> >> ------------------------------------------------------------------------------
>> >> _______________________________________________
>> >> Fornax-developer mailing list
>> >> Fornax-developer@...
>> >> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > _______________________________________________
>> > Fornax-developer mailing list
>> > Fornax-developer@...
>> > https://lists.sourceforge.net/lists/listinfo/fornax-developer
>> >
>>
>>
>> ------------------------------------------------------------------------------
>>
>> _______________________________________________
>> Fornax-developer mailing list
>> Fornax-developer@...
>> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>>
>
>
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Fornax-developer mailing list
> Fornax-developer@...
> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>
>


------------------------------------------------------------------------------

_______________________________________________
Fornax-developer mailing list
Fornax-developer@...
https://lists.sourceforge.net/lists/listinfo/fornax-developer

fornax-cartridges-sculptor-maven-plugin-checksum.zip.arch (46K) Download Attachment

Re: Annoying regeneration problem

by Patrik Nordwall :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Now I have read your suggestions and I think it looks good. This is independent of Sculptor, isn't it? It is useful for anyone using fornax-oaw-m2-plugin. Should it be included in that plugin or should it be a separate maven plugin, located in Subversion in same structure as fornax-oaw-m2-plugin?

/Patrik  

Re: Annoying regeneration problem

by PaloT :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hello Patrik,
I don't know how to make run something before 'install' and after
install task in normal lifecycle. Clean wasn't that difficult because
it's running as part of clean but with install it's different story.
Can you help me? Do you know how to declare plugin in POM file that is
started before specific phase?
If we can solve it somehow that it should be totally independent of
fornax-oaw-m2-plugin. If not, we have to include it inside.

TNX

Pavel


On Fri, Jul 17, 2009 at 9:10 PM, Patrik
Nordwall<patrik.nordwall@...> wrote:

>
> Now I have read your suggestions and I think it looks good. This is
> independent of Sculptor, isn't it? It is useful for anyone using
> fornax-oaw-m2-plugin. Should it be included in that plugin or should it be a
> separate maven plugin, located in Subversion in same structure as
> fornax-oaw-m2-plugin?
>
> /Patrik
> --
> View this message in context: http://www.nabble.com/Annoying-regeneration-problem-tp24304851s17564p24540106.html
> Sent from the Fornax-Platform mailing list archive at Nabble.com.
>
>
> ------------------------------------------------------------------------------
> Enter the BlackBerry Developer Challenge
> This is your chance to win up to $100,000 in prizes! For a limited time,
> vendors submitting new applications to BlackBerry App World(TM) will have
> the opportunity to enter the BlackBerry Developer Challenge. See full prize
> details at: http://p.sf.net/sfu/Challenge
> _______________________________________________
> Fornax-developer mailing list
> Fornax-developer@...
> https://lists.sourceforge.net/lists/listinfo/fornax-developer
>

------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
Fornax-developer mailing list
Fornax-developer@...
https://lists.sourceforge.net/lists/listinfo/fornax-developer