[Studio] Using the Maven Release Plugin

[Pierre-Arnaud Marcelot-2]

Hi all,

I want to give you an update on where I am with the use of the Maven Release Plugin in Studio.

I still can't get the release plugin to work correctly, but I have made significant progress...

I'm not working on Apache's SVN repository to do my testing but on my employer's private SVN repository.
So unfortunately, I can't show you the modifications I had to make until now, but here's a quick summup.

First, I had to change our version numbering scheme for SNAPSHOT version. We're currently using ".SNAPSHOT" instead of "-SNAPSHOT" and it was causing issues, so I had to switch back to the original "-SNAPSHOT".
To do so, I had to update the Maven Studio Plugin because "-SNAPSHOT" at the end of the version is not valid for an OSGI bundle.

Then, I was able to provide all the informations the Maven Release Plugin (resolve snapshot dependencies, provide final version and next iteration version, as well as SCM tag name).
The Release Plugin has modified all the poms with the informations I provided and started to run a "build" with the following command: "mvn clean verify".
Everything went fine until the build of the updatesite project which made the build failed as it could not find the artifact org.apache.directory.studio:aciitemeditor:1.3.0.
The problem is that this project was successfully built by the same command a few seconds before...

Do you have any idea on what's going on ?

After the build stopped, I tried to re-build it myself (not using the Release plugin but using "mvn clean install". Everything went fine, the updatesite project included...
So, I have no idea why it's failing when I run it using the Release plugin.

If you have an idea. Feel free to tell me... ;)

Thanks,
Pierre-Arnaud

[Felix Knecht-2]

Pierre-Arnaud Marcelot schrieb:

> Hi all,
>
> I want to give you an update on where I am with the use of the Maven
> Release Plugin in Studio.
>
> I still can't get the release plugin to work correctly, but I have
> made significant progress...
>
> I'm not working on Apache's SVN repository to do my testing but on my
> employer's private SVN repository.
> So unfortunately, I can't show you the modifications I had to make
> until now, but here's a quick summup.
>
> First, I had to change our version numbering scheme for SNAPSHOT
> version. We're currently using ".SNAPSHOT" instead of "-SNAPSHOT" and
> it was causing issues, so I had to switch back to the original
> "-SNAPSHOT".
> To do so, I had to update the Maven Studio Plugin because "-SNAPSHOT"
> at the end of the version is not valid for an OSGI bundle.
>
> Then, I was able to provide all the informations the Maven Release
> Plugin (resolve snapshot dependencies, provide final version and next
> iteration version, as well as SCM tag name).
> The Release Plugin has modified all the poms with the informations I
> provided and started to run a "build" with the following command: "mvn
> clean verify".
> Everything went fine until the build of the updatesite project which
> made the build failed as it could not find the artifact
> org.apache.directory.studio:aciitemeditor:1.3.0. <http://1.3.0.>

Just some ideas
- Did your run using the -Prelease profile?
- Did you verify if at this moment the artifact exists in your local
repository?

> The problem is that this project was successfully built by the same
> command a few seconds before...
>
> Do you have any idea on what's going on ?
>
> After the build stopped, I tried to re-build it myself (not using the
> Release plugin but using "mvn clean install". Everything went fine,
> the updatesite project included...
> So, I have no idea why it's failing when I run it using the Release
> plugin.
>
> If you have an idea. Feel free to tell me... ;)
>
> Thanks,
> Pierre-Arnaud

[Felix Knecht-2]

Felix Knecht schrieb:
>
>
> Just some ideas
> - Did your run using the -Prelease profile?
> - Did you verify if at this moment the artifact exists in your local
> repository?
>
>  
- Change the default preparationGoals to 'clean install verify' (see
http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html)

[Pierre-Arnaud Marcelot-2]

Hi Felix,

On Fri, Sep 19, 2008 at 2:06 PM, Felix Knecht <felixk@...> wrote:

Just some ideas
- Did your run using the -Prelease profile?

Nope.
Is this mandatory ? What does it do ?
 

- Did you verify if at this moment the artifact exists in your local
repository?

I have an artifact for any project that is with version 1.3.0-SNAPSHOT, but absolutely none with version 1.3.0...
Which makes me wondering how Maven has managed to build the others projects (projects that have dependencies on other Studio projects)...

Pierre-Arnaud

[Pierre-Arnaud Marcelot-2]

On Fri, Sep 19, 2008 at 2:11 PM, Felix Knecht <felixk@...> wrote:

- Change the default preparationGoals to 'clean install verify' (see
http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html)

Yeah, I've seen this in the documentation. But I was thinking the maven default value should work out of the box...

I'll give it a try.

Thanks Felix!

Pierre-Arnaud

[Felix Knecht-2]

Pierre-Arnaud Marcelot schrieb:

> Hi Felix,
>
> On Fri, Sep 19, 2008 at 2:06 PM, Felix Knecht <felixk@...
> <mailto:felixk@...>> wrote:
>
>     Just some ideas
>     - Did your run using the -Prelease profile?
>
>
> Nope.
> Is this mandatory ? What does it do ?

More or less mandatory. It's defined in the TLP pom and forces pgp
signing of the artifacts, generate source jars for deployment (I think)
and runs a king of forked deploy goal (which implicitly invokes install
via maven lifecycle). So this could also help :-)

>  
>
>     - Did you verify if at this moment the artifact exists in your local
>     repository?
>
>
> I have an artifact for any project that is with version
> 1.3.0-SNAPSHOT, but absolutely none with version 1.3.0. <http://1.3.0.>..
> Which makes me wondering how Maven has managed to build the others
> projects (projects that have dependencies on other Studio projects)...

I think that release:perform does a checkout from the tagged version
into the target directory and builds it from there. This sould give you
the 1.3.0 versions in your local repository.
>
> Pierre-Arnaud
>

[Felix Knecht-2]

Pierre-Arnaud Marcelot schrieb:
> On Fri, Sep 19, 2008 at 2:11 PM, Felix Knecht <felixk@...
> <mailto:felixk@...>> wrote:
>
>     - Change the default preparationGoals to 'clean install verify' (see
>     http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html)
>
>
> Yeah, I've seen this in the documentation. But I was thinking the
> maven default value should work out of the box...
Maybe we do bypass this using the release profile with it's forked
deploy goal.
>
> I'll give it a try.
>
> Thanks Felix!
>
> Pierre-Arnaud

[Pierre-Arnaud Marcelot-2]

On Fri, Sep 19, 2008 at 2:18 PM, Felix Knecht <felixk@...> wrote:

More or less mandatory. It's defined in the TLP pom and forces pgp
signing of the artifacts, generate source jars for deployment (I think)
and runs a king of forked deploy goal (which implicitly invokes install
via maven lifecycle). So this could also help :-)

I think the deployment is done with the release:perform goal, but I'll use it anyway with release:prepare.
 

I think that release:perform does a checkout from the tagged version
into the target directory and builds it from there. This sould give you
the 1.3.0 versions in your local repository.

Yeah, it will do all that, but I need to have a successful end for the 'mvn release:prepare' command first... ;)

Thanks,
Pierre-Arnaud 

[Pierre-Arnaud Marcelot-2]

On Fri, Sep 19, 2008 at 2:11 PM, Felix Knecht <felixk@...> wrote:

- Change the default preparationGoals to 'clean install verify' (see
http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html)

It worked. The build is now successful.

However, I still got an error when doing the tag...
=====================================================
[INFO] Executing: svn --non-interactive commit --file /tmp/maven-scm-1581641791.commit --targets /tmp/maven-scm-61222-targets
[INFO] Working directory: /Users/pajbam/Development/Apache/studio-release-testing/trunk
[INFO] Tagging release with the label parent-1.3.0...
[INFO] Executing: svn --non-interactive copy --file /tmp/maven-scm-760076483.commit . https://svn.iktek.com/repos/iktek_corp/studio-release-testing/tags/parent-1.3.0
[INFO] Working directory: /Users/pajbam/Development/Apache/studio-release-testing/trunk
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Unable to tag SCM
Provider message:
The svn tag command failed.
Command output:
svn: Commit failed (details follow):
svn: File '/repos/iktek_corp/studio-release-testing/tags/parent-1.3.0/aciitemeditor/pom.xml' already exists

[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7 minutes 35 seconds
[INFO] Finished at: Fri Sep 19 15:20:16 CEST 2008
[INFO] Final Memory: 26M/46M
[INFO] ------------------------------------------------------------------------
=====================================================

But I've found that this error is related to SVN 1.5.1 on Mac OS X.
I'll update to 1.5.2 and keep you posted.

I think it's in a good way... ;)

Regards,
Pierre-Arnaud

[Felix Knecht-2]

Pierre-Arnaud Marcelot schrieb:
> On Fri, Sep 19, 2008 at 2:11 PM, Felix Knecht <felixk@...
> <mailto:felixk@...>> wrote:
>
>     - Change the default preparationGoals to 'clean install verify' (see
>     http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html)
>
>
> It worked. The build is now successful.
>
Big thanks :-)

> However, I still got an error when doing the tag...
> =====================================================
> [INFO] Executing: svn --non-interactive commit --file
> /tmp/maven-scm-1581641791.commit --targets /tmp/maven-scm-61222-targets
> [INFO] Working directory:
> /Users/pajbam/Development/Apache/studio-release-testing/trunk
> [INFO] Tagging release with the label parent-1.3.0...
> [INFO] Executing: svn --non-interactive copy --file
> /tmp/maven-scm-760076483.commit .
> https://svn.iktek.com/repos/iktek_corp/studio-release-testing/tags/parent-1.3.0
> [INFO] Working directory:
> /Users/pajbam/Development/Apache/studio-release-testing/trunk
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Unable to tag SCM
> Provider message:
> The svn tag command failed.
> Command output:
> svn: Commit failed (details follow):
> svn: File
> '/repos/iktek_corp/studio-release-testing/tags/parent-1.3.0/aciitemeditor/pom.xml'
> already exists
>
> [INFO]
> ------------------------------------------------------------------------
> [INFO] For more information, run Maven with the -e switch
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 7 minutes 35 seconds
> [INFO] Finished at: Fri Sep 19 15:20:16 CEST 2008
> [INFO] Final Memory: 26M/46M
> [INFO]
> ------------------------------------------------------------------------
> =====================================================
>
> But I've found that this error is related to SVN 1.5.1 on Mac OS X.
> I'll update to 1.5.2 and keep you posted.

Not sure if it's a Mac OS X problem - I had it also on linux. mvn
release:rollback will rollback the versioning in your poms but won't
delete the svn tag it has created in the repository. I had to delete it
manually before running mvn release... again. It's something like svn
del https://boo/bar/yourproject/tags/{tag} when using via https.
>
> I think it's in a good way... ;)
>
> Regards,
> Pierre-Arnaud
>

[Pierre-Arnaud Marcelot-2]

Unfortunaltely, version 1.5.2 does not solves the problem.

There's a Jira opened for this issue[1].

I think that's what happened when Alex tried to release Apache DS 1.5.4 the past week.

Regards,
Pierre-Arnaud

[1] - http://jira.codehaus.org/browse/SCM-406

On Fri, Sep 19, 2008 at 3:25 PM, Pierre-Arnaud Marcelot <pa@...> wrote:

On Fri, Sep 19, 2008 at 2:11 PM, Felix Knecht <felixk@...> wrote:

- Change the default preparationGoals to 'clean install verify' (see
http://maven.apache.org/plugins/maven-release-plugin/prepare-mojo.html)

It worked. The build is now successful.

However, I still got an error when doing the tag...
=====================================================
[INFO] Executing: svn --non-interactive commit --file /tmp/maven-scm-1581641791.commit --targets /tmp/maven-scm-61222-targets
[INFO] Working directory: /Users/pajbam/Development/Apache/studio-release-testing/trunk
[INFO] Tagging release with the label parent-1.3.0...
[INFO] Executing: svn --non-interactive copy --file /tmp/maven-scm-760076483.commit . https://svn.iktek.com/repos/iktek_corp/studio-release-testing/tags/parent-1.3.0
[INFO] Working directory: /Users/pajbam/Development/Apache/studio-release-testing/trunk
[INFO] ------------------------------------------------------------------------
[ERROR] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Unable to tag SCM
Provider message:
The svn tag command failed.
Command output:
svn: Commit failed (details follow):
svn: File '/repos/iktek_corp/studio-release-testing/tags/parent-1.3.0/aciitemeditor/pom.xml' already exists

[INFO] ------------------------------------------------------------------------
[INFO] For more information, run Maven with the -e switch
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7 minutes 35 seconds
[INFO] Finished at: Fri Sep 19 15:20:16 CEST 2008
[INFO] Final Memory: 26M/46M
[INFO] ------------------------------------------------------------------------
=====================================================

But I've found that this error is related to SVN 1.5.1 on Mac OS X.
I'll update to 1.5.2 and keep you posted.

I think it's in a good way... ;)

Regards,
Pierre-Arnaud

[Pierre-Arnaud Marcelot-2]

On Fri, Sep 19, 2008 at 3:37 PM, Felix Knecht <felixk@...> wrote:

Not sure if it's a Mac OS X problem - I had it also on linux. mvn

release:rollback will rollback the versioning in your poms but won't
delete the svn tag it has created in the repository. I had to delete it
manually before running mvn release... again. It's something like svn
del https://boo/bar/yourproject/tags/{tag} when using via https.

Yeah, it seems this problem is also running on Ubuntu.
The first blog post I read was saying the problem was only occuring on Mac OS X but I believe the problem is the same for any OS using version 1.5.1 or 1.5.2.

Hopefully there's a workaround:

# mvn release:prepare
=> fails
# svn up -r head 
# mvn release:prepare -Dresume


Actually, the only thing that has been committed is the modification of the poms. I never succeeded in having the release tagged...


Pierre-Arnaud

[Pierre-Arnaud Marcelot-2]

On Fri, Sep 19, 2008 at 3:44 PM, Pierre-Arnaud Marcelot <pa@...> wrote:

Hopefully there's a workaround:

# mvn release:prepare
=> fails
# svn up -r head 
# mvn release:prepare -Dresume

I confirm that the workaround is working.

I finally get the release:prepare goal to finish successfully.

Now let's move to another problem, we still need to resolve.
When modifying the pom.xml (removing the -SNAPSHOT from the version), the MANIFEST.MF file don't get updated.

I think we could use this command 'clean studio:eclipse install verify' in the Release plugin configuration to fix this.

Pierre-Arnaud

[Felix Knecht-2]

Pierre-Arnaud Marcelot schrieb:

> On Fri, Sep 19, 2008 at 3:44 PM, Pierre-Arnaud Marcelot
> <pa@... <mailto:pa@...>> wrote:
>
>     Hopefully there's a workaround:
>
>     # mvn release:prepare
>     => fails
>     # svn up -r head
>     # mvn release:prepare -Dresume
>        
>
> I confirm that the workaround is working.
>
> I finally get the release:prepare goal to finish successfully.
>
> Now let's move to another problem, we still need to resolve.
> When modifying the pom.xml (removing the -SNAPSHOT from the version),
> the MANIFEST.MF file don't get updated.
>
> I think we could use this command 'clean studio:eclipse install
> verify' in the Release plugin configuration to fix this.

Yep. Otherwise we make it working or add another goal especially for this.
>
> Pierre-Arnaud
>

[Pierre-Arnaud Marcelot-2]

On Fri, Sep 19, 2008 at 4:01 PM, Felix Knecht <felixk@...> wrote:

Yep. Otherwise we make it working or add another goal especially for this.

I was too optimistic... It's not working very good.

There are two problems:

• only a few projects get updated
• the changes to the manifest files are not committed by the Release plugin

I'm maybe thinking about getting the MANIFEST.MF file generated by maven via the Maven Bundle Plugin [1], instead of having it in SVN.
We'd have two files to generate:

• one in /META-INF/MANIFEST.MF, to be able to run the plugin from Eclipse (with the META-INF folder added to svn:ignore)
• one in /target/META-INF/MANIFEST.MF to be used by the Maven Jar plugin.

I think it's the best solution...

Regards,
Pierre-Arnaud

[1] - http://felix.apache.org/site/maven-bundle-plugin-bnd.html

[Felix Knecht-2]

Pierre-Arnaud Marcelot schrieb:

> On Fri, Sep 19, 2008 at 4:01 PM, Felix Knecht <felixk@...
> <mailto:felixk@...>> wrote:
>
>     Yep. Otherwise we make it working or add another goal especially for
>     this.
>
>
> I was too optimistic... It's not working very good.
>
> There are two problems:
>
>     * only a few projects get updated
>     * the changes to the manifest files are not committed by the Release
>       plugin
>
> I'm maybe thinking about getting the MANIFEST.MF file generated by maven
> via the Maven Bundle Plugin [1], instead of having it in SVN.
> We'd have two files to generate:
>
>     * one in /META-INF/MANIFEST.MF, to be able to run the plugin from
>       Eclipse (with the META-INF folder added to svn:ignore)
>     * one in /target/META-INF/MANIFEST.MF to be used by the Maven Jar
>       plugin.
>
> I think it's the best solution...

If the 2 MF files are identical the one from META-INF can be copied to the target using the <resources /> section in the
pom.

Maybe we can also skip the studio plugin if the features of the bundle plugin fits our needs.

>
> Regards,
> Pierre-Arnaud
>
> [1] - http://felix.apache.org/site/maven-bundle-plugin-bnd.html

[Pierre-Arnaud Marcelot-2]

On Fri, Sep 19, 2008 at 5:49 PM, Felix Knecht <felixk@...> wrote:

If the 2 MF files are identical the one from META-INF can be copied to the target using the <resources /> section in the
pom.

The idea is to get rid of the current META-INF folder. This will not be in SVN anymore.

The first MF file is to be created when the eclipse:eclipse goal is run (I think this is possible, right?), and to be used by eclipse when launching the plugin to test/debug it.
The second MF file is to be used by the Maven Jar plugin when creating the final jar.
 

Maybe we can also skip the studio plugin if the features of the bundle plugin fits our needs.

Yeah, I think the UpdateManifest Mojo won't be used anymore. But, we'll still use the others Mojos defined in the Studio plugin.

FYI, I successfully managed to generate a perfectly working Manifest file for the Jars plugin.
I'll try modify all the projects on Monday with this new functionnality.

With this update, I think we'll be able to safely use the Maven Release plugin. Which will help a lot in terms of release time and release reliability.
Less hand edits and more automated processes, means a stronger build. :D

Regards,
Pierre-Arnaud

[Pierre-Arnaud Marcelot-2]

On Fri, Sep 19, 2008 at 6:58 PM, Pierre-Arnaud Marcelot <pa@...> wrote:

On Fri, Sep 19, 2008 at 5:49 PM, Felix Knecht <felixk@...> wrote:

If the 2 MF files are identical the one from META-INF can be copied to the target using the <resources /> section in the
pom.

The idea is to get rid of the current META-INF folder. This will not be in SVN anymore.

The first MF file is to be created when the eclipse:eclipse goal is run (I think this is possible, right?), and to be used by eclipse when launching the plugin to test/debug it.
The second MF file is to be used by the Maven Jar plugin when creating the final jar.

Actually, there's no need to generate 2 Manifest files. One at ${project}/META-INF/MANIFEST.MF is enough.

Before, I do any modification, I want to make sure, everyone is ok with that.

So, the idea is to:

• have the META-INF/MANIFEST.MF file removed from SVN and added to svn:ignore
• have the META-INF/MANIFEST.MF file generated by the Felix Bundle plugin (the configuration of the Manifest is in the pom.xml)

Is this ok with everyone (+1/+-0/-1) ?

Thanks,
Pierre-Arnaud

[Felix Knecht-2]

Pierre-Arnaud Marcelot schrieb:

> On Fri, Sep 19, 2008 at 6:58 PM, Pierre-Arnaud Marcelot
> <pa@... <mailto:pa@...>> wrote:
>
>     On Fri, Sep 19, 2008 at 5:49 PM, Felix Knecht <felixk@...
>     <mailto:felixk@...>> wrote:
>
>         If the 2 MF files are identical the one from META-INF can be
>         copied to the target using the <resources /> section in the
>         pom.
>
>
>     The idea is to get rid of the current META-INF folder. This will
>     not be in SVN anymore.
>
>     The first MF file is to be created when the eclipse:eclipse goal
>     is run (I think this is possible, right?), and to be used by
>     eclipse when launching the plugin to test/debug it.
>     The second MF file is to be used by the Maven Jar plugin when
>     creating the final jar.
>
>
> Actually, there's no need to generate 2 Manifest files. One at
> ${project}/META-INF/MANIFEST.MF is enough.
>
> Before, I do any modification, I want to make sure, everyone is ok
> with that.
>
> So, the idea is to:
>
>     * have the META-INF/MANIFEST.MF file removed from SVN and added to
>       svn:ignore
>     * have the META-INF/MANIFEST.MF file generated by the Felix Bundle
>       plugin (the configuration of the Manifest is in the pom.xml)
>
>
> Is this ok with everyone (+1/+-0/-1) ?

+1
IIRC the MANIFEST.MF is also used by Eclipse (but I'm not sure). If so
we need to adapt the documentation about the intial mvn eclipse:eclipse
studio:eclipse stuff.
Note also that studio:eclipse does some modifications in the
MANIFEST.MF. This need to be verified if it still works or isn't needed
anymore when switching to felix bundle plugin.

Regards
Felix

[Pierre-Arnaud Marcelot-2]

On Tue, Sep 23, 2008 at 11:48 AM, Felix Knecht <felixk@...> wrote:

+1
IIRC the MANIFEST.MF is also used by Eclipse (but I'm not sure).

Yep, it is used to launch the plugin from within Eclipse to debug the plugin and it's also used as Manifest of the final jar.
 

If so we need to adapt the documentation about the intial mvn eclipse:eclipse
studio:eclipse stuff.

Yeah, and no... We will still need the use of the studio:eclipse plugin I think (at least for copying the dependencies into the 'lib' folder).
BTW, I was wondering if it's possible in Maven to attach the launch of a goal of a plugin to another goal (of another plugin), or if it's only possible to attach it to a phase of the build.
Do you have an idea ?
 

Note also that studio:eclipse does some modifications in the
MANIFEST.MF. This need to be verified if it still works or isn't needed
anymore when switching to felix bundle plugin.

This won't be need anymore. The generated Manifest will work out of the box.

Regards,
Pierre-Arnaud