unix-maven-plugin philosophy

[Jörg Hohwiller]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Trygve, hi everybody,

I started to get into unix-maven-plugin.
So far I like the general configuration of
common package meta-data and your excellent
documentation how this is mapped down according
to the actual format.

However currently the site has various dead links:
http://mojo.codehaus.org/unix/unix-maven-plugin/usage.html
http://mojo.codehaus.org/unix/unix-handbook/handbook
http://mojo.codehaus.org/unix/unix-maven-plugin/unix-maven-plugin/plugin-info.html
http://mojo.codehaus.org/unix/unix-maven-plugin/developers/

For the meta-data I would like to work on
adding dependencies. Here I am NOT talking
about maven deps but about dependencies of
the generated package.

Further what I want to understand is, why you added
all this <copyArtifact> and <extractArtifact>
in <assembly>?
We already had the "re-invent-the-wheel" discussion
about things like this when talking about the
philosophy of solaris-maven-plugin.
I mean there is maven-dependency-plugin,
maven-resource-plugin and filtering
and all what maven already offers.
Why not just let the user build the package
content with all features of maven
to build.outputDirectory and then build
a package from that. Sure you will still
need support for symlinks and permissions.
What was your motivation to go a different way?
Or am I just missing something?

Did you have a look at pkg-maven-plugin.
It is just raw minimum but it should just
show how simple you can make maven to
build packages with all your custom needs.
Should I also add an example project
or is it self-explanatory.

Regards
  Jörg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpBPKwACgkQmPuec2Dcv/9j1QCeOr6Babvmg5DyuKogf+hdQ3oK
CsMAn2OFdvz+WI9n8fRJ7C5fSaadZEMY
=G524
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Trygve Laugstøl]

Joerg Hohwiller wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi Trygve, hi everybody,
>
> I started to get into unix-maven-plugin.
> So far I like the general configuration of
> common package meta-data and your excellent
> documentation how this is mapped down according
> to the actual format.
>
> However currently the site has various dead links:
> http://mojo.codehaus.org/unix/unix-maven-plugin/usage.html
> http://mojo.codehaus.org/unix/unix-handbook/handbook
> http://mojo.codehaus.org/unix/unix-maven-plugin/unix-maven-plugin/plugin-info.html
> http://mojo.codehaus.org/unix/unix-maven-plugin/developers/

Those are broken if you go through other modules that the front page.
I'm not sure how to fix that, any help/hints from a Maven Site guru
would be very much appreciated.

> For the meta-data I would like to work on
> adding dependencies. Here I am NOT talking
> about maven deps but about dependencies of
> the generated package.

That would be great. I would be nice if you could create a note on what
kind of data you would like and how it would map in the different formats.

I would also like to add the ability to state extra dependencies (IOW,
manual dependencies) in each of the formats in their format extension
parts (<rpm>, <dpkg> etc).

> Further what I want to understand is, why you added
> all this <copyArtifact> and <extractArtifact>
> in <assembly>?
> We already had the "re-invent-the-wheel" discussion
> about things like this when talking about the
> philosophy of solaris-maven-plugin.
> I mean there is maven-dependency-plugin,
> maven-resource-plugin and filtering
> and all what maven already offers.
> Why not just let the user build the package
> content with all features of maven
> to build.outputDirectory and then build
> a package from that. Sure you will still
> need support for symlinks and permissions.
> What was your motivation to go a different way?
> Or am I just missing something?

Mainly because those utils one or more of 1) buggy, 2) slow, 3)
incomplete and 4) not sufficient.

You didn't mention the assembly plugin but that also falls into that list.

I know we had the discussion about not reinventing the wheel, which I
still stand by but I've come to the conclusion that this is not
reinventing the wheel and that it was required.

Features that I can support that the other plugins can't support:

o Streaming of data directly from inside a file in the repository.

  I can pick out a file from inside another JAR directly from the
artifact. This saves a copy to the local directory, another full copy of
the extracted artifact.

o Uniform renaming of files

o File attributes like owner, group owner and permissions. In addition
there are the package format specific attributes like %doc and %config
for RPM and "request", "space" for pkg.

It would be possible to use the other plugins that you mention to do one
half of the work, but then the plugin would have to do the other half of
work through a similar mechanism to what I do today.

That is where I started and was what I implemented. Once I implemented a
"set attributes"-like thing and created a good model of a file system,
adding the rest of the features was easy.

It has even spun of a whole new set of operations that I want to
implement, in addition to document how to create your own operations.
For example:

o <touch> for creating empty files.

o <echo> for creating files with hardcoded content from the pom.xml.
Hard coded means from when the plugin run so you can use expressions
there. Think "${version}".

There are more example, but I don't have my notebook here.

> Did you have a look at pkg-maven-plugin.
> It is just raw minimum but it should just
> show how simple you can make maven to
> build packages with all your custom needs.
> Should I also add an example project
> or is it self-explanatory.

I haven't done an in-depth look yet, but from the browse I've given it
from subversion it seems do to about the same things that I do. I think
an example pom would be useful to see how it is actually used.

I see that there isn't much code there which definitely is a positive
attribute. I realize that my plugin is quite huge in number of lines
(and number of modules) but I think the relevant parts of the plugin is
quite small.

I'm looking forward to the example pom, it seems that you have some
smart ideas on how to set/change the file attributes there.

--
Trygve

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Lee Thompson]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Joerg Hohwiller <joerg@...> wrote:
> I mean there is maven-dependency-plugin,
> maven-resource-plugin and filtering
> and all what maven already offers.
> Why not just let the user build the package
> content with all features of maven
> to build.outputDirectory and then build
> a package from that.

rpm-cbuilds-maven-plugins works that way.  It lets other plugins prepare the files for insertion into the RPM in ${project.build.directory}/rpm-basedir and does all the voodo to create the RPM.  Of course it only does RPMs, not Deb, and Solaris pkgs.

http://mojo.codehaus.org/cbuild-parent/cbuild-plugin-parent/rpm-cbuild-maven-plugin/build-mojo.html

Lee

[Trygve Laugstøl]

Lee Thompson wrote:

>
> **Joerg Hohwiller <joerg@...> wrote:
> **> I mean there is maven-dependency-plugin,
>  > maven-resource-plugin and filtering
>  > and all what maven already offers.
>  > Why not just let the user build the package
>  > content with all features of maven
>  > to build.outputDirectory and then build
>  > a package from that.
>
> rpm-cbuilds-maven-plugins works that way.  It lets other plugins prepare
> the files for insertion into the RPM in
> |${project.build.directory}/rpm-basedir| and does all the voodo to
> create the RPM.  Of course it only does RPMs, not Deb, and Solaris pkgs.
>
> http://mojo.codehaus.org/cbuild-parent/cbuild-plugin-parent/rpm-cbuild-maven-plugin/build-mojo.html

Have you looked at the other RPM plugin? It seems that Brett Okken is
putting a lot of effort into it.

--
Trygve

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jörg Hohwiller]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Lee,

>
> **Joerg Hohwiller <joerg@...> wrote:
> **> I mean there is maven-dependency-plugin,
>> maven-resource-plugin and filtering
>> and all what maven already offers.
>> Why not just let the user build the package
>> content with all features of maven
>> to build.outputDirectory and then build
>> a package from that.
>
> rpm-cbuilds-maven-plugins works that way.  It lets other plugins prepare
> the files for insertion into the RPM in
> |${project.build.directory}/rpm-basedir| and does all the voodo to
> create the RPM.  Of course it only does RPMs, not Deb, and Solaris pkgs.

Thanks for the hint. I like this idea and also did the same
when I created solaris-maven-plugin.

But I also love the idea of having one plugin for different binary formats.
Then you can build deb,rpm,... from the same config and an added feature
may help for all formats.
In pkg-maven-plugin I also went the "let existing plugins do the major work
and just focus on packaging" way but added a plugin concept into the
maven-plugin so formats can be easily added even by thirdparty coders without
touching the plugin. That was my vision and I had just started a minimum codebase.
Now I found unix-maven-plugin and I definitely want to give it a try
in getting together with Trygve so we put our ideas and energy together.

I think cbuilds also has a different scope and philosophy.
My impression is that it could be really useful if we bring
our ideas together rather than building all our own nice plugins.
We will see what happens - it might work out or not.
So far I am happy to chat with all of you and willing to participate
on unix-maven-plugin. Trygve has already solved a lot of problems
that I do NOT want to solve again.
As its version-number does not sound very mature
there might be chances to change some things or to come to the point
that it is just good the way it is.

>
> http://mojo.codehaus.org/cbuild-parent/cbuild-plugin-parent/rpm-cbuild-maven-plugin/build-mojo.html
>
> Lee
>

regards
  Jörg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpD8tMACgkQmPuec2Dcv/918ACcCBfM28F+PrNoIQKEXD7xSjHy
6fMAoIP6hP14VwjmYD3VDUXGIwfOYEQu
=3fXq
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jörg Hohwiller]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

> Mainly because those utils one or more of 1) buggy, 2) slow, 3)
> incomplete and 4) not sufficient.

I see that problem but do NOT agree with the general argument.

1) we are a community and progress is made
2) not very important to me
3) see 1)
4) important topic, see below

>
> You didn't mention the assembly plugin but that also falls into that list.

jup. But it also does some reinvent the wheel already.

>
> I know we had the discussion about not reinventing the wheel, which I
> still stand by but I've come to the conclusion that this is not
> reinventing the wheel and that it was required.

I see, but I am not convinced.

>
> Features that I can support that the other plugins can't support:
>
> o Streaming of data directly from inside a file in the repository.

OK. understood.
Definite point for 2) (performance)

>
>  I can pick out a file from inside another JAR directly from the
> artifact. This saves a copy to the local directory, another full copy of
> the extracted artifact.

Same as above.

>
> o Uniform renaming of files

Are you talking about those Windows limitations?

>
> o File attributes like owner, group owner and permissions. In addition
> there are the package format specific attributes like %doc and %config
> for RPM and "request", "space" for pkg.

Yep. This can not be done by other plugins and is something to be
solved by unix-maven-plugin.

>
> It would be possible to use the other plugins that you mention to do one
> half of the work, but then the plugin would have to do the other half of
> work through a similar mechanism to what I do today.

exactly.

>
> That is where I started and was what I implemented. Once I implemented a
> "set attributes"-like thing and created a good model of a file system,
> adding the rest of the features was easy.

OK.

>
> It has even spun of a whole new set of operations that I want to
> implement, in addition to document how to create your own operations.
> For example:
>
> o <touch> for creating empty files.

Just add an empty file to src/main/resources.

>
> o <echo> for creating files with hardcoded content from the pom.xml.
> Hard coded means from when the plugin run so you can use expressions
> there. Think "${version}".

Just add that as file to src/main/resources and use filtering.
You would already have this feature if you would not
have reinvented the wheel ;)

>
> There are more example, but I don't have my notebook here.

Give me all arguments...

>
>> Did you have a look at pkg-maven-plugin.
>> It is just raw minimum but it should just
>> show how simple you can make maven to
>> build packages with all your custom needs.
>> Should I also add an example project
>> or is it self-explanatory.
>
> I haven't done an in-depth look yet, but from the browse I've given it
> from subversion it seems do to about the same things that I do. I think
> an example pom would be useful to see how it is actually used.

I will add some example project to pkg-maven-plugin/src/it/example1
at the beginning of next week. This week I am quite busy (filing
all those Galileo bugs, etc.)

>
> I see that there isn't much code there which definitely is a positive
> attribute. I realize that my plugin is quite huge in number of lines
> (and number of modules) but I think the relevant parts of the plugin is
> quite small.

What I checked in can only create zip and tar so far which is kind
of pointless. Anyhow tar already allows testing permissions, symlinks, etc.

>
> I'm looking forward to the example pom, it seems that you have some
> smart ideas on how to set/change the file attributes there.

I simply re-implemented (the wheel) what unix/posix defines with
umaks, chmod, chown. In that point our solution fits because
here I also have a virtual FS concept but so simple that I did not mention before.

>
> --
> Trygve

Best regards
  Jörg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpD9d8ACgkQmPuec2Dcv/+u6ACeMW7vtPtkwHgzC79+meMPfYYa
G/IAn3wpHPyj3wVjf99CwVZpo7GY/1fX
=cxNn
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Lee Thompson]

Some parts of this message have been removed. Learn more about Nabble's security policy.

Trygve Laugstøl <trygvis@...> wrote

> Have you looked at the other RPM plugin? It seems that Brett Okken is putting a lot of effort into it.

Not yet.  Hopefully some weekend down the road. 

[Trygve Laugstøl]

Joerg Hohwiller wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>> Mainly because those utils one or more of 1) buggy, 2) slow, 3)
>> incomplete and 4) not sufficient.
>
> I see that problem but do NOT agree with the general argument.
>
> 1) we are a community and progress is made
> 2) not very important to me
> 3) see 1)

1) and 3) is an issue because 1) it will take me *way* longer time to
implement the features that I want and 2) I'm not even sure that their
goals match what I want to do so it is even hard to "fix" them.

Speed is an important aspect when you're trying to assemble enough of
these packages.

> 4) important topic, see below
>
>> You didn't mention the assembly plugin but that also falls into that list.
>
> jup. But it also does some reinvent the wheel already.
>
>> I know we had the discussion about not reinventing the wheel, which I
>> still stand by but I've come to the conclusion that this is not
>> reinventing the wheel and that it was required.
>
> I see, but I am not convinced.
>
>> Features that I can support that the other plugins can't support:
>>
>> o Streaming of data directly from inside a file in the repository.
>
> OK. understood.
> Definite point for 2) (performance)
>
>>  I can pick out a file from inside another JAR directly from the
>> artifact. This saves a copy to the local directory, another full copy of
>> the extracted artifact.
>
> Same as above.

Also, no other plugin support this.

>> o Uniform renaming of files
>
> Are you talking about those Windows limitations?

No, I can use regular expressions to rename files, for example to remove
version strings on a file (foo-1.0.jar => foo.jar)

>> o File attributes like owner, group owner and permissions. In addition
>> there are the package format specific attributes like %doc and %config
>> for RPM and "request", "space" for pkg.
>
> Yep. This can not be done by other plugins and is something to be
> solved by unix-maven-plugin.
>
>> It would be possible to use the other plugins that you mention to do one
>> half of the work, but then the plugin would have to do the other half of
>> work through a similar mechanism to what I do today.
>
> exactly.
>
>> That is where I started and was what I implemented. Once I implemented a
>> "set attributes"-like thing and created a good model of a file system,
>> adding the rest of the features was easy.
>
> OK.
>
>> It has even spun of a whole new set of operations that I want to
>> implement, in addition to document how to create your own operations.
>> For example:
>>
>> o <touch> for creating empty files.
>
> Just add an empty file to src/main/resources.

I would still have to add xml in my pom to control the other file
attributes, and creating support for <touch> is a no-brainer, so why not
add the convenience?

>> o <echo> for creating files with hardcoded content from the pom.xml.
>> Hard coded means from when the plugin run so you can use expressions
>> there. Think "${version}".
>
> Just add that as file to src/main/resources and use filtering.
> You would already have this feature if you would not
> have reinvented the wheel ;)

I'm not going to require users to learn another plugin and at *lots* of
xml when it can be done so much easier.

>> There are more example, but I don't have my notebook here.
>
> Give me all arguments...

I've given you all my arguments, but I can summarize:

1) Existing plugins are missing required features

2) Ease of use for users

>>> Did you have a look at pkg-maven-plugin.
>>> It is just raw minimum but it should just
>>> show how simple you can make maven to
>>> build packages with all your custom needs.
>>> Should I also add an example project
>>> or is it self-explanatory.
>> I haven't done an in-depth look yet, but from the browse I've given it
>> from subversion it seems do to about the same things that I do. I think
>> an example pom would be useful to see how it is actually used.
>
> I will add some example project to pkg-maven-plugin/src/it/example1
> at the beginning of next week. This week I am quite busy (filing
> all those Galileo bugs, etc.)
>
>> I see that there isn't much code there which definitely is a positive
>> attribute. I realize that my plugin is quite huge in number of lines
>> (and number of modules) but I think the relevant parts of the plugin is
>> quite small.
>
> What I checked in can only create zip and tar so far which is kind
> of pointless. Anyhow tar already allows testing permissions, symlinks, etc.
>
>> I'm looking forward to the example pom, it seems that you have some
>> smart ideas on how to set/change the file attributes there.
>
> I simply re-implemented (the wheel) what unix/posix defines with
> umaks, chmod, chown. In that point our solution fits because
> here I also have a virtual FS concept but so simple that I did not mention before.

This is in my opinion not re-invention, it is new software/features
required to implement the plugins that we work on. I have the same
concept which is very central in the assembly part and metadata file
generation (.spec, debian/control, pkginfo, prototype) of the plugin.

I argue that I'm not re-inventing the wheel, but using existing code,
just like the other plugins are using existing code. To me, reuse is as
much about reusing code as reusing the actual plugins.

--
Trygve

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jörg Hohwiller]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Trygve,

thanks for your reply. Now worries for the delay.
Usually I am the one who gets lost for a long time...

>>> Mainly because those utils one or more of 1) buggy, 2) slow, 3)
>>> incomplete and 4) not sufficient.
>>
>> I see that problem but do NOT agree with the general argument.
>>
>> 1) we are a community and progress is made
>> 2) not very important to me
>> 3) see 1)
>
> 1) and 3) is an issue because 1) it will take me *way* longer time to
> implement the features that I want and 2) I'm not even sure that their
> goals match what I want to do so it is even hard to "fix" them.

I still do not see the real bugs that are left in
resource-plugin and dependency-plugin.
Both have made great progress and I am really happy with them...
But yes, some of the fixes took ages in the last 2 years,
especially for resource-plugin.

>
> Speed is an important aspect when you're trying to assemble enough of
> these packages.

OK. Point for you.

>
>> 4) important topic, see below
>>
>>> You didn't mention the assembly plugin but that also falls into that
>>> list.
>>
>> jup. But it also does some reinvent the wheel already.
>>
>>> I know we had the discussion about not reinventing the wheel, which I
>>> still stand by but I've come to the conclusion that this is not
>>> reinventing the wheel and that it was required.
>>
>> I see, but I am not convinced.
>>
>>> Features that I can support that the other plugins can't support:
>>>
>>> o Streaming of data directly from inside a file in the repository.
>>
>> OK. understood.
>> Definite point for 2) (performance)
>>
>>>  I can pick out a file from inside another JAR directly from the
>>> artifact. This saves a copy to the local directory, another full copy of
>>> the extracted artifact.
>>
>> Same as above.
>
> Also, no other plugin support this.

OK, but "just" a performance issue.

>
>>> o Uniform renaming of files
>>
>> Are you talking about those Windows limitations?
>
> No, I can use regular expressions to rename files, for example to remove
> version strings on a file (foo-1.0.jar => foo.jar)

dependency-plugin can handle this as well.
For resources there is typically no such need,
because you have them under your control.

>
>>> o File attributes like owner, group owner and permissions. In addition
>>> there are the package format specific attributes like %doc and %config
>>> for RPM and "request", "space" for pkg.
>>
>> Yep. This can not be done by other plugins and is something to be
>> solved by unix-maven-plugin.
>>
>>> It would be possible to use the other plugins that you mention to do one
>>> half of the work, but then the plugin would have to do the other half of
>>> work through a similar mechanism to what I do today.
>>
>> exactly.
>>
>>> That is where I started and was what I implemented. Once I implemented a
>>> "set attributes"-like thing and created a good model of a file system,
>>> adding the rest of the features was easy.
>>
>> OK.
>>
>>> It has even spun of a whole new set of operations that I want to
>>> implement, in addition to document how to create your own operations.
>>> For example:
>>>
>>> o <touch> for creating empty files.
>>
>> Just add an empty file to src/main/resources.
>
> I would still have to add xml in my pom to control the other file
> attributes, and creating support for <touch> is a no-brainer, so why not
> add the convenience?

Typically you will have defaults for file permission and ownership.
I also added support to have rules via patterns (glob or regexp).
So I can do chmod a+x on **/*.sh, etc.
In such case I will typically have no additional config.

>
>>> o <echo> for creating files with hardcoded content from the pom.xml.
>>> Hard coded means from when the plugin run so you can use expressions
>>> there. Think "${version}".
>>
>> Just add that as file to src/main/resources and use filtering.
>> You would already have this feature if you would not
>> have reinvented the wheel ;)
>
> I'm not going to require users to learn another plugin and at *lots* of
> xml when it can be done so much easier.

What lots of xml? Did you look at my example. I guess you will
need more XML to do the same thing with you plugin.

>
>>> There are more example, but I don't have my notebook here.
>>
>> Give me all arguments...
>
> I've given you all my arguments, but I can summarize:
>
> 1) Existing plugins are missing required features
>
> 2) Ease of use for users
>
>>>> Did you have a look at pkg-maven-plugin.
>>>> It is just raw minimum but it should just
>>>> show how simple you can make maven to
>>>> build packages with all your custom needs.
>>>> Should I also add an example project
>>>> or is it self-explanatory.
>>> I haven't done an in-depth look yet, but from the browse I've given it
>>> from subversion it seems do to about the same things that I do. I think
>>> an example pom would be useful to see how it is actually used.
>>
>> I will add some example project to pkg-maven-plugin/src/it/example1
>> at the beginning of next week. This week I am quite busy (filing
>> all those Galileo bugs, etc.)
>>
>>> I see that there isn't much code there which definitely is a positive
>>> attribute. I realize that my plugin is quite huge in number of lines
>>> (and number of modules) but I think the relevant parts of the plugin is
>>> quite small.
>>
>> What I checked in can only create zip and tar so far which is kind
>> of pointless. Anyhow tar already allows testing permissions, symlinks,
>> etc.
>>
>>> I'm looking forward to the example pom, it seems that you have some
>>> smart ideas on how to set/change the file attributes there.
>>
>> I simply re-implemented (the wheel) what unix/posix defines with
>> umaks, chmod, chown. In that point our solution fits because
>> here I also have a virtual FS concept but so simple that I did not
>> mention before.
>
> This is in my opinion not re-invention, it is new software/features
> required to implement the plugins that we work on. I have the same
> concept which is very central in the assembly part and metadata file
> generation (.spec, debian/control, pkginfo, prototype) of the plugin.
>
> I argue that I'm not re-inventing the wheel, but using existing code,
> just like the other plugins are using existing code. To me, reuse is as
> much about reusing code as reusing the actual plugins.

I want to draw a conclusion:
You have already implemented cool features and those have
better performance that I belief without measuring.
My goal is NOT to convince you to give this up and kick it out.
What do you think of bringing both approaches together.
You will simply add everything in ${project.outputDirectory)
to the package as well. In you current use-case that directory
will be empty and nothing changes so far. However users that
are familiar with common maven features and plugins can
generate file to ${project.outputDirectory) and are also happy.
Further you can also mix both ways.

Please note that using existing plugins will offer you features
that you have not implemented and some of which you will never
implement. There is ultimate flexibility then.

What do you think?

>
> --
> Trygve

Regards
  Jörg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpNE2cACgkQmPuec2Dcv/+HFQCfTZN8/MaqAF5RaVQ13Sk2XHHB
T3YAn2I+4VaEihWqc9vGpX6m4LmXTUhr
=NMZ4
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Trygve Laugstøl]

Joerg Hohwiller wrote:

> Hi Trygve,
>
> thanks for your reply. Now worries for the delay.
> Usually I am the one who gets lost for a long time...
>
>>>> Mainly because those utils one or more of 1) buggy, 2) slow, 3)
>>>> incomplete and 4) not sufficient.
>>> I see that problem but do NOT agree with the general argument.
>>>
>>> 1) we are a community and progress is made
>>> 2) not very important to me
>>> 3) see 1)
>> 1) and 3) is an issue because 1) it will take me *way* longer time to
>> implement the features that I want and 2) I'm not even sure that their
>> goals match what I want to do so it is even hard to "fix" them.
>
> I still do not see the real bugs that are left in
> resource-plugin and dependency-plugin.
> Both have made great progress and I am really happy with them...
> But yes, some of the fixes took ages in the last 2 years,
> especially for resource-plugin.

The bugs comment was a general comment.

>> Speed is an important aspect when you're trying to assemble enough of
>> these packages.
>
> OK. Point for you.
>
>>> 4) important topic, see below
>>>
>>>> You didn't mention the assembly plugin but that also falls into that
>>>> list.
>>> jup. But it also does some reinvent the wheel already.
>>>
>>>> I know we had the discussion about not reinventing the wheel, which I
>>>> still stand by but I've come to the conclusion that this is not
>>>> reinventing the wheel and that it was required.
>>> I see, but I am not convinced.
>>>
>>>> Features that I can support that the other plugins can't support:
>>>>
>>>> o Streaming of data directly from inside a file in the repository.
>>> OK. understood.
>>> Definite point for 2) (performance)
>>>
>>>>  I can pick out a file from inside another JAR directly from the
>>>> artifact. This saves a copy to the local directory, another full copy of
>>>> the extracted artifact.
>>> Same as above.
>> Also, no other plugin support this.
>
> OK, but "just" a performance issue.
>
>>>> o Uniform renaming of files
>>> Are you talking about those Windows limitations?
>> No, I can use regular expressions to rename files, for example to remove
>> version strings on a file (foo-1.0.jar => foo.jar)
>
> dependency-plugin can handle this as well.
> For resources there is typically no such need,
> because you have them under your control.

No it can't.

This is from the Jetty integration test:

<extractArtifact>
  <artifact>org.mortbay.jetty:jetty-assembly:zip</artifact>
  <to>/opt/jetty</to>
  <includes>
    <include>**/bin/*</include>
  </includes>
  <pattern>/jetty-7.0.0pre3/bin/(.*)</pattern>
  <replacement>bin/$1</replacement>
  <fileAttributes>
    <mode>0755</mode>
  </fileAttributes>
</extractArtifact>

This will pick out all the files under **/bin/* and rename them to
/opt/jetty/bin/$1

>>>> o File attributes like owner, group owner and permissions. In addition
>>>> there are the package format specific attributes like %doc and %config
>>>> for RPM and "request", "space" for pkg.
>>> Yep. This can not be done by other plugins and is something to be
>>> solved by unix-maven-plugin.
>>>
>>>> It would be possible to use the other plugins that you mention to do one
>>>> half of the work, but then the plugin would have to do the other half of
>>>> work through a similar mechanism to what I do today.
>>> exactly.
>>>
>>>> That is where I started and was what I implemented. Once I implemented a
>>>> "set attributes"-like thing and created a good model of a file system,
>>>> adding the rest of the features was easy.
>>> OK.
>>>
>>>> It has even spun of a whole new set of operations that I want to
>>>> implement, in addition to document how to create your own operations.
>>>> For example:
>>>>
>>>> o <touch> for creating empty files.
>>> Just add an empty file to src/main/resources.
>> I would still have to add xml in my pom to control the other file
>> attributes, and creating support for <touch> is a no-brainer, so why not
>> add the convenience?
>
> Typically you will have defaults for file permission and ownership.
> I also added support to have rules via patterns (glob or regexp).
> So I can do chmod a+x on **/*.sh, etc.
> In such case I will typically have no additional config.

I'm not comparing the amount of config required for unix-m-p compared to
pkg-m-u, I'm comparing to the typical use you have with the solaris-m-p
(which is what a lot of my projects use. For example; copying a single file:

<plugin>
 <artifactId>maven-dependency-plugin</artifactId>
 <executions>
   <execution>
     <id>unpack</id>
     <phase>generate-sources</phase>
     <goals>
       <goal>unpack</goal>
     </goals>
     <configuration>
       <artifactItems>
         <artifactItem>
          <groupId>no.java.portal.solaris</groupId>
          <artifactId>solaris-pkg-support</artifactId>

<outputDirectory>${project.build.directory}/solaris-pkg-support</outputDirectory>
          </artifactItem>
        </artifactItems>
      </configuration>
    </execution>
  </executions>
</plugin>

That's a lot of configuration. And you still need to configure the
permissions.

And the next chunk is as bit which involves the assembly plugin in
*addition* to its assembly.xml.

>>>> o <echo> for creating files with hardcoded content from the pom.xml.
>>>> Hard coded means from when the plugin run so you can use expressions
>>>> there. Think "${version}".
>>> Just add that as file to src/main/resources and use filtering.
>>> You would already have this feature if you would not
>>> have reinvented the wheel ;)
>> I'm not going to require users to learn another plugin and at *lots* of
>> xml when it can be done so much easier.
>
> What lots of xml? Did you look at my example. I guess you will
> need more XML to do the same thing with you plugin.
>
>>>> There are more example, but I don't have my notebook here.
>>> Give me all arguments...
>> I've given you all my arguments, but I can summarize:
>
>> 1) Existing plugins are missing required features
>
>> 2) Ease of use for users
>
>>>>> Did you have a look at pkg-maven-plugin.
>>>>> It is just raw minimum but it should just
>>>>> show how simple you can make maven to
>>>>> build packages with all your custom needs.
>>>>> Should I also add an example project
>>>>> or is it self-explanatory.
>>>> I haven't done an in-depth look yet, but from the browse I've given it
>>>> from subversion it seems do to about the same things that I do. I think
>>>> an example pom would be useful to see how it is actually used.
>>> I will add some example project to pkg-maven-plugin/src/it/example1
>>> at the beginning of next week. This week I am quite busy (filing
>>> all those Galileo bugs, etc.)
>>>
>>>> I see that there isn't much code there which definitely is a positive
>>>> attribute. I realize that my plugin is quite huge in number of lines
>>>> (and number of modules) but I think the relevant parts of the plugin is
>>>> quite small.
>>> What I checked in can only create zip and tar so far which is kind
>>> of pointless. Anyhow tar already allows testing permissions, symlinks,
>>> etc.
>>>
>>>> I'm looking forward to the example pom, it seems that you have some
>>>> smart ideas on how to set/change the file attributes there.
>>> I simply re-implemented (the wheel) what unix/posix defines with
>>> umaks, chmod, chown. In that point our solution fits because
>>> here I also have a virtual FS concept but so simple that I did not
>>> mention before.
>> This is in my opinion not re-invention, it is new software/features
>> required to implement the plugins that we work on. I have the same
>> concept which is very central in the assembly part and metadata file
>> generation (.spec, debian/control, pkginfo, prototype) of the plugin.
>
>> I argue that I'm not re-inventing the wheel, but using existing code,
>> just like the other plugins are using existing code. To me, reuse is as
>> much about reusing code as reusing the actual plugins.
>
> I want to draw a conclusion:
> You have already implemented cool features and those have
> better performance that I belief without measuring.
> My goal is NOT to convince you to give this up and kick it out.
> What do you think of bringing both approaches together.
> You will simply add everything in ${project.outputDirectory)
> to the package as well. In you current use-case that directory
> will be empty and nothing changes so far. However users that
> are familiar with common maven features and plugins can
> generate file to ${project.outputDirectory) and are also happy.
> Further you can also mix both ways.

That was one of the first things I wanted to implement in the plugin but
I've done many rounds of design on it and it is finally done:

To summarize:

src/main/unix/files goes in all packages, all formats
src/main/unix/files-rpm goes in all packages, "rpm" format only
src/main/unix/files-a goes in package "a", all formats
src/main/unix/files-b goes in package "b", all formats
src/main/unix/files-b-rpm goes in package "b", "rpm" format only

In addition to these directories it will suck in the files from
target/unix/files and the similar permutations (IIRC)

http://jira.codehaus.org/browse/MUNIX-19

This is very similar to the script mechanism.

> Please note that using existing plugins will offer you features
> that you have not implemented and some of which you will never
> implement. There is ultimate flexibility then.

Definitely and I've never meant to say otherwise. I just want to make
the plugin easy to use for most cases when only simple assembly is required.

--
Trygve


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jörg Hohwiller]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi there,

> [cut.]
> That was one of the first things I wanted to implement in the plugin but
> I've done many rounds of design on it and it is finally done:
>
> To summarize:
>
> src/main/unix/files goes in all packages, all formats
> src/main/unix/files-rpm goes in all packages, "rpm" format only
> src/main/unix/files-a goes in package "a", all formats
> src/main/unix/files-b goes in package "b", all formats
> src/main/unix/files-b-rpm goes in package "b", "rpm" format only
>
> In addition to these directories it will suck in the files from
> target/unix/files and the similar permutations (IIRC)

"In addition" means that stuff under src/main/unix will not
be processed by resources-plugin but directly go into the
package without filtering?
But to have filtering I would
just add my stuff to src/main/resources/files
and configure filtering and output directory as
target/unix?

Great. Seems as if I am convinced now.
Sorry for the long way ;)

The only thing left might be to think over
the permission stuff.
You already have <set-attributes>.
But if I get it right, you have to
specify entire directories (no glob- or
regexp-patterns to select files and folders
of the virtual fs) and entire mode-masks (no
chmod-like a+x g-r).
Maybe here some inspiration from my approach
could help to improve.

One more question:
To build different formats from the same module,
I would have to change the packaging
in pom.xml, right?
I used a virtual packaging "pkg" and
configured the actual packaging as plugin
config. This allows to build different
formats using -D options.
Anyways its just fine if it is
a strict packaging...

>
> http://jira.codehaus.org/browse/MUNIX-19
>
> This is very similar to the script mechanism.
>
>> Please note that using existing plugins will offer you features
>> that you have not implemented and some of which you will never
>> implement. There is ultimate flexibility then.
>
> Definitely and I've never meant to say otherwise. I just want to make
> the plugin easy to use for most cases when only simple assembly is required.

Just perfect. I will stop begging you with the
reinvent-the-wheel topic. And rather focus on
how I can add new value to your existing work.

For package-dependencies my thoughts are currently:

1. Avoid the term dependency/-cies because
it might cause confusion with maven and other things.
My suggestion is to call it "relationships"
which is also the super-term used in debian and rpm
for all this kind of stuff:
http://www.debian.org/doc/debian-policy/ch-relationships.html
http://www.rpm.org/max-rpm/s1-rpm-depend-manual-dependencies.html

2. Ability to configure such relation(ship)s.
2.a. Explicitly structure them as list rather than
comma-separated string.
2.b. Separate package-name and version(range) as
debian control and rpm spec use different
syntax. For ultra specific variants we may
NOT want to handle, people can still write
everything in one string and omit version.
2.c. Define abstract normalized types of
relationships and specify how they map in
specific formats.
I am not ready with this, but my mind is
already quite clear.

To give you an idea here is some example:

<relationships>
  <relationship>
    <name>libc6</name>
    <!-- optional -->
    <version>>=2.2.1</version>
    <type>required</type>
  </relationship>
</relationships>

this could mean the following:
[Debian]
Pre-Depends: libc6 (>=2.2.1)
[RPM]
requires: libc6 >=2.2.1
[ZIP/TAR/...]
(nothing)

Does everyone agree that
RPM-requires is equivalent
to Debian-Pre-Depends rather
than Debian-Depends?

Feedback is very welcome.

Regards
  Jörg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkpTpasACgkQmPuec2Dcv/+5ywCgmNh5ydoQUR6BkK4b2SK4rDfu
MMUAn09oPv6LysCHNOVipwz/imcLe3RR
=xUb+
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Trygve Laugstøl]

Joerg Hohwiller wrote:

> Hi there,
>
>> [cut.]
>> That was one of the first things I wanted to implement in the plugin but
>> I've done many rounds of design on it and it is finally done:
>
>> To summarize:
>
>> src/main/unix/files goes in all packages, all formats
>> src/main/unix/files-rpm goes in all packages, "rpm" format only
>> src/main/unix/files-a goes in package "a", all formats
>> src/main/unix/files-b goes in package "b", all formats
>> src/main/unix/files-b-rpm goes in package "b", "rpm" format only
>
>> In addition to these directories it will suck in the files from
>> target/unix/files and the similar permutations (IIRC)
>
> "In addition" means that stuff under src/main/unix will not
> be processed by resources-plugin but directly go into the
> package without filtering?
> But to have filtering I would
> just add my stuff to src/main/resources/files
> and configure filtering and output directory as
> target/unix?
>
> Great. Seems as if I am convinced now.
> Sorry for the long way ;)

It probably should be filtered. Filtering is something I also have on my
TODO list, but I'm not sure how to do it or to what extent to do it.
Ideas are welcome.

> The only thing left might be to think over
> the permission stuff.
> You already have <set-attributes>.
> But if I get it right, you have to
> specify entire directories (no glob- or
> regexp-patterns to select files and folders
> of the virtual fs) and entire mode-masks (no
> chmod-like a+x g-r).
> Maybe here some inspiration from my approach
> could help to improve.

It support includes/excludes.

On setting the whole pattern that is the only thing that made sense for
all the other operations as they create the file at the same time (so
there are no existing modes to modify).

However, as you note that is not the case for set attributes. Actually
it is not a case for the other operations either as they should be based
on the default values.

I will definitely add support for modifying only parts of the mode bits.

> One more question:
> To build different formats from the same module,
> I would have to change the packaging
> in pom.xml, right?
> I used a virtual packaging "pkg" and
> configured the actual packaging as plugin
> config. This allows to build different
> formats using -D options.
> Anyways its just fine if it is
> a strict packaging...

That really violate how Maven is supposed to work, and you will probably
get an exception if you only have attached files to install. I remember
getting some exceptions on that at one time.

What I've done in those cases it so create a "unix-zip" as the primary
artifact and use the other goals (package-{rpm,deb,sysv-pkg}-attached)
to create the specific packages.

Right now each of those goals has to be configured in a profile if you
want to create a very generic pom that will build on as many platforms
as possible, but I might add a flag to automatically skip formats that
are not supported.

>> http://jira.codehaus.org/browse/MUNIX-19
>
>> This is very similar to the script mechanism.
>
>>> Please note that using existing plugins will offer you features
>>> that you have not implemented and some of which you will never
>>> implement. There is ultimate flexibility then.
>> Definitely and I've never meant to say otherwise. I just want to make
>> the plugin easy to use for most cases when only simple assembly is required.
>
> Just perfect. I will stop begging you with the
> reinvent-the-wheel topic. And rather focus on
> how I can add new value to your existing work.

Excellent! :)

I'm looking forward to it.

> For package-dependencies my thoughts are currently:
>
> 1. Avoid the term dependency/-cies because
> it might cause confusion with maven and other things.
> My suggestion is to call it "relationships"
> which is also the super-term used in debian and rpm
> for all this kind of stuff:
> http://www.debian.org/doc/debian-policy/ch-relationships.html
> http://www.rpm.org/max-rpm/s1-rpm-depend-manual-dependencies.html

That's a good point, I'll rename what I'm working on now.

> 2. Ability to configure such relation(ship)s.
> 2.a. Explicitly structure them as list rather than
> comma-separated string.
> 2.b. Separate package-name and version(range) as
> debian control and rpm spec use different
> syntax. For ultra specific variants we may
> NOT want to handle, people can still write
> everything in one string and omit version.
> 2.c. Define abstract normalized types of
> relationships and specify how they map in
> specific formats.
> I am not ready with this, but my mind is
> already quite clear.
>
> To give you an idea here is some example:
>
> <relationships>
>   <relationship>
>     <name>libc6</name>
>     <!-- optional -->
>     <version>>=2.2.1</version>
>     <type>required</type>
>   </relationship>
> </relationships>
>
> this could mean the following:
> [Debian]
> Pre-Depends: libc6 (>=2.2.1)
> [RPM]
> requires: libc6 >=2.2.1
> [ZIP/TAR/...]
> (nothing)

I've started to work on a very manual dep^H^H^H relationship mechanism
right now which will be just a simple list of dependencies that you can
specify in <deb>. A manual way to control the dependecies will most
likely always be required so I'll start there so satisfy some customers.

One issue I can't really figure out is the very different naming all the
distributions use for the different packages, even core packages. RPM
also can specify a dependency on files (like "/bin/sh has to be
present"). Your thoughs on this would be nice.

> Does everyone agree that
> RPM-requires is equivalent
> to Debian-Pre-Depends rather
> than Debian-Depends?

I'm not sure about that exact example, but there probably isn't much
harm in defining them as such from what I can read on the different
definitions.

--
Trygve

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Jörg Hohwiller]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

> One issue I can't really figure out is the very different naming all the
> distributions use for the different packages, even core packages. RPM
> also can specify a dependency on files (like "/bin/sh has to be
> present"). Your thoughs on this would be nice.

I do NOT beliefe that we will be able to solve the problem about
normalized package names. There is no rules for that and the only
cache is to build and maintain a huge "database" for that.
Some other OSS projects have already tried this but all failed.
I would give up on this and rather let the users allow a way
to optinally specify a package-name per package format that
overrides the default.
Why normalizing this at all?
Simply because users will often define relationships to their
own packages if they build many for the project.
What if a project wants to build and provide their packages
for debian, SLES, etc.
Generally if I decide between structured or unstructured
data I always prefer structured data. It is easy to build
the comma-separated list with brackets, etc. from the stuctured
data but difficult vice versa.
Just think if a maven pom.xml was like this:

<dependencies>org.springframework:spring-core:2.5.6,xalan:xalan:2.7.1:exclude[:xalan:serializer]</dependencies>

However for advanced features there might also be a way to
configure a string that goes directly into the
relationship metadata of the package (appended
with the stuff from the structured relations-ships).
Maybe the latter idea would also address features
such as depend on files that is not supported
by some package-formats.

>
>> Does everyone agree that
>> RPM-requires is equivalent
>> to Debian-Pre-Depends rather
>> than Debian-Depends?
>
> I'm not sure about that exact example, but there probably isn't much
> harm in defining them as such from what I can read on the different
> definitions.

- From my experience in debian if a relationship is missing
at it is only "Depends" then the package is installed but not
configured while "Pre-Depends" will cause the complete
installation to fail.

This is what happens in RPM with "requires".

>
> --
> Trygve

Regards
  Jörg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkphCMQACgkQmPuec2Dcv/95pQCfesMYQiPD6SDhMF+VVhWT78W2
b5wAnj8zNuMo7EG0WPqGDKIdK7b91ZXZ
=Hkd3
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email