[gnu.org #742057] Make standard targets

[Jason Self via RT]

Forwarding...

> [Dave@... - Mon Apr 09 18:50:57 2012]:
>
> http://www.gnu.org/prep/standards/html_node/Standard-Targets.html
>
> Where would I communicate the request that the above standard should
> include a “binary-distribution" target that deletes everything except
> files necessary for a “build install”.
>
> For example, I would build gcc on one server, then do a “make binary-
> distribution”, then rsync it to other servers, where I can to a make
> install.
>
> To go along with this,
> there should be a make target that builds a runtime-requirements file,
> containing an inventory of everything that will be required for the
> package to run, and
> there should be a program that compares the information in runtime-
> requirements with the current system to diagnose any problems that
> might interfere with an install.
> Thanks
>
>
>

--

--
Support the FSF as an Associate Member: http://www.fsf.org/associate/

[Alfred M. Szmidt]

   > [Dave@... - Mon Apr 09 18:50:57 2012]:
   >
   > http://www.gnu.org/prep/standards/html_node/Standard-Targets.html
   >
   > Where would I communicate the request that the above standard should
   > include a “binary-distribution" target that deletes everything except
   > files necessary for a “build install”.
   >
   > For example, I would build gcc on one server, then do a “make
   > binary-distribution”, then rsync it to other servers, where I can
   > to a make install.
   >
   > To go along with this, there should be a make target that builds
   > a runtime-requirements file, containing an inventory of
   > everything that will be required for the package to run, and
   > there should be a program that compares the information in
   > runtime-requirements with the current system to diagnose any
   > problems that might interfere with an install.

You can easily do this using `make install DESTDIR=...' where DESTDIR
specifies a temporary directory.  E.g.

./configure --prefix=/usr/local
make install DESTDIR=/tmp/FOO
tar -C /tmp/FOO -cvf /tmp/FOO.tar .

Now you can easily install something.tar on your systems in
/usr/local; or even rsync /tmp/FOO to your targets.

[Dave Yost]

Thanks.

Thinking about this some more, being able to change the prefix with minimal rebuilding is a feature that goes along with my proposal.

Examples:

1. I installed temporarily into /usr/local/gcc-4.7.0/ until we decide that we are ready to cut over. Then I will install into /usr/local/. It would be nice to be able to edit a file, say “config-prefix”, then do a make install, and not have to wait hours.

2. After doing the proposed “make binary-distribution”, someone who receives the resulting pruned tree could edit the config-prefix file and do a make install into their desired destination.



On Tue 2012-04-10, at 02:27 AM, ams@... (Alfred M. Szmidt) wrote:

>> [Dave@... - Mon Apr 09 18:50:57 2012]:
>>
>> http://www.gnu.org/prep/standards/html_node/Standard-Targets.html
>>
>> Where would I communicate the request that the above standard should
>> include a “binary-distribution" target that deletes everything except
>> files necessary for a “build install”.
>>
>> For example, I would build gcc on one server, then do a “make
>> binary-distribution”, then rsync it to other servers, where I can
>> to a make install.
>>
>> To go along with this, there should be a make target that builds
>> a runtime-requirements file, containing an inventory of
>> everything that will be required for the package to run, and
>> there should be a program that compares the information in
>> runtime-requirements with the current system to diagnose any
>> problems that might interfere with an install.
>
> You can easily do this using `make install DESTDIR=...' where DESTDIR
> specifies a temporary directory.  E.g.
>
> ./configure --prefix=/usr/local
> make install DESTDIR=/tmp/FOO
> tar -C /tmp/FOO -cvf /tmp/FOO.tar .
>
> Now you can easily install something.tar on your systems in
> /usr/local; or even rsync /tmp/FOO to your targets.
>
>

[Dave Yost]

Actually, I think the configure “prefix” feature should be deprecated.

Software should be written to find everything it needs at runtime relative to the path of the invoked binary.

On Tue 2012-04-10, at 02:27 AM, ams@... (Alfred M. Szmidt) wrote:

>> [Dave@... - Mon Apr 09 18:50:57 2012]:
>>
>> http://www.gnu.org/prep/standards/html_node/Standard-Targets.html
>>
>> Where would I communicate the request that the above standard should
>> include a “binary-distribution" target that deletes everything except
>> files necessary for a “build install”.
>>
>> For example, I would build gcc on one server, then do a “make
>> binary-distribution”, then rsync it to other servers, where I can
>> to a make install.
>>
>> To go along with this, there should be a make target that builds
>> a runtime-requirements file, containing an inventory of
>> everything that will be required for the package to run, and
>> there should be a program that compares the information in
>> runtime-requirements with the current system to diagnose any
>> problems that might interfere with an install.
>
> You can easily do this using `make install DESTDIR=...' where DESTDIR
> specifies a temporary directory.  E.g.
>
> ./configure --prefix=/usr/local
> make install DESTDIR=/tmp/FOO
> tar -C /tmp/FOO -cvf /tmp/FOO.tar .
>
> Now you can easily install something.tar on your systems in
> /usr/local; or even rsync /tmp/FOO to your targets.
>
>

[Peter Johansson-6]

On 04/11/2012 09:01 AM, Dave Yost wrote:
> Actually, I think the configure “prefix” feature should be deprecated.
>
> Software should be written to find everything it needs at runtime relative to the path of the invoked binary.
'./configure --prefix=/foo'

means that package should get installed under '/foo', which is a very
useful feature.

http://www.gnu.org/prep/standards/standards.html#Configuration

Peter

[Peter Johansson-6]

[please don't top-post; please keep list in cc]


On 04/11/2012 03:58 PM, Dave Yost wrote:
> I don’t understand the point of your message. What the prefix feature does is well understood. And yes it is better than having the built program work only in a fixed location, such as in /usr/local/.
>
> I’m proposing something that is better than the prefix feature and makes it obsolete.
>
>

In your suggestion, how does 'make install' know where to copy programs?


> On Tue 2012-04-10, at 05:44 PM, Peter Johansson<trojkan@...>  wrote:
>
>> On 04/11/2012 09:01 AM, Dave Yost wrote:
>>> Actually, I think the configure “prefix” feature should be deprecated.
>>>
>>> Software should be written to find everything it needs at runtime relative to the path of the invoked binary.
>

[Alfred M. Szmidt]

   Actually, I think the configure “prefix” feature should be
   deprecated.

   Software should be written to find everything it needs at runtime
   relative to the path of the invoked binary.

The problem with that is two fold, one it would require changing
several hundred of thousands of programs.  Second, it would be
impossible to find system configuration files, libexec type programs,
etc without alot of complicated FOO_PATH and FOO_PROGRAM variables.

It is simple enough to recompile a program, and there is little
benefit in looking things up at run-time.

[Dave Yost]

On Wed 2012-04-11, at 12:03 PM, Alfred M. Szmidt <ams@...> wrote:

> It is simple enough to recompile a program, and there is little
> benefit in looking things up at run-time.

Position independence is generally a good idea. Position dependence is brittle.

Here’s an example: I have two versions of apache, the installed version, and the new version for testing. Once the new version is blessed, I move the installed version from the main installed location to /usr/local/apache-previous or suchlike, and move the staging version into the main installed location. I can still test on the old version for sanity checks by using it in its new location. Having to use make install again twice to make this work is waste and invites errors.

>> Thinking about this some more, being able to change the prefix with
>> minimal rebuilding is a feature that goes along with my proposal.
>
> `make install' doesn't rebuild anything in a already built directory;
> even if you set prefix during `make install' it will not rebuild (if
> it does, it is a bug in the package).

I don’t recall ever reading in configure documentation that prefix= works as an argument to make install

So I tried an experiment
  Z% ./configure prefix=/usr/local/gcc-4.7.0
  Z% make
  Z% make install
  Z% make prefix=/home/yost/gcc-4.7.0 install
  Z% find /home/yost/gcc-4.7.0 -type f | exec tr '\012' '\000' | xargs -0 "$@" strings | grep /usr/local/gcc-4.7.0 | wc
    139     259    7642
  Z% find gcc-4.7.0 -type f | exec tr '\012' '\000' | xargs -0 "$@" strings | grep /home/yost | wc
      0       0       0


>> 1. I installed temporarily into /usr/local/gcc-4.7.0/ until we
>>    decide that we are ready to cut over. Then I will install into
>>    /usr/local/. It would be nice to be able to edit a file, say
>>    “config-prefix”, then do a make install, and not have to wait
>>    hours.
>
> `make install' is fairly quick; if it takes hours then you are doing
> something terribly wrong.

What does take quite a bit of time is running configure again with a different prefix. It was my understanding that you had to do this to change prefix.

>> 2. After doing the proposed “make binary-distribution”, someone who
>>    receives the resulting pruned tree could edit the config-prefix
>>    file and do a make install into their desired destination.
>
> As noted before, DESTDIR provides a more flexible mechanism; so such a
> target is essentially useless.

OK, that’s a perfectly good name for the feature.

>> Actually, I think the configure “prefix” feature should be
>> deprecated.
>>
>> Software should be written to find everything it needs at runtime
>> relative to the path of the invoked binary.
>
> The problem with that is two fold, one it would require changing
> several hundred of thousands of programs.

That is not an argument against a better way of doing things.

> Second, it would be
> impossible to find system configuration files, libexec type programs,
> etc without a lot of complicated FOO_PATH and FOO_PROGRAM variables.

I don’t understand point two. Can you give an example?

[Karl Berry]

Hi Dave,

    Position independence is generally a good idea.

I don't disagree in principle, having expended zillions of hours making
TeX Live be that way.  However, as pointed out, a requirement that all
programs following GNU coding standards become "position independent" at
runtime is a radical change.  If you want to make the argument for it to
rms, feel free.  He's the only one who could make that kind of decision.

Meanwhile, it's certainly true that configure && make is not a package
management system (installing test versions, etc.), but it was never
intended to be.  So that argument is unlikely to fly.

FWIW, for myself, for packages like Apache where I want to control the
version I'm using, I build it with something like
  configure --prefix=/opt/apache-2.2.21
and have a symlink /opt/apache which points to the version-specific
installed tree.  Then it's trivial to change the symlink back and forth.
Works for me.

Of course, for more complicated methods, there is also stow and all the
other multitudes of package management software out there, most of which
I have no knowledge of.

Happy installing,
karl

[Alfred M. Szmidt]

   > It is simple enough to recompile a program, and there is little
   > benefit in looking things up at run-time.

   Position independence is generally a good idea. Position dependence
   is brittle.

   Here’s an example: I have two versions of apache, the installed
   version, and the new version for testing. Once the new version is
   blessed, I move the installed version from the main installed
   location to /usr/local/apache-previous or suchlike, and move the
   staging version into the main installed location. I can still test
   on the old version for sanity checks by using it in its new
   location. Having to use make install again twice to make this work
   is waste and invites errors.

You can achive this better by doing,

./configure --prefix=/usr/local/program-0.01 && make install
./configure --prefix=/usr/local/program-0.02 && make install

where 0.01 is your stable version, and 0.02 is the one you are
testing.  When you are done testing, you can update your start
scripts, or PATH to use the correct version.

I use something like this,

./configure --prefix=~/local/gcc-4.7.0 --exec-prefix=~/local/gcc-4.7.0-$(./config.guess)
make install

so I can have the same version for multiple operating system, and save
some bytes for shared data, and have the same configuration file
(since it will look for it in ~/local/program/etc since --sysconfdir
is ${prefix}/etc by default) across all machines I work on.  Immensly
useful!

   >> Thinking about this some more, being able to change the prefix
   >> with minimal rebuilding is a feature that goes along with my
   >> proposal.
   >
   > `make install' doesn't rebuild anything in a already built
   > directory; even if you set prefix during `make install' it will
   > not rebuild (if it does, it is a bug in the package).

   I don’t recall ever reading in configure documentation that prefix=
   works as an argument to make install

From (standards.info) Directory Variables: "Installers are expected to
override these values when calling `make' (e.g., `make prefix=/usr
install' or `configure' (e.g., `configure --prefix=/usr')."

   So I tried an experiment
     Z% ./configure prefix=/usr/local/gcc-4.7.0
     Z% make
     Z% make install
     Z% make prefix=/home/yost/gcc-4.7.0 install
     Z% find /home/yost/gcc-4.7.0 -type f | exec tr '\012' '\000' | xargs -0 "$@" strings | grep /usr/local/gcc-4.7.0 | wc
       139     259    7642
     Z% find gcc-4.7.0 -type f | exec tr '\012' '\000' | xargs -0 "$@" strings | grep /home/yost | wc
         0       0       0

I am not sure what you are trying to explain here; but I misunderstood
you the first time.  `make install prefix=...' does not recompile the
program, so whatever was used in the program will be what it was when
it was compiled/configured.

You wish to rebuild the program (quickly) with a different prefix in
the compiled executables.  `./configure -C' and ccache go along way in
speeding up compiles in most cases.  There is no other way of doing
that currently, nor would it be possible to do in an easy fashion.

   >> 1. I installed temporarily into /usr/local/gcc-4.7.0/ until we
   >>    decide that we are ready to cut over. Then I will install
   >>    into /usr/local/. It would be nice to be able to edit a file,
   >>    say “config-prefix”, then do a make install, and not have to
   >>    wait hours.
   >
   > `make install' is fairly quick; if it takes hours then you are
   > doing something terribly wrong.

   What does take quite a bit of time is running configure again with
   a different prefix. It was my understanding that you had to do this
   to change prefix.

If you are recompiling/reconfiguring you can use the -C flag to build
a cache to speed up ./configure.

   > Second, it would be impossible to find system configuration
   > files, libexec type programs, etc without a lot of complicated
   > FOO_PATH and FOO_PROGRAM variables.

   I don’t understand point two. Can you give an example?

In the normal case, inetd reads SYSCONFDIR/inetd.conf (where
SYSCONFDIR is PREFIX/etc by default).  If now inetd is changed to
automatically find inetd.conf, where does it find it if I put my inetd
executable in ~/, /usr/libexec and /usr/local/libexec and execute eash
one of those?

Would inetd search the current working directory for it (lots of
security problems there)?  Would it search the directory where the
executable is is for inetd.conf (security again, won't typically exist
if you are hacking)?  Maybe the variants of the previous but ../etc
(~/../etc/inetd.conf would be weird)?

What if my copy in ~/ uses a newer configuration file format?


And what about for example, emacs that uses the libexec program
(executables that are not supposed to be run by a user directly, and
thus not found in PATH) movemail.  How would emacs find this in a
dynamic manner?  Currently it looks at a variable called exec-path
that contains, amongst other, LIBEXECDIR/ which is set during
configure/compile time.

Keep it simple, and don't suprise the user.  The current way is
flexible, simple, and doesn't suprise you.  What you suggest wouldn't
provide much benefit over the existing method.

[Dave Yost]

On Thu 2012-04-12, at 04:09 PM, ams@... (Alfred M. Szmidt) wrote:
>
> And what about for example, emacs that uses the libexec program
> (executables that are not supposed to be run by a user directly, and
> thus not found in PATH) movemail.  How would emacs find this in a
> dynamic manner?  Currently it looks at a variable called exec-path
> that contains, amongst other, LIBEXECDIR/ which is set during
> configure/compile time.

instead, LIBEXECDIR would be set at runtime, relative to the emacs invoked binary.

[Alfred M. Szmidt]

   > And what about for example, emacs that uses the libexec program
   > (executables that are not supposed to be run by a user directly, and
   > thus not found in PATH) movemail.  How would emacs find this in a
   > dynamic manner?  Currently it looks at a variable called exec-path
   > that contains, amongst other, LIBEXECDIR/ which is set during
   > configure/compile time.

   instead, LIBEXECDIR would be set at runtime, relative to the emacs
   invoked binary.

Won't work, where do I find libexec if I put my emacs binary in ~/?
Where is libexec relative to that? What about where all the shared
data is?

(emacs does support setting a different version of movemail using the
rmail-movemail-program variable, but we are talking about totally
dynamic look up here so that isn't relevant)