checking for libraries that link but do not run

[Ben Pfaff]

In GNU PSPP, we have had a number of problems with users who pass
correct "configure" flags to link against a library
(e.g. LDFLAGS='-L/usr/local/lib') but do not pass the correct
flags to let binaries linked against those libraries run
(e.g. LDFLAGS='-Wl,-rpath,/usr/local/lib' or LD_LIBRARY_PATH).
This means that AC_RUN_IFELSE calls after those libraries are
added to LIBS always fail.  This causes bizarre symptoms, such as
build assertion failures in gnulib-generated stdint.h (e.g. see
http://lists.gnu.org/archive/html/pspp-dev/2009-10/msg00002.html).

To try to head off the problem, I'm thinking about putting
something like this after each command that adds to LIBS:
    AC_RUN_IFELSE([AC_LANG_PROGRAM([], [])],
                  [:],
                  [AC_MSG_FAILURE([Cannot run program linked against $LIBS.])],
                  [:])

Am I on the right track?  It seems to me that there should
already be a mechanism to help with this, but I do not see one.

Thanks,

Ben.
--
Ben Pfaff
http://benpfaff.org


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Bob Friesenhahn]

On Mon, 5 Oct 2009, Ben Pfaff wrote:

>
> To try to head off the problem, I'm thinking about putting
> something like this after each command that adds to LIBS:
>    AC_RUN_IFELSE([AC_LANG_PROGRAM([], [])],
>                  [:],
>                  [AC_MSG_FAILURE([Cannot run program linked against $LIBS.])],
>                  [:])
>
> Am I on the right track?  It seems to me that there should
> already be a mechanism to help with this, but I do not see one.

This is a rather complex issue since the problem can be corrected by
LD_LIBRARY_LFLAGS, ldconfig/crle, by formally installing the library,
or by libtool at link time.  Since autoconf does not use libtool in
its testing, it does not benefit from the extra smarts provided by
libtool, and so autoconf tests are more fragile.  For example,
autoconf can't know about library dependencies that libtool may be
able to figure out later (via installed .la files) and so the
configure script developer needs to provide way to add all of these
dependencies to LIBS.

In summary, your idea seems nice, but it might block builds which
would otherwise succeed and work.

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ben Pfaff]

Bob Friesenhahn <bfriesen@...> writes:

> On Mon, 5 Oct 2009, Ben Pfaff wrote:
>>
>> To try to head off the problem, I'm thinking about putting
>> something like this after each command that adds to LIBS:
>>    AC_RUN_IFELSE([AC_LANG_PROGRAM([], [])],
>>                  [:],
>>                  [AC_MSG_FAILURE([Cannot run program linked against $LIBS.])],
>>                  [:])
>>
>> Am I on the right track?  It seems to me that there should
>> already be a mechanism to help with this, but I do not see one.
>
> This is a rather complex issue since the problem can be corrected by
> LD_LIBRARY_LFLAGS, ldconfig/crle, by formally installing the library,
> or by libtool at link time.  Since autoconf does not use libtool in
> its testing, it does not benefit from the extra smarts provided by
> libtool, and so autoconf tests are more fragile.  For example,
> autoconf can't know about library dependencies that libtool may be
> able to figure out later (via installed .la files) and so the
> configure script developer needs to provide way to add all of these
> dependencies to LIBS.
>
> In summary, your idea seems nice, but it might block builds which
> would otherwise succeed and work.

If your advice is correct, then any use of AC_RUN_IFELSE (if any
libraries are added to LIBS) must be incorrect, because Autoconf
does not have the correct knowledge to run a program.  It's hard
for me to believe that, because the Autoconf manual only mentions
problems with AC_RUN_IFELSE when cross-compiling and doesn't say
anything about other pitfalls.
--
Ben Pfaff
http://benpfaff.org


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Bob Friesenhahn]

On Mon, 5 Oct 2009, Ben Pfaff wrote:
>
> If your advice is correct, then any use of AC_RUN_IFELSE (if any
> libraries are added to LIBS) must be incorrect, because Autoconf
> does not have the correct knowledge to run a program.  It's hard
> for me to believe that, because the Autoconf manual only mentions
> problems with AC_RUN_IFELSE when cross-compiling and doesn't say
> anything about other pitfalls.

There are many pitfalls associated with running a program.  If you do
choose to validate that a program will actually run from the build
directory with the currently installed libraries, then you need to be
prepared for dealing with the pitfalls. :-)

Most configure scripts don't seem to try to run programs.  The
configure script for my package does run a program for one of its
tests.

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ben Pfaff]

Bob Friesenhahn <bfriesen@...> writes:

> On Mon, 5 Oct 2009, Ben Pfaff wrote:
>>
>> If your advice is correct, then any use of AC_RUN_IFELSE (if any
>> libraries are added to LIBS) must be incorrect, because Autoconf
>> does not have the correct knowledge to run a program.  It's hard
>> for me to believe that, because the Autoconf manual only mentions
>> problems with AC_RUN_IFELSE when cross-compiling and doesn't say
>> anything about other pitfalls.
>
> There are many pitfalls associated with running a program.  If you do
> choose to validate that a program will actually run from the build
> directory with the currently installed libraries, then you need to be
> prepared for dealing with the pitfalls. :-)
>
> Most configure scripts don't seem to try to run programs.  The
> configure script for my package does run a program for one of its
> tests.

I'm not sure what "be prepared for dealing with the pitfalls"
amounts to.  Can you point to an example of a correct way to deal
with the pitfalls?  What does your package do to deal with them?

Gnulib uses AC_RUN_IFELSE in many places.  I suspect that most
programs that use Gnulib will fall afoul of these problems too.
--
Ben Pfaff
http://benpfaff.org


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Richard Ash-6]

On Mon, 2009-10-05 at 12:13 -0500, Bob Friesenhahn wrote:

> On Mon, 5 Oct 2009, Ben Pfaff wrote:
> >
> > To try to head off the problem, I'm thinking about putting
> > something like this after each command that adds to LIBS:
> >    AC_RUN_IFELSE([AC_LANG_PROGRAM([], [])],
> >                  [:],
> >                  [AC_MSG_FAILURE([Cannot run program linked against $LIBS.])],
> >                  [:])
> >
> > Am I on the right track?  It seems to me that there should
> > already be a mechanism to help with this, but I do not see one.
>
> This is a rather complex issue since the problem can be corrected by
> LD_LIBRARY_LFLAGS, ldconfig/crle, by formally installing the library,
> or by libtool at link time.  Since autoconf does not use libtool in
> its testing, it does not benefit from the extra smarts provided by
> libtool, and so autoconf tests are more fragile.  For example,
> autoconf can't know about library dependencies that libtool may be
> able to figure out later (via installed .la files) and so the
> configure script developer needs to provide way to add all of these
> dependencies to LIBS.

Not to mention that a more sophisticated configure script might work out
that the local source tree contains sources to some library you use, but
they haven't been compiled yet. If you start trying to link against
those libraries, you will get a link error, although by the time you
link your final program, make has built the relevant libraries and you
can link them.

My solution for this problem in Audacity's configure was to keep two
LDFLAGS variables, one set that are the default for configure, and into
which compiler feature flags etc go, and another set which will be used
in the build, but aren't valid yet. The latter are added to the former
either right at the end of configure, or actually in the Makefiles, I
can't remember which.

Richard



_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ben Pfaff]

Richard Ash <richard@...> writes:

> On Mon, 2009-10-05 at 12:13 -0500, Bob Friesenhahn wrote:
>> On Mon, 5 Oct 2009, Ben Pfaff wrote:
>> >
>> > To try to head off the problem, I'm thinking about putting
>> > something like this after each command that adds to LIBS:
>> >    AC_RUN_IFELSE([AC_LANG_PROGRAM([], [])],
>> >                  [:],
>> >                  [AC_MSG_FAILURE([Cannot run program linked against $LIBS.])],
>> >                  [:])
>> >
>> > Am I on the right track?  It seems to me that there should
>> > already be a mechanism to help with this, but I do not see one.
>>
>> This is a rather complex issue since the problem can be corrected by
>> LD_LIBRARY_LFLAGS, ldconfig/crle, by formally installing the library,
>> or by libtool at link time.  Since autoconf does not use libtool in
>> its testing, it does not benefit from the extra smarts provided by
>> libtool, and so autoconf tests are more fragile.  For example,
>> autoconf can't know about library dependencies that libtool may be
>> able to figure out later (via installed .la files) and so the
>> configure script developer needs to provide way to add all of these
>> dependencies to LIBS.
> Not to mention that a more sophisticated configure script might work out
> that the local source tree contains sources to some library you use, but
> they haven't been compiled yet. If you start trying to link against
> those libraries, you will get a link error, although by the time you
> link your final program, make has built the relevant libraries and you
> can link them.

This is a good point, and I think that Bob Friesenhahn might be
pointing out some related issues, but PSPP doesn't add any
libraries that it compiles itself to LIBS, only system libraries,
so this is not the problem that I am trying to solve in my
specific case.
--
Ben Pfaff
http://benpfaff.org



_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Bob Friesenhahn]

On Mon, 5 Oct 2009, Ben Pfaff wrote:
>
> I'm not sure what "be prepared for dealing with the pitfalls"
> amounts to.  Can you point to an example of a correct way to deal
> with the pitfalls?  What does your package do to deal with them?

The unfortunate solution is to do huge amount of configure script
testing and analyze the possible library dependencies and how to
become aware of them.  Library order is often important.  Recent Linux
makes things much easier due good support for implicit shared library
dependencies but this just makes it more likely that your configure
script will only work properly on Linux and only if shared libraries
are installed.  It becomes necessary to test configure on other OSs
without the implicit dependency support.

And then there are the OSs that don't run an executable before it has
been formally blessed, or won't run a binary executable from a network
share, run afoul of a virus checker, or where the mount options fail
to allow execution.

> Gnulib uses AC_RUN_IFELSE in many places.  I suspect that most
> programs that use Gnulib will fall afoul of these problems too.

It seems unlikely that Gnulib is dependent on very many other
libraries.  Once your program/library becomes dependent on over 60
other libraries then you will know that you have arrived.

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ben Pfaff]

Bob Friesenhahn <bfriesen@...> writes:

> On Mon, 5 Oct 2009, Ben Pfaff wrote:
>>
>> I'm not sure what "be prepared for dealing with the pitfalls"
>> amounts to.  Can you point to an example of a correct way to deal
>> with the pitfalls?  What does your package do to deal with them?
>
> The unfortunate solution is to do huge amount of configure script
> testing and analyze the possible library dependencies and how to
> become aware of them.  Library order is often important.  Recent Linux
> makes things much easier due good support for implicit shared library
> dependencies but this just makes it more likely that your configure
> script will only work properly on Linux and only if shared libraries
> are installed.  It becomes necessary to test configure on other OSs
> without the implicit dependency support.

I am well aware that link order is important for making sure that
programs link properly.  But I am talking about a case where
programs linked fine, without warnings or errors, and produced an
executable, but the executable could not run, because the dynamic
linker could not find the shared libraries at runtime.

> And then there are the OSs that don't run an executable before it has
> been formally blessed, or won't run a binary executable from a network
> share, run afoul of a virus checker, or where the mount options fail
> to allow execution.

Yes, those problems will prevent a program from linking but not
running.  Those are the sort that I would want to make
"configure" fail, because they will make the configuration
broken.  I believe that the macro that I suggested at the start
of this thread will do an adequate job of detecting these problem
as well the problem that I originally identified.

>> Gnulib uses AC_RUN_IFELSE in many places.  I suspect that most
>> programs that use Gnulib will fall afoul of these problems too.
>
> It seems unlikely that Gnulib is dependent on very many other
> libraries.  Once your program/library becomes dependent on over 60
> other libraries then you will know that you have arrived.

At that point, I am impressed that your program can work at all.
I have no desire to make my program depend on that many other
libraries.
--
Ben Pfaff
http://benpfaff.org


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Mike Frysinger]

On Monday 05 October 2009 15:21:10 Ben Pfaff wrote:
> Gnulib uses AC_RUN_IFELSE in many places.  I suspect that most
> programs that use Gnulib will fall afoul of these problems too.

gnulib isnt really a relevant example.  the vast majority of gnulib is testing
either the system C library or the operating system, neither of which require
any libraries at all (since the default toolchain already provides them).  
plus, their examples are searching for specific/known bugs in various
implementations -- they arent doing sanity checks that the library in question
is "available" and "usable".  spurious running of code by configure scripts is
the bane of anyone doing cross-system builds and really should be avoided
unless truly necessary.

your original issue (people linking against libraries but then failing to
properly configure their system such that the ldso can find them at runtime)
sounds like pure PEBKAC to me.  i'd personally write a FAQ and tell users to
RTFF instead of bloating the autotool code to cater to such misbehavior.
-mike


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ralf Wildenhues]

* Ben Pfaff wrote on Mon, Oct 05, 2009 at 06:20:47PM CEST:

> In GNU PSPP, we have had a number of problems with users who pass
> correct "configure" flags to link against a library
> (e.g. LDFLAGS='-L/usr/local/lib') but do not pass the correct
> flags to let binaries linked against those libraries run
> (e.g. LDFLAGS='-Wl,-rpath,/usr/local/lib' or LD_LIBRARY_PATH).
> This means that AC_RUN_IFELSE calls after those libraries are
> added to LIBS always fail.  This causes bizarre symptoms, such as
> build assertion failures in gnulib-generated stdint.h (e.g. see
> http://lists.gnu.org/archive/html/pspp-dev/2009-10/msg00002.html).
>
> To try to head off the problem, I'm thinking about putting
> something like this after each command that adds to LIBS:
>     AC_RUN_IFELSE([AC_LANG_PROGRAM([], [])],
>                   [:],
>                   [AC_MSG_FAILURE([Cannot run program linked against $LIBS.])],
>                   [:])
>
> Am I on the right track?  It seems to me that there should
> already be a mechanism to help with this, but I do not see one.

One possibility is to use the gnulib havelib module.

But apart from that, I also think that Autoconf should make it easier
for the developer to state:

- At this point I require the compile command to be able to link an
  executable (AC_PROG_{CC,..} don't do this consistently enough)

- At this point I require being able to run an executable (in non-
  cross-compile mode).

Thanks,
Ralf


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ben Pfaff]

Ralf Wildenhues <Ralf.Wildenhues@...> writes:

> * Ben Pfaff wrote on Mon, Oct 05, 2009 at 06:20:47PM CEST:
>> To try to head off the problem, I'm thinking about putting
>> something like this after each command that adds to LIBS:
>>     AC_RUN_IFELSE([AC_LANG_PROGRAM([], [])],
>>                   [:],
>>                   [AC_MSG_FAILURE([Cannot run program linked against $LIBS.])],
>>                   [:])
>
> One possibility is to use the gnulib havelib module.

Thank you for pointing that out.  I had never looked at the
havelib module before.  It might fix the problem too.
--
"doe not call up Any that you can not put downe."
--H. P. Lovecraft



_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ralf Wildenhues]

* Ralf Wildenhues wrote on Thu, Oct 08, 2009 at 07:27:47AM CEST:
> But apart from that, I also think that Autoconf should make it easier
> for the developer to state:
>
> - At this point I require the compile command to be able to link an
>   executable (AC_PROG_{CC,..} don't do this consistently enough)

This one was wrong.  The defaults should probably be the other way
round: the AC_PROG_* macros should require to be able to link, and
packages which want it differently (such as GCC, binutils, Libtool)
should announce that through some macro.  The thing I'm not yet sure
of is whether this will work smoothly with AC_PROG_LIBTOOL.

> - At this point I require being able to run an executable (in non-
>   cross-compile mode).


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf