pkg-config wisdom

[Murray S. Kucherawy-2]

What's the current general wisdom on using the pkg-config extensions?  I presume there's a reason they've not been incorporated into basic autoconf, so I'm keen to learn what common practices there are toward adopting it into people's builds (or avoiding it).

Cheers,
-MSK
_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ben Pfaff]

"Murray S. Kucherawy" <msk@...> writes:

> What's the current general wisdom on using the pkg-config
> extensions?  I presume there's a reason they've not been
> incorporated into basic autoconf, so I'm keen to learn what
> common practices there are toward adopting it into people's
> builds (or avoiding it).

My specific negative experience is that people run Autoconf
without having pkg-config installed, which fails mysteriously
when the "configure" script runs.  So I recommend also adding
        m4_pattern_forbid([PKG_CHECK_MODULES])
in your configure.ac, so that running "autoconf" itself will fail
in a more transparent manner if pkg-config is not installed
properly.

I imagine that pkg-config has not been integrated into Autoconf
because it does not fit well into the Autoconf philosophy.
--
"Long noun chains don't automatically imply security."
--Bruce Schneier



_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Murray S. Kucherawy-2]

> -----Original Message-----
> From: autoconf-bounces+msk=cloudmark.com@... [mailto:autoconf-
> bounces+msk=cloudmark.com@...] On Behalf Of Ben Pfaff
> Sent: Thursday, October 22, 2009 12:31 PM
> To: autoconf@...
> Subject: Re: pkg-config wisdom
>
> I imagine that pkg-config has not been integrated into Autoconf
> because it does not fit well into the Autoconf philosophy.

Right, so I'm wondering what that philosophy is, I suppose.

I'm also wondering whether some set of macros that follows "if pkg-config installed then use it, else try to find stuff using this heuristic" exists out there someplace that I can use.



_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ben Pfaff]

"Murray S. Kucherawy" <msk@...> writes:

>> I imagine that pkg-config has not been integrated into Autoconf
>> because it does not fit well into the Autoconf philosophy.
>
> Right, so I'm wondering what that philosophy is, I suppose.

Usually Autoconf tests for particular features, by attempting to
compile (etc.) programs that use those features.  pkg-config
instead tests for a particular version of a particular library.

Also, from the introduction to the Autoconf manual:

       Because of its mission, the Autoconf package itself
    includes only a set of often-used macros that have already
    demonstrated their usefulness.  Nevertheless, if you wish to
    share your macros, or find existing ones, see the Autoconf
    Macro Archive (http://autoconf-archive.cryp.to/), which is
    kindly run by Peter Simons <simons@...>.

> I'm also wondering whether some set of macros that follows "if
> pkg-config installed then use it, else try to find stuff using
> this heuristic" exists out there someplace that I can use.

I do not know of one, but I have not looked for one.
--
Ben Pfaff
http://benpfaff.org


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Adam Mercer]

On Thu, Oct 22, 2009 at 14:31, Ben Pfaff <blp@...> wrote:

> I imagine that pkg-config has not been integrated into Autoconf
> because it does not fit well into the Autoconf philosophy.

I use pkg-config quite heavily in one of my projects, I'm just
wondering is there a more "autoconf" way of performing the same task
as I constantly run into different problems related to finding flags
etc.. using pkg-config.

Cheers

Adam


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Alfred M. Szmidt]

   > I imagine that pkg-config has not been integrated into Autoconf
   > because it does not fit well into the Autoconf philosophy.

   I use pkg-config quite heavily in one of my projects, I'm just
   wondering is there a more "autoconf" way of performing the same
   task as I constantly run into different problems related to finding
   flags etc.. using pkg-config.

The way is to simply not use pkg-config, and use AC_CHECK_* functions
to find what is needed; and let the user specify where/what, using
*FLAGS.


pkg-config tries to solve an important problem, but it does so in the
wrong way.  pkg-config checks for an exact library name, and
optionally exact version numbers.  A similar library can exist with
the same API, but with different name and version, in which case any
checks done by PKG_CHECK_* during configure will fail.

Autoconf works by testing specific features, hence why it doesn't fit
with the autoconf philosophy.


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Tim Post-3]

On Thu, 2009-10-22 at 11:44 -0700, Murray S. Kucherawy wrote:
> What's the current general wisdom on using the pkg-config extensions?  
> I presume there's a reason they've not been incorporated into basic
> autoconf, so I'm keen to learn what common practices there are toward
> adopting it into people's builds (or avoiding it).

I have experienced many issues using pkg-config. Most of the issues
stemmed from users compiling and installing their own libraries, where
one (or more) of the following happens:

* Their custom built library is not used, the system's is.

* pkg-config did not locate an installed library, due to the library not
installing the files that pkg-config needs to work

* Conflicts with the users preference regarding CFLAGS

Additionally, you will encounter additional problems on systems ~3 years
old, of which many are still in production.

I really recommend avoiding it. In my experience, it creates more
frustration than it solves.

Regards,
--Tim

--
Monkey + Typewriter = Echoreply ( http://echoreply.us )



_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[mpsuzuki]

Although pkg-config is useful in some cases, I agree with
others' negative evaluation against the idea to builtin
pkg-config support of autoconf. I want autoconf to keep
the library detection without pkg-config.

On Fri, 23 Oct 2009 09:48:30 +0800
Tim Post <echo@...> wrote:
>I have experienced many issues using pkg-config. Most of the issues
>stemmed from users compiling and installing their own libraries, where
>one (or more) of the following happens:

The most popular scenario I think is: the pkg-config
itself is bundled to the system (/usr/bin/pkg-config etc)
but the users install their own libraries to non-system
directory (e.g. /usr/local/xxx), and the users slipped
to set PKG_CONFIG_PATH manually.

>* Their custom built library is not used, the system's is.

Indeed. It might be popular when default pkg-config prefix
is differnt from the prefix that users install their own
libraries. Have you experienced the troubles that pkg-config
in configure process detects the correct location of custom
built library but the system's is used in building process?

>* pkg-config did not locate an installed library, due to the library not
>installing the files that pkg-config needs to work

I'm not sure if this trouble is popular. It sounds as if
the author of Makefile.in (or Makefile.am) slipped to
add (or break) appropriate target to install xxx.pc files,
or install to wrong directory that pkg-config does not
search.

>Additionally, you will encounter additional problems on systems ~3 years
>old, of which many are still in production.

Indeed.

Regards,
mpsuzuki


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Rhys Ulerich-2]

> The most popular scenario I think is: the pkg-config
> itself is bundled to the system (/usr/bin/pkg-config etc)
> but the users install their own libraries to non-system
> directory (e.g. /usr/local/xxx), and the users slipped
> to set PKG_CONFIG_PATH manually.

Definitely very useful, especially in environments where modules [1]
are used.  A custom module load can simply prepend the custom install
location to the PKG_CONFIG_PATH.  Then the custom install's .pc file
will be found first and override any system-wide version.  Amazingly
handy on HPC systems where the compiler+MPI stack+solver version in
use can make or break you.

- Rhys

[1] http://modules.sourceforge.net/


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Bob Friesenhahn]

On Fri, 23 Oct 2009, mpsuzuki@... wrote:
>
> The most popular scenario I think is: the pkg-config
> itself is bundled to the system (/usr/bin/pkg-config etc)
> but the users install their own libraries to non-system
> directory (e.g. /usr/local/xxx), and the users slipped
> to set PKG_CONFIG_PATH manually.

There is also a problem when an updated version of just one library is
installed under a different path but an older version of this library
is located elsewhere and refered by some other package's
configuration.  Configure scripts which trust pkg-config include and
library paths and simpy concatenate them together (often in some
random order) cause big problems for users since the user has no
control over the paths used.

I have found this to be a nightmare under Solaris 10 which has an
older Gnome environment, but some packages want to link with newer
libraries and don't need all of Gnome.  The only workaround has been
to temporarily rename system files.

This is not to say that pkg-config doesn't work well; it is just best
for a carefully built and well integrated environment rather than one
incrementally put together over time.

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[william pursell]

Bob Friesenhahn wrote:

> Configure scripts which trust pkg-config include and library paths and
> simpy concatenate them together (often in some random order) cause big
> problems for users since the user has no control over the paths used.

I don't understand the comment about "random order".  The ordering
is as imposed by the author of configure.ac.   @FOO_LIBS@ @BAR_LIBS@
leads to -L/foo -L/bar, while @BAR_LIBS@ @FOO_LIBS@ leads
to -L/bar -L/foo.

--
William Pursell


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[william pursell]

Alfred M. Szmidt wrote:
>
> pkg-config tries to solve an important problem, but it does so in the
> wrong way.  pkg-config checks for an exact library name,

PKG_CHECK_MODULES does not check for a library name at all,
but for the name of the .pc file.  This gives the administrator
one extra level of indirection.

AC_CHECK_LIB checks for an exact library name.

and
> optionally exact version numbers.  A similar library can exist with
> the same API, but with different name and version, in which case any
> checks done by PKG_CHECK_* during configure will fail.

Only if the author of configure.ac uses the option to check for
a version number.  PKG_CHECK_MODULES can be used without
an explicit version number check, and many people do
explicit version number checks without pkg-config, so this
is not a compelling argument against pkg-config.

>
> Autoconf works by testing specific features, hence why it doesn't fit
> with the autoconf philosophy.

I think that's invalid.  pkg-config does provide the option to
do an explicit version check, but it is the maintainer that
exercises that option that is departing from the autoconf
philosophy.


--
William Pursell


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Bob Friesenhahn]

On Fri, 23 Oct 2009, William Pursell wrote:

>> Configure scripts which trust pkg-config include and library paths and
>> simpy concatenate them together (often in some random order) cause big
>> problems for users since the user has no control over the paths used.
>
> I don't understand the comment about "random order".  The ordering
> is as imposed by the author of configure.ac.   @FOO_LIBS@ @BAR_LIBS@
> leads to -L/foo -L/bar, while @BAR_LIBS@ @FOO_LIBS@ leads
> to -L/bar -L/foo.

It is true that within one configure script the order is not random
but different configure script authors may concatenate the package
information in different orders.  The requirements for packages may
overlap so the same -L options and -I options may appear multiple
times.

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Andrew W. Nosenko]

On Fri, Oct 23, 2009 at 00:50, Alfred M. Szmidt <ams@...> wrote:
> The way is to simply not use pkg-config, and use AC_CHECK_* functions
> to find what is needed; and let the user specify where/what, using
> *FLAGS.

Can I ask hard to me and seems easy to you question:
how I can detect using AC_CHECK_*  2nd, 3rd etc libraries, which need
to resolve symbols in 1st layer library?  For example, I want to link
libxml2.  How I should to obtain LDFLAGS, which would to statisfy
libxml2 link needs?  Whether I need to add -lzlib?  -lpthread?  Or
just -pthread (no library but C compiler flag)?  How using AC_CHECK_*
I can guess _preprocessor_ flags?  (Hint -D_REENTRANT, IIRC)?  How
using the same, or any another check I can guess name of library if
package maintainer changed it for strong for he, but absolutely
unpredictable for me way and reason?  (Hint SleepyCat's Berkeley DB
v4.1 has official name libdb-4.1 from author's (sleepycat) point of
view, but FreeBSD port maintaner(s) decided to rename it to libdb41.
How I can to read his minds?????  How I can to guess that someone
doesn't like characters '-' and '.' so strong that decide to break
compatibility with upstream????)

And please, don't say about "Linux has interlibrary dependency for
shared libraries".  First at all, not all libraries are shared (even
under Linux).  Second, Linux is not only one flavor of Unix.  Third,
sometime developers are forced to use OS'es different from Unix at
all.  And forth, recall the CPPFLAFS!

--
Andrew W. Nosenko <andrew.w.nosenko@...>


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[John Calcote-2]

Hi Andrew,

On 10/23/2009 5:34 PM, Andrew W. Nosenko wrote:

> On Fri, Oct 23, 2009 at 00:50, Alfred M. Szmidt<ams@...>  wrote:
>    
>> The way is to simply not use pkg-config, and use AC_CHECK_* functions
>> to find what is needed; and let the user specify where/what, using
>> *FLAGS.
>>      
> Can I ask hard to me and seems easy to you question:
> how I can detect using AC_CHECK_*  2nd, 3rd etc libraries, which need
> to resolve symbols in 1st layer library?  For example, I want to link
> libxml2.  How I should to obtain LDFLAGS, which would to statisfy
> libxml2 link needs?  Whether I need to add -lzlib?  -lpthread?  Or
>    

If your project uses libxml's API, then you as the maintainer should be
very aware of requisite dependencies of that library. The AC_CHECK_LIB
macro accepts a fifth argument, other-libraries, which is a
whitespace-separated list of dependent libraries (actually command-line
options, e.g., [-lzlib -lpthread -lX11]) that are required by the
primary library that you're checking for. Note, however, that if you've
already tested for the presence of any of these 2nd and 3rd level
libraries in previous AC_CHECK_LIB tests, then these references will
already be in the LIBS variable, and thus used in this test automatically.

> just -pthread (no library but C compiler flag)?  How using AC_CHECK_*
> I can guess _preprocessor_ flags?  (Hint -D_REENTRANT, IIRC)?  How
>    

Preprocessor (CPP) flags are irrelevant, because AC_CHECK_LIB merely
links to the symbol for which you are looking. It doesn't try to execute
the program at all. If the compiler and linker are able to build a
program that references the library symbol, then the check passes.
CPPFLAGS would only matter if the check needed to properly execute code
from the library, which it does not. There is another macro called
AC_RUN_IFELSE, but it's use is discouraged because it assumes about the
environment. Generally, if you can link to a symbol from a library, then
you've accomplished your goal of testing for a symbol from that library.

> using the same, or any another check I can guess name of library if
> package maintainer changed it for strong for he, but absolutely
> unpredictable for me way and reason?  (Hint SleepyCat's Berkeley DB
> v4.1 has official name libdb-4.1 from author's (sleepycat) point of
> view, but FreeBSD port maintaner(s) decided to rename it to libdb41.
> How I can to read his minds?????  How I can to guess that someone
> doesn't like characters '-' and '.' so strong that decide to break
> compatibility with upstream????)
>    

Issues like this are the reason why AC_SEARCH_LIBS was invented. Using
AC_SEARCH_LIBS, you can specify a list of library names from most
probable to least probable in the library argument. The system will
attempt to search for the specified symbol in any of those library
names. In general, unless you know the name of a library on a given
system type, you can't link to that library. Thus, part of porting your
application to a given platform is to find out the proper library names
on that platform, and make sure that AC_SEARCH_LIBS contains that name
in it's list of possible candidates for the desired functionality.

> And please, don't say about "Linux has interlibrary dependency for
> shared libraries".  First at all, not all libraries are shared (even
> under Linux).  Second, Linux is not only one flavor of Unix.  Third,
> sometime developers are forced to use OS'es different from Unix at
> all.  And forth, recall the CPPFLAFS!
>    

I don't know of any flavor of Unix that has "interlibrary dependencies"
as you describe them here - at least most people don't use those linker
options. Additionally, most people on this list are not necessarily
pro-Linux, and anti-Solaris/BSD/MacOSX/AIX, and so on. The Autotools
attempts to be Unix-flavor agnostic, which is one of the strengths of
these tools.

Regards,
John



_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Ben Pfaff]

John Calcote <john.calcote@...> writes:

> If your project uses libxml's API, then you as the maintainer should
> be very aware of requisite dependencies of that library. The
> AC_CHECK_LIB macro accepts a fifth argument, other-libraries, which is
> a whitespace-separated list of dependent libraries (actually
> command-line options, e.g., [-lzlib -lpthread -lX11]) that are
> required by the primary library that you're checking for. Note,
> however, that if you've already tested for the presence of any of
> these 2nd and 3rd level libraries in previous AC_CHECK_LIB tests, then
> these references will already be in the LIBS variable, and thus used
> in this test automatically.

It's too bad how complicated it gets sometimes, though.  On first
reading, I thought that maybe the following quote from the GNU
libplot manual had some kind of embedded joke that I was missing:

       To link your application with GNU `libplot', you would use the
    appropriate `-l' option(s) on the command line when compiling it.  You
    would use

         -lplot -lXaw -lXmu -lXt -lXext -lX11 -lpng -lz -lm

    or, in recent releases of the X Window System,

         -lplot -lXaw -lXmu -lXt -lSM -lICE -lXext -lX11 -lpng -lz -lm

    These linking options assume that your version of `libplot' has been
    compiled with PNG support; if not, you would omit the `-lpng -lz'
    options.

       As an alternative to the preceding, you may need to use `-lplot -lXm
    -lXt -lXext -lX11 -lpng -lz -lm', `-lplot -lXm -lXt -lXext -lX11 -lpng
    -lz -lm -lc -lgen', or `-lplot -lXm -lXt -lXext -lX11 -lpng -lz -lm -lc
    -lPW', on systems that provide Motif widgets instead of Athena widgets.
    In recent releases of the X Window System, you would insert `-lSM
    -lICE'.  Recent releases of Motif require `-lXp' and possibly `-lXpm'
    as well.)

--
Ben Pfaff
http://benpfaff.org



_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Alfred M. Szmidt]

   > pkg-config tries to solve an important problem, but it does so in the
   > wrong way.  pkg-config checks for an exact library name,

   PKG_CHECK_MODULES does not check for a library name at all,
   but for the name of the .pc file.  This gives the administrator
   one extra level of indirection.

The .pc file specifies the library name, so my point stands.  A user
cannot modify the .pc files, so it makes it hard to modify that file
for a user without setting PKG_CONFIG_PATH or similar, in which case
you are back to square one with CFLAGS/LDFLAGS which are far more
flexible.

   > optionally exact version numbers.  A similar library can exist with
   > the same API, but with different name and version, in which case any
   > checks done by PKG_CHECK_* during configure will fail.

   Only if the author of configure.ac uses the option to check for
   a version number.  PKG_CHECK_MODULES can be used without
   an explicit version number check, and many people do
   explicit version number checks without pkg-config, so this
   is not a compelling argument against pkg-config.

Which will break things silently if things are changed in the API from
one version to another.  Checking for the features you need will never
break, since you are checking for the behaviour you want/need.

pkg-config is simply broken in far to many ways.


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Alfred M. Szmidt]

   And please, don't say about "Linux has interlibrary dependency for
   shared libraries".  First at all, not all libraries are shared
   (even under Linux).  Second, Linux is not only one flavor of Unix.

Linux is a kernel, the operating system you are refering to is called
GNU or in conjuction with Linux, GNU+Linux..  Please see
http://www.gnu.org/gnu/linux-and-gnu.html for more information.


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Russell Shaw]

Alfred M. Szmidt wrote:

>    > pkg-config tries to solve an important problem, but it does so in the
>    > wrong way.  pkg-config checks for an exact library name,
>
>    PKG_CHECK_MODULES does not check for a library name at all,
>    but for the name of the .pc file.  This gives the administrator
>    one extra level of indirection.
>
> The .pc file specifies the library name, so my point stands.  A user
> cannot modify the .pc files, so it makes it hard to modify that file
> for a user without setting PKG_CONFIG_PATH or similar, in which case
> you are back to square one with CFLAGS/LDFLAGS which are far more
> flexible.
>
>    > optionally exact version numbers.  A similar library can exist with
>    > the same API, but with different name and version, in which case any
>    > checks done by PKG_CHECK_* during configure will fail.
>
>    Only if the author of configure.ac uses the option to check for
>    a version number.  PKG_CHECK_MODULES can be used without
>    an explicit version number check, and many people do
>    explicit version number checks without pkg-config, so this
>    is not a compelling argument against pkg-config.
>
> Which will break things silently if things are changed in the API from
> one version to another.  Checking for the features you need will never
> break, since you are checking for the behaviour you want/need.
>
> pkg-config is simply broken in far to many ways.

Why?

I regard pkg-config as useful only for one-off libraries that have
no generic substitutes, and for this purpose i haven't figured out
any problems.


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf

[Bob Friesenhahn]

On Fri, 23 Oct 2009, John Calcote wrote:

>
> If your project uses libxml's API, then you as the maintainer should be very
> aware of requisite dependencies of that library. The AC_CHECK_LIB macro
> accepts a fifth argument, other-libraries, which is a whitespace-separated
> list of dependent libraries (actually command-line options, e.g., [-lzlib
> -lpthread -lX11]) that are required by the primary library that you're
> checking for. Note, however, that if you've already tested for the presence
> of any of these 2nd and 3rd level libraries in previous AC_CHECK_LIB tests,
> then these references will already be in the LIBS variable, and thus used in
> this test automatically.

Unfortunately, the libraries used by the library being tested may be
only optionally used and assuming that the library is used may result
in an uneeded library being added to the build.  There may be an
option of using several other libraries (e.g. an XML library like
libxml2 or expat) to perform a function and it is easy to guess wrong
if both option libraries happen to be on the system.

In this case life would be better if all libraries had a ".la" file
and if Autoconf used libtool type functionality (e.g. consult the .la
files) as part of its testing.

> I don't know of any flavor of Unix that has "interlibrary dependencies" as
> you describe them here - at least most people don't use those linker options.
> Additionally, most people on this list are not necessarily pro-Linux, and
> anti-Solaris/BSD/MacOSX/AIX, and so on. The Autotools attempts to be
> Unix-flavor agnostic, which is one of the strengths of these tools.

The Linux linker does seem to automatically inject library
dependencies, which tends to paper over any configuration problems.

Bob
--
Bob Friesenhahn
bfriesen@..., http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/


_______________________________________________
Autoconf mailing list
Autoconf@...
http://lists.gnu.org/mailman/listinfo/autoconf