problems with final link for xml-security-c and solaris

[Philip Brown-10]

I'm trying to compile xml-security-c, in 64bits, in solaris.

I have had no problems compiling all the myriad dependancies required.
However, when it comes to xml-security-c, I hit a silent failure to link.

Configuration happens with little problem. I have tried version 1.4.0, and 1.5.1
I have tried with, and without, xalan-c.

All with basically the same result.
Things get hidden in libtool obscurity:


/bin/bash ../libtool --mode=link CC -m64  -xO2 -KPIC -mt -norunpath -mt
-D_REENTRANT  -DXSEC_LIBRARY_BUILD -xO2 -DNDEBUG  -R/var/local/64/lib
-L/var/local/64/lib -o libxml-security-c.la -rpath /var/local/64/lib
-version-info 15:1 XSECC14n20010315.lo XSECXMLNSStack.lo XSECCanon.lo
   ......
  OpenSSLCryptoKeyHMAC.lo  -L/var/local/64/lib -lxalan-c -L/var/local/64/lib
-lxerces-c -lsocket -lm  -lpthread -lxalanMsg -lcrypto
(cd .libs && rm -f libxml-security-c.so.15 && ln -s
libxml-security-c.so.15.0.1 libxml-security-c.so.15)
(cd .libs && rm -f libxml-security-c.so && ln -s libxml-security-c.so.15.0.1
libxml-security-c.so)
creating libxml-security-c.la
(cd .libs && rm -f libxml-security-c.la && ln -s ../libxml-security-c.la
libxml-security-c.la)
gmake[1]: Leaving directory
`/auto/src/common/local/xml-security-c/xml-security-c-1.5.1-sparc64/lib'
Making all in bin

As you can see, it seems to continue as if all is well.
However, all it has succeeded indoing, is creating a  libxml-security-c.la,
without an actual shared library??

(i hate libtool, for just this reason!)

Then it eventually fails, a little later, with

CC -m64 -xO2 -KPIC -norunpath -D_REENTRANT -DXSEC_LIBRARY_BUILD -xO2
-DNDEBUG -o .libs/xtest xtest.o  -mt -L/var/local/64/lib
../lib/.libs/libxml-security-c.so -lxalan-c -lxerces-c -lsocket -lm
-lpthread -lxalanMsg -lcrypto
ld: fatal: file ../lib/.libs/libxml-security-c.so: open failed: No such file
or directory
ld: fatal: File processing errors. No output written to .libs/xtest


The failure is not surprising, because "../lib/.libs/libxml-security-c.so"
is now a symlink -> libxml-security-c.so.15.0.1
which does not exist.

Can someone help me out here please?

Solaris 10, CC -V
CC: Sun C++ 5.9 SunOS_sparc Patch 124863-01 2007/07/25

I've just built everything from scratch today.

I thought there was some magic libtool command to use the .la file, and
force generation of the shared lib, to somehow debug that process.... but I
have no idea what that is.

[Scott Cantor]

Philip Brown wrote on 2009-10-09:
> I'm trying to compile xml-security-c, in 64bits, in solaris.

I've never done it (the 64-bit case), and since nothing but a Shibboleth
build would be likely to try this, I doubt that it works.

These particular makefiles have a long history of screwing up on every
dynamic linking variant that gets introduced, and I imagine this is just
another case of that. This isn't likely to be a libtool problem, it's
probably an autotools issue related to the non-automake history of this
project.

I have no idea how to attempt a 64-bit build, but if you provide the
necessary instructions in a bug report, I'll take a look when I have a
chance. Needless to say, though, the chance that I would understand how to
fix something on Solaris more than you would is small.

> I thought there was some magic libtool command to use the .la file, and
> force generation of the shared lib, to somehow debug that process.... but
I
> have no idea what that is.

Normally what I do is run the libtool commands to get the actual command
line it's generating and then run the command line myself, but I do that
much more often with compiles than links.

It doesn't really sound like this is the problem, but I guess I'd check to
make sure the bug isn't more with the symlink commands or other shell
behavior after the link completes. That certainly used to come up in the
other projects that didn't have 64-bit support initially and assumed certain
paths.

-- Scott

[Philip Brown-10]

Scott Cantor wrote:
>
> I have no idea how to attempt a 64-bit build, but if you provide the
> necessary instructions in a bug report, I'll take a look when I have a
> chance.

that's pretty easy...just replace

CC=x

with

CC="x -m64"

works for both sun cc and gcc

Although you then have to make sure whatever your $prefix is, is pretty much
empty before you start, or you'll end up trying to link against 32bit libs

[Philip Brown-10]

Scott Cantor wrote:

> Philip Brown wrote on 2009-10-09:
>> I'm trying to compile xml-security-c, in 64bits, in solaris.
>
> I've never done it (the 64-bit case), and since nothing but a Shibboleth
> build would be likely to try this, I doubt that it works.
>
> These particular makefiles have a long history of screwing up on every
> dynamic linking variant that gets introduced, and I imagine this is just
> another case of that. This isn't likely to be a libtool problem, it's
> probably an autotools issue related to the non-automake history of this
> project.
>

Hmm

well, previously, I was using sun cc with stuff like,


CFLAGS="-xO2  -m64 -mt -xnorunpath"
CXXFLAGS="-xO2  -m64 -mt -norunpath"


I just went back and rebuilt xerces2 with   plain gcc/g++ -m64

then rebuilt xml-security-c with same.

it appears to be happier this way.

progress i suppose... but i'd much rather have it work with sun CC !

[Scott Cantor]

Philip Brown wrote on 2009-10-09:
> I just went back and rebuilt xerces2 with   plain gcc/g++ -m64
> then rebuilt xml-security-c with same.
> it appears to be happier this way.
> progress i suppose... but i'd much rather have it work with sun CC !

That's particularly true in the case of Shibboleth because I don't support
gcc on Solaris.

-- Scott

[Philip Brown-10]

A bit more on the solaris, sun CC, 64bit oddessy ;

Rebuilding base stuff and trying again with sun cc, even though I know gcc
"works".


Once again, I'm trying to compile xml-security-c with sun (v12) compiler, in
64bit mode.

I was previously building things with

CXX="CC -m64".

Now I'm trying with

CXX=CC
CXXFLAGS=-xO2  -m64 -mt -norunpath
LDFLAGS="-R/var/local/64/lib -L/var/local/64/lib -m64"

it actually builds pretty well, until the last final link. The complains
about 32bit vs 64bit object mismatch.
Turns out that when libtool is called with

bash ../libtool --mode=link CC  -xO2  -m64 -mt -norunpath -mt -D_REENTRANT
  -DXSEC_LIBRARY_BUILD blah blah...


for some odd reason, it decides to throw away a bunch of its arguments and
instead uses:

CC -G -nolib -hlibxml-security-c.so.15 -o .libs/libxml-security-c.so.15.0.1
   .libs/XSECC14n20010315.o  blah blah...


No more "-m64". So it tries to generate a 32bit final library, and bombs.

This may be a key to the mystery, even though a clean way of how to SOLVE
it, is unclear.


Mandatory statement: Libtool is Eeeevil. (and broken)

[Philip Brown-10]

Philip Brown wrote:
> ....
> This may be a key to the mystery, even though a clean way of how to
> SOLVE it, is unclear.
>

hmm.
well, apparently, installing the most recent libtool (2.2.6a), and then
after the compile fails to link the library, doing

cd lib;  LIBTOOL=libtool gmake -e

makes it link

[Scott Cantor]

Philip Brown wrote on 2009-10-16:
> well, apparently, installing the most recent libtool (2.2.6a), and then
> after the compile fails to link the library, doing
>
> cd lib;  LIBTOOL=libtool gmake -e
>
> makes it link

So presumably starting from scratch with that version is the resolution?
It's probably overdue on my part to regenerate and check in updated libtool
files for most of my projects, and this one.

-- Scott

[Philip Brown-10]

Scott Cantor wrote:
> Philip Brown wrote on 2009-10-16:
>> well, apparently, installing the most recent libtool (2.2.6a), and then
>> after the compile fails to link the library, doing
>>
>> cd lib;  LIBTOOL=libtool gmake -e
>>
>> makes it link
>
> So presumably starting from scratch with that version is the resolution?

maybe, dunno.


> It's probably overdue on my part to regenerate and check in updated libtool
> files for most of my projects, and this one.

I should mention that xmltooling has similar problems, except worse.

it actually CONTINUES the build after silently failing to link
libxmltooling.so.3.0.0


and a similar hack, fixes it.

cd xmltooling; rm libxmltooling.la
LIBTOOL=libtool gmake  -e

[Scott Cantor]

Philip Brown wrote on 2009-10-16:
>> It's probably overdue on my part to regenerate and check in updated
>> libtool files for most of my projects, and this one.
>
> I should mention that xmltooling has similar problems, except worse.

The projects are all checked in with ltmain.sh in svn, so the only way to
get bugfixes is to regenerate that with libtool and update the checked-in
copy. I was basically told that "this is how to do it".

I have no idea *why* I was told to do that, since you still need
automake/autoconf to bootstrap from svn, but that's what I was told. It
seems pointless to me not to just regenerate the libtool files in the
bootstrap and keep it current, so I may just do that. Anybody without
libtool is not likely to have autotools installed either.

-- Scott