risk of mixing libstdc++ versions in one executable

[jhfrontz]

[I posted this in gcc-help but didn't really get a satisfying answer]

In reviewing the [gcc-help] archives, I see admonitions to avoid using different versions of libstdc++ in one binary (specifically libstcd++5 and libstdc++6, which results in the confusing-to-the-unsuspecting "/usr/bin/ld: warning: libstdc++.so.5, needed by some-undersupported-IRRATIONAL-INSTRUMENTS-third-party.so, may conflict with libstdc++.so.6" message).

But, really, what is the basis for the warning?  Is it that the binary may in one place use an object created by one library version and then in another place hand the object off to a member function in the other library version?

I'm in a conundrum because I'm dealing with various third-parties' software; I can't get access to everything I need to make them all play well together.  But, I'm hoping it's not so bad--in my case, the C++ code in the third-party software requiring the older library is disjoint from every other bit of C++ code in the process; there is a strict C calling-convention/pass-only-intrinsic-data-types "firewall" between the poorly supported third-party library and the rest of the C++ code in the process.

If I have such a firewall, can I safely ignore the ominous warning from ld?

I've received a suggestion that "the ABIs of libraries differ" but if I "test it and it works, I would say go ahead and ignore the warning."  However, I'm not sure how to test it.  I mean, I can run the resulting executable and it seems to behave well, but I'm not sure if that's an appropriate/sufficient test.  Before I can design an appropriate test, I really need to understand the failure modes that I'm looking for.  What are the failure modes for mixing the two libraries?  Will these failures only occur in code that attempts to make use of a library for which it was not compiled?

To put it another way-- someone had some idea of bad-things-to-come when they added the warning message; what are these bad things and what would induce their appearance?

Also, it's not a flat-out error, so there must be conditions where mixing the libraries is acceptable (or even appropriate)-- what are these conditions?

Ideas?

Thanks,
Jeff

[Ian Lance Taylor-3]

jhfrontz <jeff.frontz@...> writes:

> In reviewing the [gcc-help] archives, I see admonitions to avoid using
> different versions of libstdc++ in one binary (specifically libstcd++5 and
> libstdc++6, which results in the confusing-to-the-unsuspecting "/usr/bin/ld:
> warning: libstdc++.so.5, needed by
> some-undersupported-IRRATIONAL-INSTRUMENTS-third-party.so, may conflict with
> libstdc++.so.6" message).
>
> But, really, what is the basis for the warning?  Is it that the binary may
> in one place use an object created by one library version and then in
> another place hand the object off to a member function in the other library
> version?

When you compile C++ code using libstdc++, some of the template
functions are instantiated in your program, and some of the template
functions come from -lstdc++.  For the template functions in your
program, some have the same name in libstdc++.so.5 and libstdc++.so.6.
When you run your program, any libstdc++ symbols that are included in
the executable will override the symbols in the two versions of the
libstdc++ shared library.  The effect is that code written against
libstdc++.so.5 may call the libstdc++.so.6 routines, and vice-versa.
That fails because the data structures are defined differently.

> I'm in a conundrum because I'm dealing with various third-parties' software;
> I can't get access to everything I need to make them all play well together.
> But, I'm hoping it's not so bad--in my case, the C++ code in the third-party
> software requiring the older library is disjoint from every other bit of C++
> code in the process; there is a strict C
> calling-convention/pass-only-intrinsic-data-types "firewall" between the
> poorly supported third-party library and the rest of the C++ code in the
> process.
>
> If I have such a firewall, can I safely ignore the ominous warning from ld?

No, it doesn't help, because of ELF symbol overriding.

> Also, it's not a flat-out error, so there must be conditions where mixing
> the libraries is acceptable (or even appropriate)-- what are these
> conditions?

The linker generally permits you to shoot yourself in the foot.

You can probably make the situation work if you can write your main
program in C only, and you don't link against the C++ libraries
directly, but instead dlopen them using RTLD_DEEPBIND.

Ian

[jhfrontz]

Ian Lance Taylor-3 wrote:

jhfrontz <[e-mail]> writes:
> Also, it's not a flat-out error, so there must be conditions where mixing
> the libraries is acceptable (or even appropriate)-- what are these
> conditions?

You can probably make the situation work if you can write your main
program in C only, and you don't link against the C++ libraries
directly, but instead dlopen them using RTLD_DEEPBIND.

Thanks for the tip--but I'm pretty ignorant about the run-time loader.  Are you suggesting
1) that I create a C-only shell around the "actual" program.
2) open each of the disparate stdc++ libraries with RTLD_DEEPBIND (causing each stdc++ library to be self-consistent).
3) jump to my actual C++ program (perhaps by having it in yet-another shared object that I would dlopen after assuaging the stdc++ clash)?

Thanks,
Jeff

[Ian Lance Taylor-3]

jhfrontz <jeff.frontz@...> writes:

> Ian Lance Taylor-3 wrote:
>>
>> jhfrontz <jeff.frontz@...> writes:
>>> Also, it's not a flat-out error, so there must be conditions where mixing
>>> the libraries is acceptable (or even appropriate)-- what are these
>>> conditions?
>>
>> You can probably make the situation work if you can write your main
>> program in C only, and you don't link against the C++ libraries
>> directly, but instead dlopen them using RTLD_DEEPBIND.
>>
>
> Thanks for the tip--but I'm pretty ignorant about the run-time loader.  Are
> you suggesting
> 1) that I create a C-only shell around the "actual" program.
> 2) open each of the disparate stdc++ libraries with RTLD_DEEPBIND (causing
> each stdc++ library to be self-consistent).
> 3) jump to my actual C++ program (perhaps by having it in yet-another shared
> object that I would dlopen after assuaging the stdc++ clash)?

Well, I said a C only program, but on further thought I don't think
that is required.  It should work to write a C++ program, and to just
dlopen the shared library which uses libstdc++.so.5.  The
RTLD_DEEPBIND should protect you from the libstdc++ version collision,
I hope.  The main difficulty is that you need to use dlsym to get the
functions you want to call; you can't call them directly.

Ian

[jhfrontz]

Ian Lance Taylor-3 wrote:

Well, I said a C only program, but on further thought I don't think
that is required.  It should work to write a C++ program, and to just
dlopen the shared library which uses libstdc++.so.5.  The
RTLD_DEEPBIND should protect you from the libstdc++ version collision,
I hope.  The main difficulty is that you need to use dlsym to get the
functions you want to call; you can't call them directly.

OK, thanks yet again. This sounds very practical and possible.  It's in line with another posting that I just found (http://www.nabble.com/-Bug-libc-6610--New%3A-NSS-modules-should-be-dlopen%28%29ed-with-RTLD_DEEPBIND-tt17739013.html#a17739013 ).  The functions I need to call in the third-party library are well-defined/known and I can easily call them by using function pointers set using dlsym().

Is there a way to confirm that the RTLD_DEEPBIND is doing what I'm expecting--namely ensuring that the third-party library is using libstdc++5 while the main-line code is using libstdc++6?  Is it sufficient to observe that libstdc++5 is loaded after the libstdc++5-dependent third-party library is opened (obviously with RTLD_DEEPBIND specified)?

Maybe I'm not completely understanding the implications of RTLD_DEEPBIND and the proof is obvious-- does RTLD_DEEPBIND prevent using any already open libraries for symbol resolution?  Is the process for run-time symbol resolution to first open all referenced shared libraries, then go through and resolve the symbols (and thus RTLD_DEEPBIND says "resolve undefined symbols contained herein by looking at shared libraries listed as dependencies for this DEEPBIND-flagged object and, only if not so resolved, look at the global scope" -- that is, the "symbols in this library" referenced in the dlsym() man page entry under RTLD_DEEPBIND include the symbols brought in as a result of opening the target library's dependency libraries)?

Jeff

[Ian Lance Taylor-3]

jhfrontz <jeff.frontz@...> writes:

> Is there a way to confirm that the RTLD_DEEPBIND is doing what I'm
> expecting--namely ensuring that the third-party library is using libstdc++5
> while the main-line code is using libstdc++6?  Is it sufficient to observe
> that libstdc++5 is loaded after the libstdc++5-dependent third-party library
> is opened (obviously with RTLD_DEEPBIND specified)?

You could confirm it by creating finding a function which your library
refers to and which should be found in libstdc++.so.5.  Define a
version of that function in your main executable, and see which
version is called.

> Maybe I'm not completely understanding the implications of RTLD_DEEPBIND and
> the proof is obvious-- does RTLD_DEEPBIND prevent using any already open
> libraries for symbol resolution?

No, it changes the order in which shared libraries are searched when
resolving dependencies.

> Is the process for run-time symbol
> resolution to first open all referenced shared libraries, then go through
> and resolve the symbols (and thus RTLD_DEEPBIND says "resolve undefined
> symbols contained herein by looking at shared libraries listed as
> dependencies for this DEEPBIND-flagged object and, only if not so resolved,
> look at the global scope" -- that is, the "symbols in this library"
> referenced in the dlsym() man page entry under RTLD_DEEPBIND include the
> symbols brought in as a result of opening the target library's dependency
> libraries)?

Yes, it should the work in the latter way.

Ian

[jhfrontz]

Ian Lance Taylor-3 wrote:

You could confirm it by creating finding a function which your library
refers to and which should be found in libstdc++.so.5.  Define a
version of that function in your main executable, and see which
version is called.

Hmm, I'm not sure how to see which version is called-- is there a "version" function in libstdc++ (or some other function that is in the older one but not the newer one)?

I think I proved to myself that it works, though- I created two versions of a function called "whoAmI()", each printing out different strings. I put each of them into a library (same library name, different versions--version 3 and version 4).  I then put a call to whoAmI() from a yet-another-function, which I put into yet-another-library (different name from the first libraries)--yet-another-library is created by linking against version 3 of the first library.

From main(), I put in a call whoAmI() to see what version of output comes out.  I then use dlopen() to open yet-another-library, use dlsym() to find the address of and call yet-another-function (which also calls whoAmI()).   I link main() against version 4 of the first library. When I run the resulting program, the output from the main-line call to whoAmI() comes from version 4; the output from the call to whoAmI() via yet-another-function comes from version 3.  I watched via strace and saw that the dlopen() on yet-another-library explicitly opened version 3 of the first library.

Thanks again for your insight here.  I'll be able to sleep easier tonight ;-)

Jeff

[Ian Lance Taylor-3]

jhfrontz <jeff.frontz@...> writes:

> Ian Lance Taylor-3 wrote:
>>
>>
>> You could confirm it by creating finding a function which your library
>> refers to and which should be found in libstdc++.so.5.  Define a
>> version of that function in your main executable, and see which
>> version is called.
>>
>>
>
> Hmm, I'm not sure how to see which version is called-- is there a "version"
> function in libstdc++ (or some other function that is in the older one but
> not the newer one)?

You can see which version of the library is used by running readelf -d
and looking at the DT_NEEDED tags.  I don't know if there is a version
function in libstdc++.

Ian

[jhfrontz]

jhfrontz wrote:

Ian Lance Taylor-3 wrote:

The RTLD_DEEPBIND should protect you from the libstdc++ version collision,
I hope.  The main difficulty is that you need to use dlsym to get the
functions you want to call; you can't call them directly.

The functions I need to call in the third-party library are well-defined/known and I can easily call them by using function pointers set using dlsym().

Update: the RTLD_DEEPBIND works -- sort of.  It pulls in the right libraries, but has some strange side effect.  I call dlsym() and get an address of a desired function; I do an indirect call off of the retrieved address and see the appropriate code being executed in the dlopen()'d library (by using stepi in gdb).  However, the behavior of the function is incorrect (as determined by the values set/returned by the function); calling the function directly (by way of linking directly against the offending library) results in the desired behavior.

Is there any side-effect of doing an indirect call off of a value returned dlsym() (vs. calling the desired function directly from a run-time loaded library)?  The timing of when the library's (or libraries') _init() routine is called as well as the memory layout are obvious ones, but on the surface these both seem irrelevant for this library.

Thanks,
Jeff