shared libraries

[Volker Braun]

Hi Clint,

I'm maintaining ATLAS for Sage and therefore really interested in
working shared libraries. My first suggestion would be to use libtool.
Generally speaking, it is a lot more difficult to build shared
libraries on all kinds of obscure systems than just static libraries.

Then use libtool versioning (three integers, see
http://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html),
and not the underlying OS versioning which might not support
everything you expect on all systems.

Finally, keep it simple to make it easy on you to maintain. I think
your suggestion #2 seems to be the easiest, just put everything into
one library.

> SHARED DESIGN2:
> ===============
> One library for serial and one for parallel:
>    libatlas_serial.so
>    libatlas_parallel.so
> ** Export list of cblas_* catlas_*, clapack_*, f77blas, f77lapack
>
> DESIGN2:
> + Simple build and link
> + Export list is minimized to API and f77lapack
> - partial builds (C-only, no netlib, etc) will satisfy app requiring full

I don't really see that as a drawback, every library on the planet
that allows one to reduce the feature set via some configure option
has this property. If you compile a system-wide library you are
supposed to include all default features, if you build your own local
library then you can do what you want.

> - No real way to version things

Just follow libtool versioning rules
(http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html),
depending on whether the new library is a drop-in replacement, a
drop-in replacement that adds new features, or requires recompile.

> - Will need testers to make sure new exporting rules are correct

Everything in software must be tested ;-)

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Math-atlas-devel mailing list
Math-atlas-devel@...
https://lists.sourceforge.net/lists/listinfo/math-atlas-devel

[whaley-8]

Volker,

Unfortunately, this feedback is too late for the stable.  After the
underwhelming response to this original question, and no volunteers to
help, I went with the "keep it like it is presently" option.

However, I would still like to have a long-term plan, so I am very interested
in continuing the discussion with people who use shared libs!

>I'm maintaining ATLAS for Sage and therefore really interested in
>working shared libraries. My first suggestion would be to use libtool.

Ahhh, that famous sentence drips from your lips so easily :)
Many have advised me to use libtool, as if that wouldn't touch the entire
build process, including a large number of generated makefiles.  My
understanding is that I have to change all the makefiles so that libtool
is used for building static and shared libs, and I am completely unwilling
to do this.  I do not even want to make libtool a required dependency, and
I believe I would rather die than touch all the makefiles in ATLAS.  I have
just spent more than an entire semester without doing any interesting work
(trying for stabilization), and I would rather quit working on ATLAS than
do another boring, rote, but hugely error-prone code monkey job at the
present time.

But, maybe you are know of a way to use libtool as I specified,
only as a separate "build dynamic libs" step after the standard static build?
If so, do tell!

>Generally speaking, it is a lot more difficult to build shared
>libraries on all kinds of obscure systems than just static libraries.

Especially if you don't yourself use shared libs directly.

>Then use libtool versioning (three integers, see
>http://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html),
>and not the underlying OS versioning which might not support
>everything you expect on all systems.

I don't understand this description at all, would have to spend quite a bit
of time trying to.  When I look at this problem again (if I do so), maybe
you can explain in more detail how this is accomplished and what it would
mean for ATLAS?

>Finally, keep it simple to make it easy on you to maintain. I think
>your suggestion #2 seems to be the easiest, just put everything into
>one library.
>
>> SHARED DESIGN2:
>> ===============
>> One library for serial and one for parallel:
>>    libatlas_serial.so
>>    libatlas_parallel.so
>> ** Export list of cblas_* catlas_*, clapack_*, f77blas, f77lapack
>>
>> DESIGN2:
>> + Simple build and link
>> + Export list is minimized to API and f77lapack
>> - partial builds (C-only, no netlib, etc) will satisfy app requiring full
>
>I don't really see that as a drawback, every library on the planet
>that allows one to reduce the feature set via some configure option
>has this property. If you compile a system-wide library you are
>supposed to include all default features, if you build your own local
>library then you can do what you want.

Interesting!  I was not aware that not specifying the reduced API was common,
but it would make life easier . . .

>> - No real way to version things
>
>Just follow libtool versioning rules
>(http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html),
>depending on whether the new library is a drop-in replacement, a
>drop-in replacement that adds new features, or requires recompile.

Would need need to understand.

>> - Will need testers to make sure new exporting rules are correct
>
>Everything in software must be tested ;-)

Yes, but software I understand I can test.  Since I have never explicitly
used shared libs for any ATLAS work, I cannot test dynamic stuff.  I got
ATLAS working to its present state because someone who used dynamic libs
wrote me a simple tester, and explained some rules.

Until I get some folks who use ATLAS dynamically volunteering to help,
I'm likely to leave ATLAS's shared library building in a fairly primitive
state.  Seems most of those who use shared libs just write their own
frameworks that build it (usually crippling a lot of ATLAS features along
the way, but I can't stop them), and until someone works with me to improve
this, I guess that'll be the way we roll . . .

Cheers,
Clint

**************************************************************************
** R. Clint Whaley, PhD ** Assist Prof, UTSA ** www.cs.utsa.edu/~whaley **
**************************************************************************

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Math-atlas-devel mailing list
Math-atlas-devel@...
https://lists.sourceforge.net/lists/listinfo/math-atlas-devel

[Volker Braun]

You don't need to use the whole auto* machinery to take advantage of
libtool, see for example libjpeg. In fact you don't have to use
libtool to compile. Of course it is then up to you to ensure that you
pass -fPIC or any other required compiler switch on the target
platform to use the object code in a shared library.

For example, right now you can already do your own ATLAS compile
(having used configure -Fa alg -fPIC) and then

cd lib
libtool --mode=link gcc -o libsatlas.la liblapack.a libf77blas.a
libcblas.a libatlas.a -rpath $INSTALLDIR -version-info 0:0:0
libtool --mode=install /usr/bin/install -c libsatlas.la $INSTALLDIR/libsatlas.la
libtool --finish $INSTALLDIR

If you compile with "libtool --mode=compile gcc ..." instead of "gcc
..." then the -fPIC wil be added automatically to the compiler command
line, but you don't have to do this.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Math-atlas-devel mailing list
Math-atlas-devel@...
https://lists.sourceforge.net/lists/listinfo/math-atlas-devel

[Volker Braun]

Correction: one needs to specify the .o files, not the .a archives on
the libtool --mode=link command line.

------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Math-atlas-devel mailing list
Math-atlas-devel@...
https://lists.sourceforge.net/lists/listinfo/math-atlas-devel