How to install extensions using bjam(?)

[nitroamos]

Hello --

I'm working on a project which is mixed C++/Python, and we use Boost
functionality in our C++, and we've set up bjam to compile our project
(on OSX). Everything seems to be working quite well.

To use our libraries from python scripts right now, we have to run this first:

#!/bin/bash
trunk=$HOME/project/trunk/
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$trunk/lib
export PYTHONPATH=$trunk:$trunk/lib
python $@


which is quite inconvenient. Now obviously I can skip this step by
pasting those two export statements into my .profile, however, at some
point we're going to start distributing this software and it would be
nice if we didn't require the end user to do something like that.

I can't believe that either of these 2 steps are how it's supposed to
work... so how should I be doing it?  Ideally, I want to set something
up so that the end user only has to run 1 (one) command to build &
install. Here are some things I've thought about:

1) Having bjam edit the user's .profile to set up these paths,
possibly by calling a script I write. This doesn't seem like the right
approach to me.

2) It looks like python's distutils is supposed to be able to help,
but it seems like a waste of time to figure out how to get distutils
to compile my code when I already have bjam pretty much figured out.
First, I assume that distutils can put my libraries on an automatic
DYLD_LIBRARY_PATH path that python knows about, because if it can't
then there's no point. But assuming that's not a problem, I can't
figure out how to tell distutils about Extensions that it shouldn't
try to build...

3) bjam includes a way to install libraries into a folder I can name,
but does it have a way to set environment variables? or automatically
figure out the right places to put all the python files? If so, I
can't find it...

4) ?


thanks!

Amos.
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Nat Goodspeed-4]

Amos Anderson wrote:

> I'm working on a project which is mixed C++/Python, and we use Boost
> functionality in our C++, and we've set up bjam to compile our project
> (on OSX). Everything seems to be working quite well.
>
> 2) It looks like python's distutils is supposed to be able to help,
> but it seems like a waste of time to figure out how to get distutils
> to compile my code when I already have bjam pretty much figured out.
> First, I assume that distutils can put my libraries on an automatic
> DYLD_LIBRARY_PATH path that python knows about, because if it can't
> then there's no point. But assuming that's not a problem, I can't
> figure out how to tell distutils about Extensions that it shouldn't
> try to build...

In a previous project I've had success with py2app:
http://svn.pythonmac.org/py2app/py2app/trunk/doc/index.html

We had a number of extensions we built ourselves (not with distutils or
py2app itself) and it happily packaged them into a working app bundle.
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Anthony Foglia-2]

Amos Anderson wrote:

> I'm working on a project which is mixed C++/Python, and we use Boost
> functionality in our C++, and we've set up bjam to compile our project
> (on OSX). Everything seems to be working quite well.
>
> To use our libraries from python scripts right now, we have to run this first:
>
> #!/bin/bash
> trunk=$HOME/project/trunk/
> export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$trunk/lib
> export PYTHONPATH=$trunk:$trunk/lib
> python $@
>
>
> which is quite inconvenient. Now obviously I can skip this step by
> pasting those two export statements into my .profile, however, at some
> point we're going to start distributing this software and it would be
> nice if we didn't require the end user to do something like that.
>
> I can't believe that either of these 2 steps are how it's supposed to
> work... so how should I be doing it?  Ideally, I want to set something
> up so that the end user only has to run 1 (one) command to build &
> install. Here are some things I've thought about:

[...]

> 3) bjam includes a way to install libraries into a folder I can name,
> but does it have a way to set environment variables? or automatically
> figure out the right places to put all the python files? If so, I
> can't find it...

        Almost certainly not.  bjam would be running in a child process and
won't be able to change the environment of the parent (shell) process.
Even if it did, any changes wouldn't persist into newly launched shells.

        I don't think there you can get the "right place" to put the python
files, especially since different users will have different "right
places".  I think your best bet is to follow the usual convention of
using a default location of "/usr/lib/python<version>/site-packages",
but allow users to override it from the command line.  I'm not 100% sure
of the syntax, but something like

bjam -s prefix=/usr/local/lib/python2.5

should set the variable prefix to /usr/local/lib/python2.5 inside the
Jamfile.

Now you just need a way to get the default python version to set the
default value.  bjam's python module has a .version-countdown variable
that might help.

The boost-build mailing list can probably give you better help.

--
Anthony Foglia
Princeton Consultants
[phone] x233

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[nitroamos]

On Mon, Oct 26, 2009 at 7:08 AM, Anthony Foglia <AFoglia@...> wrote:

> Amos Anderson wrote:
>>
>> I'm working on a project which is mixed C++/Python, and we use Boost
>> functionality in our C++, and we've set up bjam to compile our project
>> (on OSX). Everything seems to be working quite well.
>>
>> To use our libraries from python scripts right now, we have to run this
>> first:
>>
>> #!/bin/bash
>> trunk=$HOME/project/trunk/
>> export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$trunk/lib
>> export PYTHONPATH=$trunk:$trunk/lib
>> python $@
>>
>>
>> which is quite inconvenient. Now obviously I can skip this step by
>> pasting those two export statements into my .profile, however, at some
>> point we're going to start distributing this software and it would be
>> nice if we didn't require the end user to do something like that.
>>
>> I can't believe that either of these 2 steps are how it's supposed to
>> work... so how should I be doing it?  Ideally, I want to set something
>> up so that the end user only has to run 1 (one) command to build &
>> install. Here are some things I've thought about:
>
> [...]
>
>> 3) bjam includes a way to install libraries into a folder I can name,
>> but does it have a way to set environment variables? or automatically
>> figure out the right places to put all the python files? If so, I
>> can't find it...
>
>        Almost certainly not.  bjam would be running in a child process and
> won't be able to change the environment of the parent (shell) process. Even
> if it did, any changes wouldn't persist into newly launched shells.

Well, what I had in mind was getting bjam to implement my solution #1.
So bjam would scan my .profile to see if it had ever modified the file
previously, and if so modify previously set variables, otherwise add
new environmental variables, etc.

>
>        I don't think there you can get the "right place" to put the python
> files, especially since different users will have different "right places".
>  I think your best bet is to follow the usual convention of using a default
> location of "/usr/lib/python<version>/site-packages", but allow users to
> override it from the command line.  I'm not 100% sure of the syntax, but
> something like
>
> bjam -s prefix=/usr/local/lib/python2.5
>
> should set the variable prefix to /usr/local/lib/python2.5 inside the
> Jamfile.
>
> Now you just need a way to get the default python version to set the default
> value.  bjam's python module has a .version-countdown variable that might
> help.

Based on your idea, I did some searching, and I found that I can do this:

python -c "from distutils.sysconfig import get_python_lib; print
get_python_lib()"
/Library/Python/2.6/site-packages

to automatically find the right location. And, it looks like:
http://developer.apple.com/mac/library/documentation/Darwin/Reference/ManPages/man1/dyld.1.html
says I can put the dylib's in one of these places:
$(HOME)/lib:/usr/local/lib:/lib:/usr/lib

and the linker will automatically find them. I guess one of those is
good enough for me, but I'd imagine that python would add a few new
places to look? I tried to put my dylib files into site-packages, but
that didn't work.

It looks like this is making some progress. It looks like at the very
least I can get bjam to put the dylib into $(HOME)/lib directly, and
edit the python scripts to add the right python paths.


Amos.


>
> The boost-build mailing list can probably give you better help.
>

I've tried a few times to sign up to that list... but it doesn't seem to work.


thanks!

Amos.

> --
> Anthony Foglia
> Princeton Consultants
> [phone] x233
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig@...
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

--
~<>~<>~<>~<>~<>~<>~<>~<>~
nitroamos@...
amosa@... (secure)

[phone] (cell)
[phone] (home)
_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig

[Anthony Foglia-2]

Amos Anderson wrote:
> Well, what I had in mind was getting bjam to implement my solution #1.
> So bjam would scan my .profile to see if it had ever modified the file
> previously, and if so modify previously set variables, otherwise add
> new environmental variables, etc.

But what if the user isn't using bash, but tcsh?  Or if they have a
.bash_profile?  Or if they source their .bashrc via their .profile and
set their PYTHONPATH in there?  There's a lot that could go wrong.

The second approach sounds much more robust and is the more conventional
approach.  Hopefully you can get it to do what you want.

--
Anthony Foglia
Princeton Consultants
[phone] x233

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig@...
http://mail.python.org/mailman/listinfo/cplusplus-sig