32/64 bit intel universal build

[Robin-62]

Hi,

I've been struggling with 32 vs 64 bit python on snow leopard. I built
an intel universal with:

./build-installer.py --sdk-path=/Developer/SDKs/MacOSX10.6.sdk/
--dep-target=10.6 --universal-archs=intel

It looks like it built correctly:

robin-mbp:bin robince$ file ./python2.6
./python2.6: Mach-O universal binary with 2 architectures
./python2.6 (for architecture i386):    Mach-O executable i386
./python2.6 (for architecture x86_64):  Mach-O 64-bit executable x86_64

But I am unable to select 32 vs 64 bit with arch:

robin-mbp:bin robince$ arch -x86_64 ./python2.6 -c "import sys; print
sys.maxint"
9223372036854775807
robin-mbp:bin robince$ arch -i386 ./python2.6 -c "import sys; print sys.maxint"
9223372036854775807

which does work with the system python:
robin-mbp:~ robince$ arch -i386 /usr/bin/python2.6 -c "import sys;
print sys.maxint"
2147483647
robin-mbp:~ robince$ arch -x86_64 /usr/bin/python2.6 -c "import sys;
print sys.maxint"
9223372036854775807

What am I missing?

Cheers

Robin
_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@...
http://mail.python.org/mailman/listinfo/pythonmac-sig

[William Kyngesburye]

Something I noticed with the Apple Python is that /usr/bin/python  
(which is it's own binary and not a symlink) *only* responds to the  
env variable VERSIONER_PYTHON_PREFER_32_BIT (I think that's the  
name).  While /usr/bin/python2.6 (which is a symlink to the executable  
in the framework) *only* responds to the arch command.

So, it looks like you configuration is using the env variable method.  
I don't compile my own Python, so I don't know if there is a  
configuration option to set this behavior.

On Oct 25, 2009, at 4:31 PM, Robin wrote:

> Hi,
>
> I've been struggling with 32 vs 64 bit python on snow leopard. I built
> an intel universal with:
>
> ./build-installer.py --sdk-path=/Developer/SDKs/MacOSX10.6.sdk/
> --dep-target=10.6 --universal-archs=intel
>
> It looks like it built correctly:
>
> robin-mbp:bin robince$ file ./python2.6
> ./python2.6: Mach-O universal binary with 2 architectures
> ./python2.6 (for architecture i386):    Mach-O executable i386
> ./python2.6 (for architecture x86_64):  Mach-O 64-bit executable  
> x86_64
>
> But I am unable to select 32 vs 64 bit with arch:
>
> robin-mbp:bin robince$ arch -x86_64 ./python2.6 -c "import sys; print
> sys.maxint"
> 9223372036854775807
> robin-mbp:bin robince$ arch -i386 ./python2.6 -c "import sys; print  
> sys.maxint"
> 9223372036854775807
>
> which does work with the system python:
> robin-mbp:~ robince$ arch -i386 /usr/bin/python2.6 -c "import sys;
> print sys.maxint"
> 2147483647
> robin-mbp:~ robince$ arch -x86_64 /usr/bin/python2.6 -c "import sys;
> print sys.maxint"
> 9223372036854775807
>
> What am I missing?
>
> Cheers
>
> Robin
> _______________________________________________
> Pythonmac-SIG maillist  -  Pythonmac-SIG@...
> http://mail.python.org/mailman/listinfo/pythonmac-sig

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

"Time is an illusion - lunchtime doubly so."

- Ford Prefect


_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@...
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Robin-62]

On Sun, Oct 25, 2009 at 10:15 PM, William Kyngesburye
<woklist@...> wrote:
> Something I noticed with the Apple Python is that /usr/bin/python (which is
> it's own binary and not a symlink) *only* responds to the env variable
> VERSIONER_PYTHON_PREFER_32_BIT (I think that's the name).  While
> /usr/bin/python2.6 (which is a symlink to the executable in the framework)
> *only* responds to the arch command.

Thanks - I see that behaviour too now I checked.

> So, it looks like you configuration is using the env variable method.  I
> don't compile my own Python, so I don't know if there is a configuration
> option to set this behavior.

I tried but I can't get the environment variable to have any effect on
the self-built python.org version.

robin-mbp:bin robince$ pwd
/Library/Frameworks/Python.framework/Versions/2.6/bin
robin-mbp:bin robince$ export VERSIONER_PYTHON_PREFER_32_BIT=yes
robin-mbp:bin robince$ ./python2.6 -c "import sys; print sys.maxint"
9223372036854775807
robin-mbp:~ robince$ /usr/bin/python -c "import sys; print sys.maxint"
2147483647
robin-mbp:bin robince$ export VERSIONER_PYTHON_PREFER_32_BIT=1
robin-mbp:bin robince$ ./python2.6 -c "import sys; print sys.maxint"
9223372036854775807

So I'm still stuck...

Cheers

Robin
_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@...
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Robin-62]

On Mon, Oct 26, 2009 at 3:17 PM, David Warde-Farley <dwf@...> wrote:

Check if there's a /Library/Frameworks/Python.framework/Versions/2.6/bin/python-64 (or /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6-64). IIRC that's how the builds produced from source work; separate executables for 32 and 64 bit mode.

Hi,

Thanks... yes thats what I was expecting initially, but there is nothing there:
jm-g26b101:bin robince$ ls /Library/Frameworks/Python.framework/Versions/2.6/bin/
2to3             pydoc            python-config    pythonw          smtpd2.6.py
idle             pydoc2.6         python2.6        pythonw2.6
idle2.6          python           python2.6-config smtpd.py

From reading the documentation I think they may only be created with the universal-archs=all target (which includes ppc and ppc64 which I dont need).

In any case - 'file' shows that it is a universal binary, just the 'arch' method of selecting architecture doesn't appear to work:
jm-g26b101:bin robince$ file /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6: Mach-O universal binary with 2 architectures
/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 (for architecture i386):        Mach-O executable i386
/Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 (for architecture x86_64):      Mach-O 64-bit executable x86_64

Perhaps I will try the 'all' variant next.

Cheers

Robin

_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@...
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Ned Deily]

In article
<2d5132a50910251527r616311b7tac4de0e04b2417fb@...>,
 Robin <robince@...> wrote:
> On Sun, Oct 25, 2009 at 10:15 PM, William Kyngesburye
> <woklist@...> wrote:
> > Something I noticed with the Apple Python is that /usr/bin/python (which is
> > it's own binary and not a symlink) *only* responds to the env variable
> > VERSIONER_PYTHON_PREFER_32_BIT (I think that's the name).  While
> > /usr/bin/python2.6 (which is a symlink to the executable in the framework)
> > *only* responds to the arch command.
[...]
> I tried but I can't get the environment variable to have any effect on
> the self-built python.org version.

That environment variable is something added by Apple to the Python they
supply with OS X 10.6.  As you've discovered, it is not part of the
official python.org distribution.

In article
<2d5132a50910260847j49c06566i6096eb6e3118f2e6@...>,
 Robin <robince@...> wrote:

> On Mon, Oct 26, 2009 at 3:17 PM, David Warde-Farley <dwf@...>wrote:
> > Check if there's a
> > /Library/Frameworks/Python.framework/Versions/2.6/bin/python-64 (or
> > /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6-64). IIRC
> > that's how the builds produced from source work; separate executables for 32
> > and 64 bit mode.
> >
>
> Hi,
>
> Thanks... yes thats what I was expecting initially, but there is nothing
> there:
> jm-g26b101:bin robince$ ls
> /Library/Frameworks/Python.framework/Versions/2.6/bin/
> 2to3             pydoc            python-config    pythonw
> smtpd2.6.py
> idle             pydoc2.6         python2.6        pythonw2.6
> idle2.6          python           python2.6-config smtpd.py
>
> >From reading the documentation I think they may only be created with the
> universal-archs=all target (which includes ppc and ppc64 which I dont need).

That is correct.  However, even if you did build with "all" (i.e. a
4-way universal build), that would still not solve your problem on 10.6.  
See below.
 

> In any case - 'file' shows that it is a universal binary, just the 'arch'
> method of selecting architecture doesn't appear to work:
> jm-g26b101:bin robince$ file
> /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6
> /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6: Mach-O
> universal binary with 2 architectures
> /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 (for
> architecture i386):        Mach-O executable i386
> /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 (for
> architecture x86_64):      Mach-O 64-bit executable x86_64

The problem here is that that "python2.6" binary in the framework bin
directory is not the interpreter itself, rather it is a launcher that
execv's to the real interpreter binary located in an app bundle within
the framework.  This is also true of all of the other variants you may
see in bin: pythonw, python32, etc.  So starting any of those
executables with "arch -i386" only forces the launcher process to run in
32-bit, the interpreter is then still execv-ed as 64-bit if available.
The reason for the indirection is explained here:

http://bugs.python.org/issue6834

Basically it is to allow certain GUI functions to work correctly when
python is invoked from the command line.  I've updated the issue to
include some workarounds until the problem is fixed; you might also want
to try Ronald's patch.

--
 Ned Deily,
 nad@...

_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@...
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Ronald Oussoren]

On 25 Oct, 2009, at 23:27, Robin wrote:

> On Sun, Oct 25, 2009 at 10:15 PM, William Kyngesburye
> <woklist@...> wrote:
>> Something I noticed with the Apple Python is that /usr/bin/python (which is
>> it's own binary and not a symlink) *only* responds to the env variable
>> VERSIONER_PYTHON_PREFER_32_BIT (I think that's the name).  While
>> /usr/bin/python2.6 (which is a symlink to the executable in the framework)
>> *only* responds to the arch command.
>
> Thanks - I see that behaviour too now I checked.
>
>> So, it looks like you configuration is using the env variable method.  I
>> don't compile my own Python, so I don't know if there is a configuration
>> option to set this behavior.
>
> I tried but I can't get the environment variable to have any effect on
> the self-built python.org version.

That's because Apple's pythonw is a completely different executable than the python.org one.

I'm working on a replacement for pythonw in 2.7 and 3.2 that will allow you to use the arch command to select an architecture. That version also works with 2.6, but I don't think I'll be able to sneak it into 2.6.5 because the new executable is not 100% backward compatible with the current one.

Ronald

_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@...
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Robin-62]

On Wed, Oct 28, 2009 at 8:37 PM, Ronald Oussoren <ronaldoussoren@...> wrote:

That's because Apple's pythonw is a completely different executable than the python.org one.

I'm working on a replacement for pythonw in 2.7 and 3.2 that will allow you to use the arch command to select an architecture. That version also works with 2.6, but I don't think I'll be able to sneak it into 2.6.5 because the new executable is not 100% backward compatible with the current one.

Thanks very much - I dropped the replacement pythonw.c from the ticket into Mac/Tools and rebuilt the Framework and now everything works as expected (I had to change the hardcoded python path to 2.6).

Is this likely to work with virtualenv (it seems to but haven't tried too much yet).

Cheers

Robin
 

_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@...
http://mail.python.org/mailman/listinfo/pythonmac-sig

[Ronald Oussoren]

On 29 Oct, 2009, at 13:13, Robin wrote:

On Wed, Oct 28, 2009 at 8:37 PM, Ronald Oussoren <ronaldoussoren@...> wrote:

That's because Apple's pythonw is a completely different executable than the python.org one.

I'm working on a replacement for pythonw in 2.7 and 3.2 that will allow you to use the arch command to select an architecture. That version also works with 2.6, but I don't think I'll be able to sneak it into 2.6.5 because the new executable is not 100% backward compatible with the current one.

Thanks very much - I dropped the replacement pythonw.c from the ticket into Mac/Tools and rebuilt the Framework and now everything works as expected (I had to change the hardcoded python path to 2.6).

Is this likely to work with virtualenv (it seems to but haven't tried too much yet).

This should work as well as the current pythonw with virtualenv. The last time I checked, which is quite a while back, virtualenv didn't support the pythonw executable at all but always copied the "real" interpreter.

My new pythonw contains hooks that should make it possible to have virtualenv do the right thing w.r.t. executables. Writing a patch that implements this is one thing I want to do before merging the new pythonw.

Ronald

Cheers

Robin
 

_______________________________________________
Pythonmac-SIG maillist  -  Pythonmac-SIG@...
http://mail.python.org/mailman/listinfo/pythonmac-sig