Re: Patch for building Boost 1.40 on Mac OS X Snow Leopard

[Vladimir Prus]

Boris Dušek wrote:

> Hi, I wrote this reply some time ago, but now it's also relevant:

[snip comprehensive explanation]

> So e.g. if you use g++-4.0 or g++-4.2 and compile for 10.5 SDK, you can get
> 4-way binaries running on 10.5 and later. If you use g++-4.0 and compile for
> 10.4 SDK, you can get 4-way binaries running on 10.4 or later, but be vary
> that on Tiger, e.g. system's libz is 4-way, but system's libbz2 is only
> 2-way (32-bit), so you have to supply your own 4-way bz2 to iostreams (only
> a few libraries on Tiger were 4-way). And if you do that with your own 4-way
> static build of libbz2, you run into boost.build's library path issue that I
> won't go into here (this post is already way longer than I intendet it to
> be).
>
> Sorry for the huge post, but I hope this attempt makes things a little bit
> more understandable w.r.t. building for ppc64 (and even others) _on_ Snow
> Leopard _for_ various target versions of Mac OS X.

Boris,

thanks for writing this up, it was very helpful in understanding the situation.
I've arrived at the attached patch to address this. The desired effect is that
with gcc 4.2, when targeting 10.6:

- if architecture=combined address-model=32_64 is specified, we build 3-way fat
  binary, skipping ppc64.
- if architecture=power address-model=32_64 is specified, or
  architecture=combined address-model=64, we emit an error.

Can you give this a try? Not having OSX, I could only verify that darwin.jam
is not a brick after applying this patch -- I did not check the actual logic.

- Volodya


[osx-ppc64.diff]

---

Index: build/toolset.jam
===================================================================
--- build/toolset.jam (revision 57269)
+++ build/toolset.jam (working copy)
@@ -449,7 +449,7 @@
 }
 
 
-rule inherit-rules ( toolset : base )
+rule inherit-rules ( toolset : base : localize ? )
 {
     # It appears that "action" creates a local rule.
     local base-generators = [ generators.generators-for-toolset $(base) ] ;
@@ -459,8 +459,8 @@
         rules += [ MATCH "[^.]*\.(.*)" : [ $(g).rule-name ] ] ;
     }
     rules = [ sequence.unique $(rules) ] ;
-    IMPORT $(base) : $(rules) : $(toolset) : $(rules) ;
-    IMPORT $(base) : $(rules) : : $(toolset).$(rules) ;
+    IMPORT $(base) : $(rules) : $(toolset) : $(rules) : $(localize) ;
+    IMPORT $(toolset) : $(rules) : : $(toolset).$(rules) ;
 }
 
 
Index: tools/darwin.jam
===================================================================
--- tools/darwin.jam (revision 57269)
+++ tools/darwin.jam (working copy)
@@ -16,6 +16,9 @@
 import common ;
 import generators ;
 import path : basename ;
+import version ;
+import property-set ;
+import regex ;
 
 ## Use a framework.
 feature framework : : free ;
@@ -60,7 +63,7 @@
 
 type.set-generated-target-suffix PCH : <toolset>darwin : gch ;
 
-toolset.inherit-rules darwin : gcc ;
+toolset.inherit-rules darwin : gcc : localize ;
 toolset.inherit-flags darwin : gcc    
   :  <runtime-link>static
      <architecture>arm/<address-model>32
@@ -121,6 +124,8 @@
         version ?= $(real-version) ;
     }
     
+    .real-version.$(version) = $(real-version) ;
+    
     # - Define the condition for this toolset instance.
     local condition =
         [ common.check-init-parameters darwin $(requirement) : version $(version) ] ;
@@ -265,10 +270,102 @@
 generators.register-c-compiler darwin.compile.m : OBJECTIVE_C : OBJ : <toolset>darwin ;
 generators.register-c-compiler darwin.compile.mm : OBJECTIVE_CPP : OBJ : <toolset>darwin ;
 
-rule compile.m
+rule setup-address-model ( targets * : sources * : properties * )
 {
+    local ps = [ property-set.create $(properties) ] ;
+    local arch = [ $(ps).get <architecture> ] ;
+    local address-model = [ $(ps).get <address-model> ] ;
+    local osx-version = [ $(ps).get <macosx-version> ] ;
+    local gcc-version = [ $(ps).get <toolset-darwin:version> ] ;
+    gcc-version = $(.real-version.$(gcc-version)) ;
+    local options ;
+    
+    local support-ppc64 = 1 ;
+    if $(osx-version) && ! [ version.less [ regex.split $(osx-version) . ] : 10 6 ]
+    {
+        # When targeting 10.6:
+        # - gcc 4.2 will give a compiler errir if ppc64 compilation is requested
+        # - gcc 4.0 will compile fine, somehow, but then fail at link time
+        support-ppc64 = ;
+    }
+          
+    switch $(arch)
+    {
+        case combined :
+        {
+            if $(address-model) = 32_64 {
+                if $(support-ppc64) {
+                    options = -arch i386 -arch ppc -arch x86_64 -arch ppc64 ;                    
+                } else {
+                    # Build 3-way binary
+                    options = -arch i386 -arch ppc -arch x86_64 ;
+                }                                
+            } else if $(address-model) = 64 {
+                if $(support-ppc64) {
+                    options = -arch x86_64 -arch ppc64 ;
+                } else {
+                    errors.user-error "64-bit PPC compilation is not supported when targeting OSX 10.6 or later" ;
+                }                
+            } else {
+                options = -arch i386 -arch ppc ;
+            }
+        }
+        
+        case x86 :
+        {
+            if $(address-model) = 32_64 {
+                options = -arch i386 -arch x86_64 ;
+            } else if $(address-model) = 64 {
+                options = -arch x86_64 ;
+            } else {
+                options = -arch i386 ;
+            }
+        }        
+        
+        case power :
+        {
+            if ! $(support-ppc64)
+              && (  $(address-model) = 32_64 || $(address-model) = 64 )
+            {
+                errors.user-error "64-bit PPC compilation is not supported when targeting OSX 10.6 or later" ;
+            }
+            
+            if $(address-model) = 32_64 {
+                options = -arch ppc -arch ppc64 ;
+            } else if $(address-model) = 64 {
+                options = -arch ppc64 ;
+            } else {
+                options = -arch ppc ;
+            }
+        }
+        
+        case arm :
+        {
+            options = -arch armv6 ;
+        }        
+    }
+    
+    if $(options)
+    {
+        OPTIONS on $(targets) += $(options) ;
+    }            
+}
+
+rule setup-threading ( targets * : sources * : properties * )
+{
+    gcc.setup-threading $(targets) : $(sources) : $(properties) ;
+}
+
+rule setup-fpic ( targets * : sources * : properties * )
+{
+    gcc.setup-fpic $(targets) : $(sources) : $(properties) ;
+}
+
+rule compile.m ( targets * : sources * : properties * )
+{
     LANG on $(<) = "-x objective-c" ;
     gcc.setup-fpic $(targets) : $(sources) : $(properties) ;
+    setup-arch-addr-flags $(targets) : $(sources) : $(properties) ;
 }
 
 actions compile.m
@@ -280,6 +377,7 @@
 {
     LANG on $(<) = "-x objective-c++" ;
     gcc.setup-fpic $(targets) : $(sources) : $(properties) ;
+    setup-arch-addr-flags $(targets) : $(sources) : $(properties) ;
 }
 
 actions compile.mm
@@ -287,37 +385,6 @@
     "$(CONFIG_COMMAND)" $(LANG) $(OPTIONS) $(USER_OPTIONS) -D$(DEFINES) -I"$(INCLUDES)" -c -o "$(<)" "$(>)"
 }
 
-# Add option selection for combined and specific architecture combinations.
-
-local rule arch-addr-flags ( toolset variable
-    : architecture : address-model + : values + : default ? )
-{
-    if $(default)
-    {
-        flags $(toolset) $(variable)
-            <architecture>$(architecture)/<address-model>
-            : $(values) ;
-    }
-    flags $(toolset) $(variable)
-        <architecture>/<address-model>$(address-model)
-        <architecture>$(architecture)/<address-model>$(address-model)
-        : $(values) ;
-}
-
-arch-addr-flags darwin OPTIONS : combined : 32 : -arch i386 -arch ppc : default ;
-arch-addr-flags darwin OPTIONS : combined : 64 : -arch x86_64 -arch ppc64 ;
-arch-addr-flags darwin OPTIONS : combined : 32_64 : -arch i386 -arch ppc -arch x86_64 -arch ppc64 ;
-
-arch-addr-flags darwin OPTIONS : x86 : 32 : -arch i386 : default ;
-arch-addr-flags darwin OPTIONS : x86 : 64 : -arch x86_64 ;
-arch-addr-flags darwin OPTIONS : x86 : 32_64 : -arch i386 -arch x86_64 ;
-
-arch-addr-flags darwin OPTIONS : power : 32 : -arch ppc : default ;
-arch-addr-flags darwin OPTIONS : power : 64 : -arch ppc64 ;
-arch-addr-flags darwin OPTIONS : power : 32_64 : -arch ppc -arch ppc64 ;
-
-arch-addr-flags darwin OPTIONS : arm : 32 : -arch armv6 : default ;
-
 # Set the max header padding to allow renaming of libs for installation.
 flags darwin.link.dll OPTIONS : -headerpad_max_install_names ;
 

---

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

[g-h-d]

Just prior to this post I have posted a  linking problem pertaining to
1.40 in conjunction with Snow.

Is the problem of the current post only for 'fat binaries' or has
someone else also found problems with Architecture = 64-bit only?

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

[g-h-d]

er wrote:

> Just prior to this post I have posted a  linking problem pertaining to
> 1.40 in conjunction with Snow.
>
> Is the problem of the current post only for 'fat binaries' or has
> someone else also found problems with Architecture = 64-bit only?
>
> _______________________________________________
> Unsubscribe & other changes:
> http://lists.boost.org/mailman/listinfo.cgi/boost-build
>

PS : I also wonder if my directory structure is ok:

include/boost
lib/
    libboost_serialization.a
    libboost_serialization.dylib
    same with wserialization ...

which is not consistent with a previous release where I had

bin.v2/libs/serialization/build/darwin-4.2.1/release/link-static/threading-multi
  containing some *.o files in addition to the *.a files above.

Would someone kindly confirm if this is fine or not?


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

[Vladimir Prus]

On Thursday 05 November 2009 er wrote:

> Just prior to this post I have posted a  linking problem pertaining to
> 1.40 in conjunction with Snow.
>
> Is the problem of the current post only for 'fat binaries' or has
> someone else also found problems with Architecture = 64-bit only?

This patch is only related to 64 builds for the PPC architecture.

- Volodya
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

[Vladimir Prus]

On Thursday 05 November 2009 er wrote:

> er wrote:
> > Just prior to this post I have posted a  linking problem pertaining to
> > 1.40 in conjunction with Snow.
> >
> > Is the problem of the current post only for 'fat binaries' or has
> > someone else also found problems with Architecture = 64-bit only?
> >
> > _______________________________________________
> > Unsubscribe & other changes:
> > http://lists.boost.org/mailman/listinfo.cgi/boost-build
> >
>
> PS : I also wonder if my directory structure is ok:
>
> include/boost
> lib/
>     libboost_serialization.a
>     libboost_serialization.dylib
>     same with wserialization ...
>
> which is not consistent with a previous release where I had
>
> bin.v2/libs/serialization/build/darwin-4.2.1/release/link-static/threading-multi
>   containing some *.o files in addition to the *.a files above.
>
> Would someone kindly confirm if this is fine or not?

I suspect for the previous release, you did not request the 'stage' target --
which the getting started guide mentions. 1.40 defaults to the 'stage' target
automatically. I don't think there's any bug here.

- Volodya
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

[Boris Dušek-2]

Hi Volodya,

2009/11/8 Vladimir Prus <ghost@...>

> > Explicitly specifying macosx-version=10.6 should have the desired effect.
> >
>
> Looks like it does not ("-arch ppc64" highlighted with asterisks):
>
> bjam toolset=darwin threading=multi link=shared runtime-link=shared
> --with-system architecture=combined address-model=32_64 macosx-version=10.6
> macosx-version-min=10.6 stage --stagedir=. variant=release -d+2

Aha. I've misquoted '.' when splitting version number.

Works now.

 

> It seems that for both macosx-version and macosx-version-min, it defaults to
> the OS X version of the build machine. I will try to find more info on this.
> To find out the OS X version at runtime, it's possible to issue this
> command:
>
> $ /usr/bin/sw_vers -productVersion
> 10.6.1

I have modified patch to invoke this, and use the result if no version is
explicitly specified.

Works!

Oh, one more thing (and a funny one): errors on issuing an error message:

$ bjam toolset=darwin threading=multi link=shared runtime-link=shared --with-system architecture=combined address-model=64 stage --stagedir=. variant=release -d+2 && echo "Report: `lipo -info lib/*`"

/Users/boris/Downloads/boost_1_40_0/tools/build/v2/tools/darwin.jam:321: in setup-address-model

rule errors.user-error unknown in module darwin.

(this is the case when it detected 64-bit PPC compilation targetting 10.6 and wants to issue errors.user-error "64-bit PPC compilation is not supported when targeting OSX 10.6 or later")

Thanks for tirelessly supporting a platform you can't test on and which I use!

Boris

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

[Vladimir Prus]

On Sunday 08 November 2009 Boris Dušek wrote:

> Hi Volodya,
>
> 2009/11/8 Vladimir Prus <ghost@...>
>
> > > > Explicitly specifying macosx-version=10.6 should have the desired
> > effect.
> > > >
> > >
> > > Looks like it does not ("-arch ppc64" highlighted with asterisks):
> > >
> > > bjam toolset=darwin threading=multi link=shared runtime-link=shared
> > > --with-system architecture=combined address-model=32_64
> > macosx-version=10.6
> > > macosx-version-min=10.6 stage --stagedir=. variant=release -d+2
> >
> > Aha. I've misquoted '.' when splitting version number.
> >
>
> Works now.
>
>
> > > It seems that for both macosx-version and macosx-version-min, it defaults
> > to
> > > the OS X version of the build machine. I will try to find more info on
> > this.
> > > To find out the OS X version at runtime, it's possible to issue this
> > > command:
> > >
> > > $ /usr/bin/sw_vers -productVersion
> > > 10.6.1
> >
> > I have modified patch to invoke this, and use the result if no version is
> > explicitly specified.
>
>
> Works!
>
> Oh, one more thing (and a funny one): errors on issuing an error message:
>
> $ bjam toolset=darwin threading=multi link=shared runtime-link=shared
> --with-system architecture=combined address-model=64 stage --stagedir=.
> variant=release -d+2 && echo "Report: `lipo -info lib/*`"
> /Users/boris/Downloads/boost_1_40_0/tools/build/v2/tools/darwin.jam:321: in
> setup-address-model
> rule errors.user-error unknown in module darwin.
>
> (this is the case when it detected 64-bit PPC compilation targetting 10.6
> and wants to issue errors.user-error "64-bit PPC compilation is not
> supported when targeting OSX 10.6 or later")

Well, at least it produces *some* diagnostics. I've added missing import
and have now checked in in trunk. I'll merge to release branch unless
something breaks. I attach the final patch for convenience.

> Thanks for tirelessly supporting a platform you can't test on and which I
> use!

Thanks for testing my random attempts!

- Volodya


[osx-ppc64-try4.diff]

---

Index: darwin.jam
===================================================================
--- darwin.jam (revision 57269)
+++ darwin.jam (working copy)
@@ -16,6 +16,10 @@
 import common ;
 import generators ;
 import path : basename ;
+import version ;
+import property-set ;
+import regex ;
+import errors ;
 
 ## Use a framework.
 feature framework : : free ;
@@ -60,7 +64,7 @@
 
 type.set-generated-target-suffix PCH : <toolset>darwin : gch ;
 
-toolset.inherit-rules darwin : gcc ;
+toolset.inherit-rules darwin : gcc : localize ;
 toolset.inherit-flags darwin : gcc    
   :  <runtime-link>static
      <architecture>arm/<address-model>32
@@ -82,6 +86,17 @@
 #
 rule init ( version ? : command * : options * : requirement * )
 {
+    # First time around, figure what is host OSX version
+    if ! $(.host-osx-version)  
+    {
+        .host-osx-version = [ MATCH "^([0-9.]+)"
+          : [ SHELL "/usr/bin/sw_vers -productVersion" ] ] ;
+        if $(.debug-configuration)
+        {
+            ECHO notice: OSX version on this machine is $(.host-osx-version) ;
+        }
+    }
+                      
     # - The root directory of the tool install.
     local root = [ feature.get-values <root> : $(options) ] ;
     
@@ -121,6 +136,8 @@
         version ?= $(real-version) ;
     }
     
+    .real-version.$(version) = $(real-version) ;
+    
     # - Define the condition for this toolset instance.
     local condition =
         [ common.check-init-parameters darwin $(requirement) : version $(version) ] ;
@@ -265,10 +282,105 @@
 generators.register-c-compiler darwin.compile.m : OBJECTIVE_C : OBJ : <toolset>darwin ;
 generators.register-c-compiler darwin.compile.mm : OBJECTIVE_CPP : OBJ : <toolset>darwin ;
 
-rule compile.m
+rule setup-address-model ( targets * : sources * : properties * )
 {
+    local ps = [ property-set.create $(properties) ] ;
+    local arch = [ $(ps).get <architecture> ] ;
+    local address-model = [ $(ps).get <address-model> ] ;
+    local osx-version = [ $(ps).get <macosx-version> ] ;
+    local gcc-version = [ $(ps).get <toolset-darwin:version> ] ;
+    gcc-version = $(.real-version.$(gcc-version)) ;
+    local options ;
+    
+    local support-ppc64 = 1 ;
+    
+    osx-version ?= $(.host-osx-version) ;
+
+    if $(osx-version) && ! [ version.version-less [ regex.split $(osx-version) \\. ] : 10 6 ]
+    {
+        # When targeting 10.6:
+        # - gcc 4.2 will give a compiler errir if ppc64 compilation is requested
+        # - gcc 4.0 will compile fine, somehow, but then fail at link time
+        support-ppc64 = ;
+    }
+          
+    switch $(arch)
+    {
+        case combined :
+        {
+            if $(address-model) = 32_64 {
+                if $(support-ppc64) {
+                    options = -arch i386 -arch ppc -arch x86_64 -arch ppc64 ;                    
+                } else {
+                    # Build 3-way binary
+                    options = -arch i386 -arch ppc -arch x86_64 ;
+                }                                
+            } else if $(address-model) = 64 {
+                if $(support-ppc64) {
+                    options = -arch x86_64 -arch ppc64 ;
+                } else {
+                    errors.user-error "64-bit PPC compilation is not supported when targeting OSX 10.6 or later" ;
+                }                
+            } else {
+                options = -arch i386 -arch ppc ;
+            }
+        }
+        
+        case x86 :
+        {
+            if $(address-model) = 32_64 {
+                options = -arch i386 -arch x86_64 ;
+            } else if $(address-model) = 64 {
+                options = -arch x86_64 ;
+            } else {
+                options = -arch i386 ;
+            }
+        }        
+        
+        case power :
+        {
+            if ! $(support-ppc64)
+              && (  $(address-model) = 32_64 || $(address-model) = 64 )
+            {
+                errors.user-error "64-bit PPC compilation is not supported when targeting OSX 10.6 or later" ;
+            }
+            
+            if $(address-model) = 32_64 {
+                options = -arch ppc -arch ppc64 ;
+            } else if $(address-model) = 64 {
+                options = -arch ppc64 ;
+            } else {
+                options = -arch ppc ;
+            }
+        }
+        
+        case arm :
+        {
+            options = -arch armv6 ;
+        }        
+    }
+    
+    if $(options)
+    {
+        OPTIONS on $(targets) += $(options) ;
+    }            
+}
+
+rule setup-threading ( targets * : sources * : properties * )
+{
+    gcc.setup-threading $(targets) : $(sources) : $(properties) ;
+}
+
+rule setup-fpic ( targets * : sources * : properties * )
+{
+    gcc.setup-fpic $(targets) : $(sources) : $(properties) ;
+}
+
+rule compile.m ( targets * : sources * : properties * )
+{
     LANG on $(<) = "-x objective-c" ;
     gcc.setup-fpic $(targets) : $(sources) : $(properties) ;
+    setup-arch-addr-flags $(targets) : $(sources) : $(properties) ;
 }
 
 actions compile.m
@@ -280,6 +392,7 @@
 {
     LANG on $(<) = "-x objective-c++" ;
     gcc.setup-fpic $(targets) : $(sources) : $(properties) ;
+    setup-arch-addr-flags $(targets) : $(sources) : $(properties) ;
 }
 
 actions compile.mm
@@ -287,37 +400,6 @@
     "$(CONFIG_COMMAND)" $(LANG) $(OPTIONS) $(USER_OPTIONS) -D$(DEFINES) -I"$(INCLUDES)" -c -o "$(<)" "$(>)"
 }
 
-# Add option selection for combined and specific architecture combinations.
-
-local rule arch-addr-flags ( toolset variable
-    : architecture : address-model + : values + : default ? )
-{
-    if $(default)
-    {
-        flags $(toolset) $(variable)
-            <architecture>$(architecture)/<address-model>
-            : $(values) ;
-    }
-    flags $(toolset) $(variable)
-        <architecture>/<address-model>$(address-model)
-        <architecture>$(architecture)/<address-model>$(address-model)
-        : $(values) ;
-}
-
-arch-addr-flags darwin OPTIONS : combined : 32 : -arch i386 -arch ppc : default ;
-arch-addr-flags darwin OPTIONS : combined : 64 : -arch x86_64 -arch ppc64 ;
-arch-addr-flags darwin OPTIONS : combined : 32_64 : -arch i386 -arch ppc -arch x86_64 -arch ppc64 ;
-
-arch-addr-flags darwin OPTIONS : x86 : 32 : -arch i386 : default ;
-arch-addr-flags darwin OPTIONS : x86 : 64 : -arch x86_64 ;
-arch-addr-flags darwin OPTIONS : x86 : 32_64 : -arch i386 -arch x86_64 ;
-
-arch-addr-flags darwin OPTIONS : power : 32 : -arch ppc : default ;
-arch-addr-flags darwin OPTIONS : power : 64 : -arch ppc64 ;
-arch-addr-flags darwin OPTIONS : power : 32_64 : -arch ppc -arch ppc64 ;
-
-arch-addr-flags darwin OPTIONS : arm : 32 : -arch armv6 : default ;
-
 # Set the max header padding to allow renaming of libs for installation.
 flags darwin.link.dll OPTIONS : -headerpad_max_install_names ;
 
@@ -360,8 +442,9 @@
     }
 }
 
-rule link
+rule link ( targets * : sources * : properties * )
 {
+    setup-address-model $(targets) : $(sources) : $(properties) ;
     prepare-framework-path $(<) ;
 }
 
@@ -375,8 +458,9 @@
     $(NEED_STRIP)"$(.STRIP)" $(NEED_STRIP)-S $(NEED_STRIP)-x $(NEED_STRIP)"$(<)"
 }
 
-rule link.dll
+rule link.dll ( targets * : sources * : properties * )
 {
+    setup-address-model $(targets) : $(sources) : $(properties) ;
     prepare-framework-path $(<) ;
 }
 

---

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build

[Boris Dušek-2]

2009/11/8 Vladimir Prus <ghost@...>

> and wants to issue errors.user-error "64-bit PPC compilation is not

> supported when targeting OSX 10.6 or later")

Well, at least it produces *some* diagnostics. I've added missing import
and have now checked in in trunk. I'll merge to release branch unless
something breaks. I attach the final patch for convenience.

Now produces the error message accordingly.

I am attaching the patch again, the only difference is that it also contains the required changes to build/toolset.jam that were present in your first patch but not the others, so that there is one standalone (hopefully final) patch against 1.40.0.

Boris

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost-build