New boost packaging suggestion for windows

[Julian Bangert-2]

Hello,

yesterday I tried to download the boostpro binaries for windows and I
had to download over a gigabyte to get all configurations for VS 2008.
Sadly the version also was outdated and in general not that great ( e.g.
Installer quite bad to handle and you have to register to download ) .
So I decided to build from source and package it for a few friends who
also needed these binaries. I managed to compress boost_1_40 ( no source
except headers, no python AFAIK , but everything else you get with a
standard build ) from over 900MB down to 37MB by using 7z.
I uploaded it here https://obliviononline.com/pub/boost/boost_1_40_0.7z .
Should boost use this form of releasing binaries in the future for
windows ? Alternatively, you could endorse this form, like boostpro is
endorsed right now and I could host it on google code, etc .
An easy to set up ( just unzip this file ) binary distribution would
make compiling boost-dependent software much easier.
I would be willing to distribute future releases compatible with VS 2005
and 2008 in this form.

Yours, Julian Bangert


__________ Information from ESET NOD32 Antivirus, version of virus signature database 4551 (20091028) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[David Abrahams-3]

on Wed Oct 28 2009, Julian Bangert <jbangert-AT-acm.org> wrote:

> Hello,
>
> yesterday I tried to download the boostpro binaries for windows and I had to download
> over a gigabyte to get all configurations for VS 2008.
> Sadly the version also was outdated

We've finally released a 1.40 installer today.  I don't expect to see a delay
like this one again; sorry for the wait.

> and in general not that great ( e.g. Installer
> quite bad to handle

Specifically?

> and you have to register to download ) .

That's not a huge problem is it?

> So I decided to build from source and package it for a few friends who also needed
> these binaries. I managed to compress boost_1_40 ( no source except headers, no python
> AFAIK , but everything else you get with a standard build ) from over 900MB down to
> 37MB by using 7z.

We ship .zip files over the wire.  Everything that the installer can
install, including all variants of all libraries for 3 different
compilers, tools, and source, for 1.40.0 comes to 360M.  I know 7zip
does better, but it's not an order of magnitude.

> I uploaded it here https://obliviononline.com/pub/boost/boost_1_40_0.7z .
> Should boost use this form of releasing binaries in the future for
> windows ?

That's not a bad idea.  Of course, if we could integrate 7(un-)zip into
our installer, that too would keep the download size way down.

> Alternatively, you could endorse this form, like boostpro is endorsed
> right now and I could host it on google code, etc .
> An easy to set up ( just unzip this file ) binary distribution would make compiling
> boost-dependent software much easier.
> I would be willing to distribute future releases compatible with VS 2005 and 2008 in
> this form.

I'm willing to reference anything in the Boost getting started guide
that will be provided *reliably*.

--
Dave Abrahams           Meet me at BoostCon: http://www.boostcon.com
BoostPro Computing
http://www.boostpro.com

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Olaf van der Spek-2]

On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@...> wrote:
>> and you have to register to download ) .
>
> That's not a huge problem is it?

It's still a problem.

Olaf
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[David Abrahams-3]

on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:

> On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@...> wrote:
>>> and you have to register to download ) .
>>
>> That's not a huge problem is it?
>
> It's still a problem.

In what way?

--
Dave Abrahams           Meet me at BoostCon: http://www.boostcon.com
BoostPro Computing
http://www.boostpro.com

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Mathias Gaunard-2]

David Abrahams a écrit :
> on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
>
>> On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@...> wrote:
>>>> and you have to register to download ) .
>>> That's not a huge problem is it?
>> It's still a problem.
>
> In what way?

As a rule, the need to register to access something is always an annoyance.

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Olaf van der Spek-2]

On Wed, Oct 28, 2009 at 7:31 PM, David Abrahams <dave@...> wrote:

>
> on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
>
>> On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@...> wrote:
>>>> and you have to register to download ) .
>>>
>>> That's not a huge problem is it?
>>
>> It's still a problem.
>
> In what way?

Unnecessary steps that take time and require an email address.

Olaf
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[David Abrahams-3]

on Wed Oct 28 2009, Mathias Gaunard <mathias.gaunard-AT-ens-lyon.org> wrote:

> David Abrahams a écrit :
>> on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
>>
>>> On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@...> wrote:
>>>>> and you have to register to download ) .
>>>> That's not a huge problem is it?
>>> It's still a problem.
>>
>> In what way?
>
> As a rule, the need to register to access something is always an annoyance.

Understood.

--
Dave Abrahams           Meet me at BoostCon: http://www.boostcon.com
BoostPro Computing
http://www.boostpro.com

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Olaf van der Spek-2]

On Wed, Oct 28, 2009 at 7:58 PM, David Abrahams <dave@...> wrote:

>
> on Wed Oct 28 2009, Mathias Gaunard <mathias.gaunard-AT-ens-lyon.org> wrote:
>
>> David Abrahams a écrit :
>>> on Wed Oct 28 2009, Olaf van der Spek <olafvdspek-AT-gmail.com> wrote:
>>>
>>>> On Wed, Oct 28, 2009 at 6:30 PM, David Abrahams <dave@...> wrote:
>>>>>> and you have to register to download ) .
>>>>> That's not a huge problem is it?
>>>> It's still a problem.
>>>
>>> In what way?
>>
>> As a rule, the need to register to access something is always an annoyance.
>
> Understood.

What is the reason to require registration?

Olaf
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Stefan Seefeld]

Hello,

On 10/28/2009 11:34 AM, Julian Bangert wrote:
> Hello,
>
> yesterday I tried to download the boostpro binaries for windows and I
> had to download over a gigabyte to get all configurations for VS 2008.

Sadly, the only point being discussed so far is the online registration.
What I find to be at least as annoying is the size of the package. And
by that I'm not (only) referring to the memory or bandwidth. The most
important issue is the burden of having to deal with such an amount of
data, even if I'm only interested into a relatively small part of boost.

I wonder whether there is still any thought or even effort spent on
breaking boost up into individual components that can be built, tested,
and ultimately also packaged separately. Any news on that ?

Thanks,
         Stefan

--

       ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Julian Bangert-2]

Hello,


 > Sadly, the only point being discussed so far is the online registration.
 > What I find to be at least as annoying is the size of the package. And
 > by that I'm not (only) referring to the memory or bandwidth. The most
 > important issue is the burden of having to deal with such an amount of
 > data, even if I'm only interested into a relatively small part of boost.
 >
My package contains everything in one archive. These 37MB offer quite
good compression  and you can choose what you wish to extract.
 > I wonder whether there is still any thought or even effort spent on
 > breaking boost up into individual components that can be built, tested,
 > and ultimately also packaged separately. Any news on that ?
 >
 > Thanks,
 >          Stefan
 >
Well, a lot of Linux distributions already have this ( all the libboost-
packages in debian for examples ) . In W32 it is possible even with
BoostPro and in my distribution you can selectively extract libs.

Tell me if this is wanted/appreciated and I can write a GUI frontend to
automatically install components.
I just uploaded the files to 2 sites for distribution, at least one of
which should be up at any given time:
1) ( the mirror, but the preferred location )
http://boost-win-bin.googlecode.com/files/boost_1_40_0-vc9-preview.7z*
2) ( main site, but not on  a good machine)
https://obliviononline.com/pub/boost/
*After a few people confirm that this works for them they should be
considered stable and I would build for other C versions as well. Then
they could also be mentioned in getting started.
*
*Thanks, Julian Bangert





__________ Information from ESET NOD32 Antivirus, version of virus signature database 4553 (20091028) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com


_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Stefan Seefeld]

On 10/28/2009 04:34 PM, Julian Bangert wrote:
>
> > I wonder whether there is still any thought or even effort spent on
> > breaking boost up into individual components that can be built, tested,
> > and ultimately also packaged separately. Any news on that ?
>
> Well, a lot of Linux distributions already have this ( all the
> libboost- packages in debian for examples ) . In W32 it is possible
> even with BoostPro and in my distribution you can selectively extract
> libs.

See, this is precisely the problem: Each platform uses a different
mechanism. This is not acceptable for boost users who may target
different platforms, as there is no uniform way to describe the
dependency on boost.
Imagine how wonderful a world would be in which boost.org itself would
define the package structure, which then would be adopted by all the
different platforms / distributions.
Add to that the lack of forward / backward compatibility between
individual boost releases, and you have got a total maintenance nightmare.

Am I the only one who finds this extremely hard to work with ?

         Stefan

--

       ...ich hab' noch einen Koffer in Berlin...

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Julian Bangert-2]

Hello,
I just noticed an error crept into my earlier posted link:
The correct link for google is:
http://boost-win-bin.googlecode.com/files/boost_1_40_0-vc9-preview.7z
I would appreciate any comments on this distribution, so that  I could
pack it up with an installer and really "release" it.
Again, this distribution pretty much contains all of boost - all
configurations for a single version of Visual C++, without Python or
MPI, but is very tightly pakced into around 30 MB  ( a 30th of the
current  boostpro distribution size )

Yrs, Julian Bangert

[jbangert.vcf]

---

begin:vcard
fn:Julian Bangert
n:;Julian Bangert
email;internet:jbangert@...
version:2.1
end:vcard

---

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Christopher Jefferson]

On 3 Nov 2009, at 22:31, Julian Bangert wrote:

> Hello,
> I just noticed an error crept into my earlier posted link:
> The correct link for google is: http://boost-win-bin.googlecode.com/files/boost_1_40_0-vc9-preview.7z
> I would appreciate any comments on this distribution, so that  I  
> could pack it up with an installer and really "release" it.
> Again, this distribution pretty much contains all of boost - all  
> configurations for a single version of Visual C++, without Python or  
> MPI, but is very tightly pakced into around 30 MB  ( a 30th of the  
> current  boostpro distribution size )

Please put all your stuff into a folder -- I extracted it into my  
downloads directory, then had to pick it apart from the other stuff  
there. Obviously a proper installer would fix this.

You don't seem to package zlib and bzip2 libraries for  
boost::iostreams compression. I've previously had great difficulties  
getting these to build on windows, so if you provided them, that would  
be nice.

>
> Yrs, Julian Bangert
> <jbangert.vcf>_______________________________________________
> Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[David Abrahams-3]

on Wed Oct 28 2009, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:

> Hello,
>
> On 10/28/2009 11:34 AM, Julian Bangert wrote:
>> Hello,
>>
>> yesterday I tried to download the boostpro binaries for windows and I had to
>> download over a gigabyte to get all configurations for VS 2008.
>
> Sadly, the only point being discussed so far is the online registration. What I find
> to be at least as annoying is the size of the package. And by that I'm not (only)
> referring to the memory or bandwidth. The most important issue is the burden of having
> to deal with such an amount of data, even if I'm only interested into a relatively
> small part of boost.

Huh?  Admittedly the headers are all-or-nothing, but the Boostpro
installer lets you choose exactly the binaries you're interested in.

> I wonder whether there is still any thought or even effort spent on breaking boost up
> into individual components that can be built, tested, and ultimately also packaged
> separately. Any news on that ?

I'm hoping Troy can give us a public report on the modularization
effort.  Troy?

--
Dave Abrahams           Meet me at BoostCon: http://www.boostcon.com
BoostPro Computing
http://www.boostpro.com

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Zachary Turner-2]

On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@...> wrote:

>
> on Wed Oct 28 2009, Stefan Seefeld <seefeld-AT-sympatico.ca> wrote:
>
> > Hello,
> >
> > On 10/28/2009 11:34 AM, Julian Bangert wrote:
> >> Hello,
> >>
> >> yesterday I tried to download the boostpro binaries for windows and I
> had to
> >> download over a gigabyte to get all configurations for VS 2008.
> >
> > Sadly, the only point being discussed so far is the online registration.
> What I find
> > to be at least as annoying is the size of the package. And by that I'm
> not (only)
> > referring to the memory or bandwidth. The most important issue is the
> burden of having
> > to deal with such an amount of data, even if I'm only interested into a
> relatively
> > small part of boost.
>
> Huh?  Admittedly the headers are all-or-nothing, but the Boostpro
> installer lets you choose exactly the binaries you're interested in.
>
> > I wonder whether there is still any thought or even effort spent on
> breaking boost up
> > into individual components that can be built, tested, and ultimately also
> packaged
> > separately. Any news on that ?
>
> I'm hoping Troy can give us a public report on the modularization
> effort.  Troy?
>

Since we're talking about the boostpro installers, it would be great if:

a) they included the option of 64-bit binaries
b) There were filters of some kind so that I could easily target a specific
compiler / platform / etc.  So maybe a set of checkboxes for platform
(x86/x64), a set of checkboxes for configuration (Debug/Release), and a set
of checkboxes for compiler version (vc8, vc9, vc10).  Then two additional
buttons "enable all" and "disable all".  So if I check
x86/x64/Debug/Release/vc9, then click enable all, it would automatically
enable or disable the appropriate items.  As it is, it takes forever to go
through the tree manually expanding every single library, then
debug/release, then the compiler version which I would think is >90% of the
use cases.
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[troy d. straszheim]

> On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@...>
>> I'm hoping Troy can give us a public report on the modularization
>> effort.  Troy?

I suppose a picture is worth a thousand words:

   http://sodium.resophonic.com/boost/boost-dependencies.jpg

Please stop reading now and look at that.  Look for cycles and
dependencies that don't seem to make sense.

Realize that if you took the dependencies of test binaries into account
you'd just add more edges.  I don't know how many more.

This graph was generated by graphviz from dependency information encoded
in Boost.CMake's 'module.cmake' files, located in each library's source
directory.  Boost.CMake has the ability to reorganize headers so that
instead of being held in one directory, they are held in ~70 (IIRC).

First I'll explain my take on the tough bits about reorganizing boost's
directory structure, then I'll explain why I think this is a bad idea.

As I recently wrote offlist, regarding what happens to include paths
when each of ~100 libraries has its own include/ directory:

>
> Here's my back-of-the-envelope sketch of the discussion as I last recall
> it.  
>
> The hard use case is the library that is dependent on all the rest of
> boost.  Here, adding a bunch of -I flags to the compile line and
> calculating some dependencies isn't the problem, as no sane person puts
> 100 header directories on one compile line, it is hard on the eyes and
> just doesn't scale.  Eventually you're going to run out of commandline
 > buffer space somewhere.

>
> CMake's 'modularize' target moves headers out of toplevel boost/ as
> follows, this was uncontroversial:
>
> libs/
>   python/
>     src/
>       ...
>     doc/
>       ...
>     test/
>       ...
>     examples/
>       ...
>     include/
>       boost/        <----  moved files
>         python.hpp
>         python/
>
> and for each library p_i in a project's P's dependencies,
> -I$BOOST_ROOT/libs/p_i/include was added to P's compile line.
>
> So with 100 dependencies, how do you present all of these things to the
> compiler such that you've got a reasonable compile line?  .rsp files may
> do it on windows.  Elsewhere, you're going to need to somehow construct
> a single header directory on disk.  Maybe the mechanism is build in to
> source control, maybe it isn't.  Possibilities:
>
> - symlinks (not on windows you don't)
> - hardlinks created on checkout by version control
> - hardlinks generated by script
> - one generated directory full of forwarding headers (Qt has done this
> for years.  They check these files in.  I implemented a python script to
> do this for boost at some point.).
> - svn:externals (I used to advocate this, not any more)
> - git submodules (I don't advocate this either)
>
> My view:  I would prefer to follow Qt's method as it is simple, does not rely
> on version control tricks, and can do sanity checking for duplicate
> files and conventions about where one is allowed to put things.  I
> believe I would distribute the generated header directory in release
> tarballs but not check it in.  Developers would have to learn to
> regenerate the main header directory when adding/removing headers.

But since then my view has changed.  Now why this is a bad idea:

Shuffling headers around just makes the problem worse.  It won't remove
any edges from that graph.

Take the parameter -> python dependency, for instance.  Parameter is
dependent on python because it uses python's internal version of
referent_storage (basically just aligned_storage) in only one place.
Python's version of boost/aligned_storage.hpp dates from 2002,
presumably before boost/aligned_storage.hpp came along.   This is easy
to fix: just point them both at the toplevel aligned_storage, and you're
done, and an edge disappears from that graph.

There are probably hundreds more cases like this.  Remove those edges.
Then you would start seeing what modularity might look like.  As when
detangling a large knot:  gently tug at the loose bits and see if you
can unravel it, being careful not to make it worse.

Shuffling headers around and making source control more complicated, on
the other hand, won't remove any edges from that graph.

So I would:

- Write a script to generate that graph from the header files themselves.
- Declare a moratorium on new libraries.
- Start removing edges.

-t
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Rene Rivera-2]

troy d. straszheim wrote:
> So I would:
>
> - Write a script to generate that graph from the header files themselves.
> - Declare a moratorium on new libraries.
> - Start removing edges.

Yep... And the really entertaining part is that removing edges might
involve violating the moratorium. So what we are talking about is
serious refactoring of code and libraries... Something that might be
called Boost 2.0.

--
-- Grafik - Don't Assume Anything
-- Redshift Software, Inc. - http://redshift-software.com
-- rrivera/acm.org (msn) - grafik/redshift-software.com
-- 102708583/icq - grafikrobot/aim,yahoo,skype,efnet,gmail
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Steven Watanabe-4]

AMDG

troy d. straszheim wrote:

>> On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@...>
>>> I'm hoping Troy can give us a public report on the modularization
>>> effort.  Troy?
>
> I suppose a picture is worth a thousand words:
>
>   http://sodium.resophonic.com/boost/boost-dependencies.jpg
>
> Please stop reading now and look at that.  Look for cycles and
> dependencies that don't seem to make sense.

I actually don't see /very/ much there that doesn't make sense.
I really don't see what the problem is.  Maybe some refactoring
is in order, but I think that major refactoring is probably a waste
of time.

> Realize that if you took the dependencies of test binaries into
> account you'd just add more edges.  I don't know how many more.

In Christ,
Steven Watanabe

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Eric Niebler-3]

Steven Watanabe wrote:

> troy d. straszheim wrote:
>>> On Wed, Nov 4, 2009 at 9:08 PM, David Abrahams <dave@...>
>>>> I'm hoping Troy can give us a public report on the modularization
>>>> effort.  Troy?
>>
>> I suppose a picture is worth a thousand words:
>>
>>   http://sodium.resophonic.com/boost/boost-dependencies.jpg
>>
>> Please stop reading now and look at that.  Look for cycles and
>> dependencies that don't seem to make sense.
>
> I actually don't see /very/ much there that doesn't make sense.

I do. Why does spirit depend on xpressive? Why does xpressive depend on
intrusive? And why *doesn't* proto depend on mpl? These are just the
first three things I checked. :-P

--
Eric Niebler
BoostPro Computing
http://www.boostpro.com
_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[troy d. straszheim]

Rene Rivera wrote:

> troy d. straszheim wrote:
>> So I would:
>>
>> - Write a script to generate that graph from the header files themselves.
>> - Declare a moratorium on new libraries.
>> - Start removing edges.
>
> Yep... And the really entertaining part is that removing edges might
> involve violating the moratorium. So what we are talking about is
> serious refactoring of code and libraries... Something that might be
> called Boost 2.0.
>

+1

-t

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost