OOo source split

View: New views
20 Messages — Rating Filter:   Alert me  
< Prev | 1 - 2 | Next >

OOo source split

by Jan Holesovsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

During the OOoCon, Petr had a presentation about the OOo package splitting.  
The most important part for a (Linux) package maintainer was to be able to
build parts of OpenOffice.org separately; the thing is that with all the
localizations, we are unable to get the build times under some 7 hours.  But
the build could be done nicely in parallel (on the level of machines, not
processors) if the sources were split correctly, with correct rpms and -devel
rpms [of course, applies to debs as well ;-)].  And of course, the -noarch
parts like the translations could be built just once for all architectures.

I propose the following split of the sources [the sizes are of the unpacked
sources]:

75M     ure
25M     ooo-bootstrap
141M    ooo-libs-core
101M    ooo-libs-guitoolkit
142M    ooo-libs-3rdparty
17M     ooo-apps-base
28M     ooo-apps-calc
38M     ooo-apps-extensions
14M     ooo-apps-chart
40M     ooo-apps-impress
40M     ooo-apps-writer
59M     ooo-artwork
107M    ooo-filters
888M    ooo-l10n
48M     ooo-sdk
72M     ooo-testing
(1.8G   total)

(See below the content of these tarballs/archives).  I don't want the
granularity to be too fine (we would get the modules we have now, but as
separate packages), and OTOH the current 5 packages are too few.  The build
order of these would be:

ooo-bootstrap
ooo-libs-3rdparty
ure
ooo-libs-guitoolkit
ooo-libs-core
[the rest in whatever sequence/in parallel]

This would tremendously decrease the learning curve for the new developers as
well.  Imagine someone who wants to start hacking on Calc.  Instead of the
monster 1.8G sources, he would have to handle 512MB.  Additionally, the goal
of the modern Linux distros should be to get rid of the ooo-libs-3rdparty
completely - it contains just stuff that is available from other sources
anyway [-system stuff], and the distros have packages for them -, thus
additional 142M down, doing it just 370M.  And that is much more pleasant,
isn't it? ;-)

Of course, this is not finalized etc. - that's why I'm asking for comments.  
So far I was able to build in this order with few hacks, eg. scp2 should be
split so that the file lists are local, the l10n part must be buildable
separtately, etc.

So - what do you think? ;-)  ooo-l10n in the current proposal contains (in
addition to the few modules) all the localize.sdf's - should we split this a
bit as well?

Following is the proposal what I think belongs where:

===== ure =====

bridges
cli_ure
codemaker
cppu
cppuhelper
cpputools
idlc
io
javaunohelper
jurt
jut
jvmaccess
jvmfwk
offapi
offuh
pyuno
rdbmaker
registry
remotebridges
ridljar
sal
salhelper
stoc
store
udkapi
unoil
ure
xml2cmp

===== ooo-apps-base =====

dbaccess
reportdesign

===== ooo-apps-calc =====

sc

===== ooo-apps-extensions =====

accessibility
automation
basctl
bean
crashrep
embedserv
extensions
forms
javainstaller2
lingucomponent
MathMLDTD
package
setup_native
UnoControls
wizards
xmlsecurity

===== ooo-apps-chart =====

chart2

===== ooo-apps-impress =====

animations
sd
slideshow

===== ooo-apps-writer =====

starmath
sw

===== ooo-artwork =====

default_images
external_images
ooo_custom_images

===== ooo-bootstrap =====

config_office
dmake
instsetoo_native
postprocess
scp2
solenv
soltools
stlport

===== ooo-filters =====

binfilter
filter
hwpfilter
unoxml
writerfilter
writerperfect
xmerge

===== ooo-libs-core =====

avmedia
basic
configmgr
connectivity
desktop
embeddedobj
eventattacher
fileaccess
fpicker
framework
idl
linguistic
officecfg
oovbaapi
sandbox
scripting
sfx2
shell
sj2
so3
svx
sysui
ucb
uui
xmlhelp
xmloff
xmlscript
XmlSearch

===== ooo-libs-guitoolkit =====

basebmp
basegfx
canvas
comphelper
cppcanvas
dtrans
goodies
i18npool
i18nutil
o3tl
padmin
psprint
psprint_config
regexp
rsc
sax
scaddins
sot
svtools
toolkit
tools
transex3
ucbhelper
unotools
vcl
vos

===== ooo-libs-3rdparty =====

afms
agg
beanshell
berkeleydb
bitstream_vera_fonts
boost
curl
dictionaries
epm
expat
external
fondu
freetype
hsqldb
icu
jfreereport
jpeg
libegg
libtextcat
libwpd
libxml2
libxmlsec
libxslt
moz
msfontextract
nas
neon
np_sdk
portaudio
python
rhino
sane
sndfile
twain
unixODBC
vigra
xalan
xt
x11_extensions
zlib

===== ooo-l10n =====

extras
helpcontent2
readlicense_oo

===== ooo-sdk =====

autodoc
cosv
odk
sdk_oo
udm
unodevtools

===== ooo-testing =====

qadevOOo
smoketestoo_native
testshl2
testtools

Regards,
Jan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Jan Holesovsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

On Monday 08 October 2007 14:57, Jan Holesovsky wrote:

> I propose the following split of the sources [the sizes are of the unpacked
> sources]:
>
> 75M     ure
> 25M     ooo-bootstrap
> 141M    ooo-libs-core
> 101M    ooo-libs-guitoolkit
> 142M    ooo-libs-3rdparty
> 17M     ooo-apps-base
> 28M     ooo-apps-calc
> 38M     ooo-apps-extensions
> 14M     ooo-apps-chart
> 40M     ooo-apps-impress
> 40M     ooo-apps-writer
> 59M     ooo-artwork
> 107M    ooo-filters
> 888M    ooo-l10n
> 48M     ooo-sdk
> 72M     ooo-testing
> (1.8G   total)

Sorry, this contains even the filesystem overhead.  Without it (counting just
the file sizes) it's:

52M     ure
17M     ooo-bootstrap
107M    ooo-libs-core
83M     ooo-libs-guitoolkit
137M    ooo-libs-3rdparty
12M     ooo-apps-base
23M     ooo-apps-calc
26M     ooo-apps-extensions
11M     ooo-apps-chart
35M     ooo-apps-impress
33M     ooo-apps-writer
14M     ooo-artwork
83M     ooo-filters
863M    ooo-l10n
40M     ooo-sdk
34M     ooo-testing
(1.6G   total)

Regards,
Jan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Rüdiger Timm :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message



Jan Holesovsky wrote:
> Hi,

Hi Jan,

>
> During the OOoCon, Petr had a presentation about the OOo package splitting.  
> The most important part for a (Linux) package maintainer was to be able to
> build parts of OpenOffice.org separately; the thing is that with all the
> localizations, we are unable to get the build times under some 7 hours.  But
> the build could be done nicely in parallel (on the level of machines, not
> processors) if the sources were split correctly, with correct rpms and -devel
> rpms [of course, applies to debs as well ;-)].  And of course, the -noarch

Sorry, I do not understand this. How do you want to build f.e.
ooo-bootstrap, ooo-libs-core, and ooo-apps-writer in parrallel?
Bootstrap stuff like dmake or solenv is needed everywhere. Building
writer needs nearly all lower libraries in place.

> parts like the translations could be built just once for all architectures.
>
> I propose the following split of the sources [the sizes are of the unpacked
> sources]:
>
> 75M     ure
> 25M     ooo-bootstrap
> 141M    ooo-libs-core
> 101M    ooo-libs-guitoolkit
> 142M    ooo-libs-3rdparty
> 17M     ooo-apps-base
> 28M     ooo-apps-calc
> 38M     ooo-apps-extensions
> 14M     ooo-apps-chart
> 40M     ooo-apps-impress
> 40M     ooo-apps-writer
> 59M     ooo-artwork
> 107M    ooo-filters
> 888M    ooo-l10n
> 48M     ooo-sdk
> 72M     ooo-testing
> (1.8G   total)
>
> (See below the content of these tarballs/archives).  I don't want the
> granularity to be too fine (we would get the modules we have now, but as
> separate packages), and OTOH the current 5 packages are too few.  The build
> order of these would be:
>
> ooo-bootstrap
> ooo-libs-3rdparty
> ure
> ooo-libs-guitoolkit
> ooo-libs-core
> [the rest in whatever sequence/in parallel]
>
> This would tremendously decrease the learning curve for the new developers as
> well.  Imagine someone who wants to start hacking on Calc.  Instead of the
> monster 1.8G sources, he would have to handle 512MB.  Additionally, the goal

How should that work with your proposal? He would need everything
'below' calc. Except he uses, what we provide anyway: download 'solver'
tarball and than check out just module 'sc'. That's exactly for the
purpose you mention.

> of the modern Linux distros should be to get rid of the ooo-libs-3rdparty
> completely - it contains just stuff that is available from other sources
> anyway [-system stuff], and the distros have packages for them -, thus
> additional 142M down, doing it just 370M.  And that is much more pleasant,
> isn't it? ;-)
>
> Of course, this is not finalized etc. - that's why I'm asking for comments.  
> So far I was able to build in this order with few hacks, eg. scp2 should be
> split so that the file lists are local, the l10n part must be buildable
> separtately, etc.
>
> So - what do you think? ;-)  ooo-l10n in the current proposal contains (in

Personally, I do not like spitting up sources at all. But that's my very
personal opinion.
Besides this, I do not understand how your proposal could work (see
above). So I would propose existing and well tested means to achieve
nearly the same goals. F.e., the build tool provides a possibility to
build distributed on several computers.

> addition to the few modules) all the localize.sdf's - should we split this a
> bit as well?

There already is work in progress on taking localize.sdf files out of
the modules and concentrate them in one place.
>
> Following is the proposal what I think belongs where:
>
> [...]

Ruediger

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Jan Holesovsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Ruediger,

On Monday 08 October 2007 17:36, Rüdiger Timm wrote:

> > During the OOoCon, Petr had a presentation about the OOo package
> > splitting. The most important part for a (Linux) package maintainer was
> > to be able to build parts of OpenOffice.org separately; the thing is that
> > with all the localizations, we are unable to get the build times under
> > some 7 hours.  But the build could be done nicely in parallel (on the
> > level of machines, not processors) if the sources were split correctly,
> > with correct rpms and -devel rpms [of course, applies to debs as well
> > ;-)].  And of course, the -noarch
>
> Sorry, I do not understand this. How do you want to build f.e.
> ooo-bootstrap, ooo-libs-core, and ooo-apps-writer in parrallel?
> Bootstrap stuff like dmake or solenv is needed everywhere. Building
> writer needs nearly all lower libraries in place.

Sorry, it seems I should have been more verbose.  As you can see below, I
don't want these particular build in parallel, their order is fixed:

> > ooo-bootstrap
> > ooo-libs-3rdparty
> > ure
> > ooo-libs-guitoolkit
> > ooo-libs-core

And from this point:

> > [the rest in whatever sequence/in parallel]

It's the ooo-apps-writer, ooo-apps-calc, ... ooo-l10n that could be built in
parallel and on different machines.

Why don't I want to merge the 'fixed order' ones into one module?  Simply
'divide et impera' ;-)  They contain stuff that belongs more or less
logically together.  Eg. I believe we can shrink ooo-libs-core by just making
some things better, but it's hard to start when one is overwhelmed by the
amount of modules.

Regards,
Jan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Shaun McDonald :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


On 8 Oct 2007, at 13:57, Jan Holesovsky wrote:

> Hi,
>
> [...]  And of course, the -noarch
> parts like the translations could be built just once for all  
> architectures.
>

Language packs are currently system specific. Thus building once for  
all archs is not currently possible. This would require quite a lot  
of rework, which I'd be very happy to see.

Some info:
http://shaunmcdonald131.blogspot.com/2007/03/openofficeorg-language- 
pack-revamp.html

Shaun
[...]

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Jan Holesovsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Shaun,

On Monday 08 October 2007 21:14, Shaun McDonald wrote:

> > [...]  And of course, the -noarch
> > parts like the translations could be built just once for all
> > architectures.
>
> Language packs are currently system specific. Thus building once for
> all archs is not currently possible. This would require quite a lot
> of rework, which I'd be very happy to see.
>
> Some info:
> http://shaunmcdonald131.blogspot.com/2007/03/openofficeorg-language-
> pack-revamp.html

Well, I'm talking of _translations_, not language packs with their current OOo
meaning.  The translations are indeed -noarch, we already ship them in
openSUSE 10.3 as such ;-)

Eg. the spellchecker (hunspell) itself is in the lingucomponent which I
propose to put to ooo-apps-extensions (and thus to ship it together with the
application).  The dictionaries for it are in ooo-libs-3rdparty/dictionaries
- the distros have their own packages, but it still must be possible to build
with the internal ones.

Regards,
Jan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Jan Holesovsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Ruediger,

I am sorry, I missed a part of the mail when I was answering previously :-(

On Monday 08 October 2007 17:36, Rüdiger Timm wrote:

> > This would tremendously decrease the learning curve for the new
> > developers as well.  Imagine someone who wants to start hacking on Calc.
> > Instead of the monster 1.8G sources, he would have to handle 512MB.
> > Additionally, the goal
>
> How should that work with your proposal? He would need everything
> 'below' calc.

Yes, but do we provide an easy way to show him/her what _exactly_ is below
calc?  I was overwhelmed when I saw the number of the modules for the first
time [and there was much less of them at that time ;-)].  With the split,
everything is much clearer - my ideal newcomer would tell himself/herself
"OK, here is some bootstrap stuff, I don't care, some libs, maybe, but what I
want is to hack on calc, so ooo-apps-calc.  If I ever need something below,
I'll learn that later."

> Except he uses, what we provide anyway: download 'solver'
> tarball and than check out just module 'sc'. That's exactly for the
> purpose you mention.

Well, if the potential contributor has to learn what is that 'solver', we
again increase the learning curve.  And using it?  I tried it once when I
started with OOo hacking (as a volunteer), and after having to have the same
compiler & toolchain that was used for the solver generation, I just gave up
and let my computer building for 24 hours.

We have o3-build CD now; but our goal should be what the world outside uses
- ./configure ; make ; make install, and ideally in each of the
modules.  ./configure in eg. ooo-apps-calc would just tell you "Sorry, you
don't have ure, please build it first" if you don't have it yet.  No need to
go through tons of documentation to learn such a simple fact.  [And of course
- in the future it might very well happen that the user already has a
sufficient version of URE in the system anyway ;-)]

> > So - what do you think? ;-)  ooo-l10n in the current proposal contains
> > (in
>
> Personally, I do not like spitting up sources at all. But that's my very
> personal opinion.

I wonder why? ;-)  We (OpenOffice.org) already distribute the sources split to
binfilter, sdk, system, l10n and core, let's make the second step...

> Besides this, I do not understand how your proposal could work (see
> above).

I hope I was clearer in the other mail to you; if not, I'll be happy to
explain more.

> So I would propose existing and well tested means to achieve
> nearly the same goals. F.e., the build tool provides a possibility to
> build distributed on several computers.

May I ask for a documentation/wiki pointer, please?

> > addition to the few modules) all the localize.sdf's - should we split
> > this a bit as well?
>
> There already is work in progress on taking localize.sdf files out of
> the modules and concentrate them in one place.

Great, whom to ask about this, please?

Regards,
Jan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Rüdiger Timm :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message



Jan Holesovsky wrote:
> Hi Ruediger,
>
> I am sorry, I missed a part of the mail when I was answering previously :-(

No problem.

>
> On Monday 08 October 2007 17:36, Rüdiger Timm wrote:
>
>>> This would tremendously decrease the learning curve for the new
>>> developers as well.  Imagine someone who wants to start hacking on Calc.
>>> Instead of the monster 1.8G sources, he would have to handle 512MB.
>>> Additionally, the goal
>> How should that work with your proposal? He would need everything
>> 'below' calc.
>
> Yes, but do we provide an easy way to show him/her what _exactly_ is below
> calc?  I was overwhelmed when I saw the number of the modules for the first
> time [and there was much less of them at that time ;-)].  With the split,
> everything is much clearer - my ideal newcomer would tell himself/herself
> "OK, here is some bootstrap stuff, I don't care, some libs, maybe, but what I
> want is to hack on calc, so ooo-apps-calc.  If I ever need something below,
> I'll learn that later."

If that really is his desire, nowadays he just needs a wiki page telling
that 'calc' basically is module 'sc'. Than check out 'sc' and start hacking.

Learning OOo is hard, no doubt. I just do not think that splitting souce
code archive would make it any easier.

>
>> Except he uses, what we provide anyway: download 'solver'
>> tarball and than check out just module 'sc'. That's exactly for the
>> purpose you mention.
>
> Well, if the potential contributor has to learn what is that 'solver', we
> again increase the learning curve.  And using it?  I tried it once when I
> started with OOo hacking (as a volunteer), and after having to have the same
> compiler & toolchain that was used for the solver generation, I just gave up
> and let my computer building for 24 hours.

OK, on unix you are right. May be we should restrict providing 'solver'
for windows. At least on linux it does not really make sense, as there
are so much different toolchains possible.
The idea may be good, but in practise ...

>
> [...]
>
>> So I would propose existing and well tested means to achieve
>> nearly the same goals. F.e., the build tool provides a possibility to
>> build distributed on several computers.
>
> May I ask for a documentation/wiki pointer, please?

http://tools.openoffice.org/servlets/ReadMsg?list=dev&msgNo=6214
and replies to that mail.

>
>>> addition to the few modules) all the localize.sdf's - should we split
>>> this a bit as well?
>> There already is work in progress on taking localize.sdf files out of
>> the modules and concentrate them in one place.
>
> Great, whom to ask about this, please?

Ivo (ihi). Ause also is involved, I think.
See also
http://www.openoffice.org/issues/show_bug.cgi?id=79750
http://eis.services.openoffice.org/EIS2/cws.ShowCWS?Path=SRC680%2Fl10ncleanup


Rüdiger

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Mathias Bauer :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Rüdiger Timm wrote:

> Personally, I do not like spitting up sources at all. But that's my very
> personal opinion.

Splitting up source definitely is a good idea. Maybe not for people
building everything anyway but it would be a huge step ahead for the
casual developer like volunteers, distro maintainers etc. And no, Solver
tarballs are not a replacement for this, they are yet another workaround
as I have learned when I had an email conversation with Petr.

So I definitely second the approach to split the source. The problem is
- as I reported in my presentation in Barcelona - that we have to rework
some libraries and even sources to move that forward and to effectively
gain anything from this. Currently we can achieve only a small benefit
but as always a large journey starts with the first step.

> Besides this, I do not understand how your proposal could work (see
> above). So I would propose existing and well tested means to achieve
> nearly the same goals. F.e., the build tool provides a possibility to
> build distributed on several computers.

You always refer to your "always build everything" approach. There's
more than this. Having a huge monolithic project structure and
workarounding this by providing tools to tame the beast is better than
nothing but perhaps it's time to improve. The result will not make a big
difference for the "always everything" people but it will help others.

So once reasonable packages have been defined we can think about
splitting the source also. The first preparations have been done (URE
split) or are under work (sdf split as you mentioned yourself). I think
there are a lot of reasonale packages that can be identified right now
where building them separately will work. I opt for helpcontent,
binfilter and all the applications.

And the next goal should be getting updated packages by just building
the source packages needed, not by always building full installation
sets. Can you imagine what a relief it would be not to build and pack
everything because you already have the binaries in your OOo
installation and only rebuild the Calc package because you only wanted
to fix a small bug in Calc?

Of course to be able to gain something from this we also need
"development packages" for the OOo packages. So there's something to do,
but why not start? Of course I take it for granted that those suggesting
the change will help doing it. ;-)

Ciao,
Mathias

--
Mathias Bauer (mba) - Project Lead OpenOffice.org Writer
OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS
Please don't reply to "nospamformba@...".
I use it for the OOo lists and only rarely read other mails sent to it.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Rüdiger Timm :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message



Mathias Bauer wrote:

> Rüdiger Timm wrote:
>
>> Personally, I do not like spitting up sources at all. But that's my very
>> personal opinion.
>
> Splitting up source definitely is a good idea. Maybe not for people
> building everything anyway but it would be a huge step ahead for the
> casual developer like volunteers, distro maintainers etc. And no, Solver
> tarballs are not a replacement for this, they are yet another workaround
> as I have learned when I had an email conversation with Petr.
>
> So I definitely second the approach to split the source. The problem is
> - as I reported in my presentation in Barcelona - that we have to rework
> some libraries and even sources to move that forward and to effectively
> gain anything from this. Currently we can achieve only a small benefit
> but as always a large journey starts with the first step.

Sorry, I was not clear in my statement.
I am not against having the possibility to get smaller, logically
connected parts of the code base separately. What I do not want (and
perhaps that was a misinterpretation of the original posting) is having
different parts in different, distinct archives. I am fine with getting
parts out of one repository (currently this would easily be possible by
introducing smaller aliases than the big OpenOffice2). But I do not want
to collect the stuff from a couple of different repositories.
And please do it with care. When OOo started our code base got
'splitted' by creating projects containg several modules. I wish we had
not done that. Unfortunately that grouping has prooven to be not
usefull. Having just plain modules side by side would be by far easier
than what we have now. We should not do such things again.

>
>> Besides this, I do not understand how your proposal could work (see
>> above). So I would propose existing and well tested means to achieve
>> nearly the same goals. F.e., the build tool provides a possibility to
>> build distributed on several computers.
>
> You always refer to your "always build everything" approach. There's
> more than this. Having a huge monolithic project structure and
> workarounding this by providing tools to tame the beast is better than
> nothing but perhaps it's time to improve. The result will not make a big
> difference for the "always everything" people but it will help others.

You are right, but that's what I've read in Jan's mail. He already
answered that I got him partly wrong.

>
> So once reasonable packages have been defined we can think about
> splitting the source also. The first preparations have been done (URE
> split) or are under work (sdf split as you mentioned yourself). I think
> there are a lot of reasonale packages that can be identified right now
> where building them separately will work. I opt for helpcontent,
> binfilter and all the applications.
>
> And the next goal should be getting updated packages by just building
> the source packages needed, not by always building full installation
> sets. Can you imagine what a relief it would be not to build and pack
> everything because you already have the binaries in your OOo
> installation and only rebuild the Calc package because you only wanted
> to fix a small bug in Calc?

Of course.

>
> Of course to be able to gain something from this we also need
> "development packages" for the OOo packages. So there's something to do,

Yes, that was my concern. Spitting code does not really help as long as
we do not provide corresponding binary packages.

> but why not start? Of course I take it for granted that those suggesting
> the change will help doing it. ;-)

My feeling is we should first do some work on our code base so that be
really can benefit from a split.

>
> Ciao,
> Mathias
>

Rüdiger

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Caolán McNamara :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message


On Thu, 2007-10-11 at 09:05 +0200, Rüdiger Timm wrote:
> My feeling is we should first do some work on our code base so that be
> really can benefit from a split.

Perhaps a good case study of such a split is the modular X effort which
broke the monolithic x.org build into separately buildable projects.
Even as a non x-hacker I found that really helpful, I could now just
build e.g. libXau and debug into it to figure out valgrind warnings and
usefully patch it to fix them and submit those fixes back. Something I
certainly wouldn't bother to have done if it was still a monolithic
multi-hour build as it just wouldn't have been worth my while.

C.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Jan Holesovsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Mathias,

On Wednesday 10 October 2007 21:14, Mathias Bauer wrote:

> > Personally, I do not like spitting up sources at all. But that's my very
> > personal opinion.
>
> Splitting up source definitely is a good idea. Maybe not for people
> building everything anyway but it would be a huge step ahead for the
> casual developer like volunteers, distro maintainers etc. And no, Solver
> tarballs are not a replacement for this, they are yet another workaround
> as I have learned when I had an email conversation with Petr.

Thank you for your support!

> So I definitely second the approach to split the source. The problem is
> - as I reported in my presentation in Barcelona - that we have to rework
> some libraries and even sources to move that forward and to effectively
> gain anything from this.

Yes, definitely.  If the libraries resulting from the split [svx was the most
problematic, right?] are going to end up in different packages (eg. one in
ooo-libs-core, and the other in ooo-apps-writer), then we have a light
version of chicken-egg problem ;-) - split first the library, and then create
the packages, or vice versa.  OTOH, the split into more source packages is
from my point of view easier to achieve, so I'd start there.

[...]
> And the next goal should be getting updated packages by just building
> the source packages needed, not by always building full installation
> sets. Can you imagine what a relief it would be not to build and pack
> everything because you already have the binaries in your OOo
> installation and only rebuild the Calc package because you only wanted
> to fix a small bug in Calc?

Fully agree :-)

> Of course to be able to gain something from this we also need
> "development packages" for the OOo packages. So there's something to do,
> but why not start? Of course I take it for granted that those suggesting
> the change will help doing it. ;-)

Oh, sure!  My original mail contains my suggestion of what belongs where -
please have a look, input is most appreciated.  I did few corrections
afterwards (accessibility, bean, crashrep, and package moved to
ooo-apps-extensions, jut moved to ure, rvpapi removed, it's not used any
more).

As I said, I was able to build using this split (after having created an
artifical module in each package containing the list of the packages in
prj/build.lst), so... ;-)  If you agree, I can setup experimental git
repositories with all this so that others can play with it as well.

Regards,
Jan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Parent Message unknown Re: OOo source split

by Martin Hollmichel - Sun Germany - ham02 - Hamburg :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Jan Holesovsky wrote:
> Eg. the spellchecker (hunspell) itself is in the lingucomponent which I
> propose to put to ooo-apps-extensions (and thus to ship it together with the
> application).  The dictionaries for it are in ooo-libs-3rdparty/dictionaries
> - the distros have their own packages, but it still must be possible to build
> with the internal ones.
>
I'd prefer to treat the dictionaries as an own package and not to bundle
them in a "super-source-package" ooo-libs-3rdparty package again. If we
have meaningful smallest possible packages we should go with them.
> Regards,
> Jan
>
Martin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Parent Message unknown Re: OOo source split

by Martin Hollmichel - Sun Germany - ham02 - Hamburg :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi,

just stumbled about that the report design extension is built during the
regular build process, wouldn't it be better at all to create a source
tarball include jfreereport and reportdesign modules. Where we already
achieved modularization in the sources we should IHMO also do the right
packaging of sources,

Martin

Jan Holesovsky wrote:

> Hi,
>
> During the OOoCon, Petr had a presentation about the OOo package splitting.  
> The most important part for a (Linux) package maintainer was to be able to
> build parts of OpenOffice.org separately; the thing is that with all the
> localizations, we are unable to get the build times under some 7 hours.  But
> the build could be done nicely in parallel (on the level of machines, not
> processors) if the sources were split correctly, with correct rpms and -devel
> rpms [of course, applies to debs as well ;-)].  And of course, the -noarch
> parts like the translations could be built just once for all architectures.
>
> I propose the following split of the sources [the sizes are of the unpacked
> sources]:
>
> 75M     ure
> 25M     ooo-bootstrap
> 141M    ooo-libs-core
> 101M    ooo-libs-guitoolkit
> 142M    ooo-libs-3rdparty
> 17M     ooo-apps-base
> 28M     ooo-apps-calc
> 38M     ooo-apps-extensions
> 14M     ooo-apps-chart
> 40M     ooo-apps-impress
> 40M     ooo-apps-writer
> 59M     ooo-artwork
> 107M    ooo-filters
> 888M    ooo-l10n
> 48M     ooo-sdk
> 72M     ooo-testing
> (1.8G   total)
>
> (See below the content of these tarballs/archives).  I don't want the
> granularity to be too fine (we would get the modules we have now, but as
> separate packages), and OTOH the current 5 packages are too few.  The build
> order of these would be:
>
> ooo-bootstrap
> ooo-libs-3rdparty
> ure
> ooo-libs-guitoolkit
> ooo-libs-core
> [the rest in whatever sequence/in parallel]
>
> This would tremendously decrease the learning curve for the new developers as
> well.  Imagine someone who wants to start hacking on Calc.  Instead of the
> monster 1.8G sources, he would have to handle 512MB.  Additionally, the goal
> of the modern Linux distros should be to get rid of the ooo-libs-3rdparty
> completely - it contains just stuff that is available from other sources
> anyway [-system stuff], and the distros have packages for them -, thus
> additional 142M down, doing it just 370M.  And that is much more pleasant,
> isn't it? ;-)
>
> Of course, this is not finalized etc. - that's why I'm asking for comments.  
> So far I was able to build in this order with few hacks, eg. scp2 should be
> split so that the file lists are local, the l10n part must be buildable
> separtately, etc.
>
> So - what do you think? ;-)  ooo-l10n in the current proposal contains (in
> addition to the few modules) all the localize.sdf's - should we split this a
> bit as well?
>
> Following is the proposal what I think belongs where:
>
> ===== ure =====
>
> bridges
> cli_ure
> codemaker
> cppu
> cppuhelper
> cpputools
> idlc
> io
> javaunohelper
> jurt
> jut
> jvmaccess
> jvmfwk
> offapi
> offuh
> pyuno
> rdbmaker
> registry
> remotebridges
> ridljar
> sal
> salhelper
> stoc
> store
> udkapi
> unoil
> ure
> xml2cmp
>
> ===== ooo-apps-base =====
>
> dbaccess
> reportdesign
>
> ===== ooo-apps-calc =====
>
> sc
>
> ===== ooo-apps-extensions =====
>
> accessibility
> automation
> basctl
> bean
> crashrep
> embedserv
> extensions
> forms
> javainstaller2
> lingucomponent
> MathMLDTD
> package
> setup_native
> UnoControls
> wizards
> xmlsecurity
>
> ===== ooo-apps-chart =====
>
> chart2
>
> ===== ooo-apps-impress =====
>
> animations
> sd
> slideshow
>
> ===== ooo-apps-writer =====
>
> starmath
> sw
>
> ===== ooo-artwork =====
>
> default_images
> external_images
> ooo_custom_images
>
> ===== ooo-bootstrap =====
>
> config_office
> dmake
> instsetoo_native
> postprocess
> scp2
> solenv
> soltools
> stlport
>
> ===== ooo-filters =====
>
> binfilter
> filter
> hwpfilter
> unoxml
> writerfilter
> writerperfect
> xmerge
>
> ===== ooo-libs-core =====
>
> avmedia
> basic
> configmgr
> connectivity
> desktop
> embeddedobj
> eventattacher
> fileaccess
> fpicker
> framework
> idl
> linguistic
> officecfg
> oovbaapi
> sandbox
> scripting
> sfx2
> shell
> sj2
> so3
> svx
> sysui
> ucb
> uui
> xmlhelp
> xmloff
> xmlscript
> XmlSearch
>
> ===== ooo-libs-guitoolkit =====
>
> basebmp
> basegfx
> canvas
> comphelper
> cppcanvas
> dtrans
> goodies
> i18npool
> i18nutil
> o3tl
> padmin
> psprint
> psprint_config
> regexp
> rsc
> sax
> scaddins
> sot
> svtools
> toolkit
> tools
> transex3
> ucbhelper
> unotools
> vcl
> vos
>
> ===== ooo-libs-3rdparty =====
>
> afms
> agg
> beanshell
> berkeleydb
> bitstream_vera_fonts
> boost
> curl
> dictionaries
> epm
> expat
> external
> fondu
> freetype
> hsqldb
> icu
> jfreereport
> jpeg
> libegg
> libtextcat
> libwpd
> libxml2
> libxmlsec
> libxslt
> moz
> msfontextract
> nas
> neon
> np_sdk
> portaudio
> python
> rhino
> sane
> sndfile
> twain
> unixODBC
> vigra
> xalan
> xt
> x11_extensions
> zlib
>
> ===== ooo-l10n =====
>
> extras
> helpcontent2
> readlicense_oo
>
> ===== ooo-sdk =====
>
> autodoc
> cosv
> odk
> sdk_oo
> udm
> unodevtools
>
> ===== ooo-testing =====
>
> qadevOOo
> smoketestoo_native
> testshl2
> testtools
>
> Regards,
> Jan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@...
> For additional commands, e-mail: dev-help@...
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Mathias Bauer :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Rüdiger Timm schrieb:

> I am not against having the possibility to get smaller, logically
> connected parts of the code base separately. What I do not want (and
> perhaps that was a misinterpretation of the original posting) is having
> different parts in different, distinct archives.
Of course. As I understand it, the idea is to get more and better
defined packages and the ability to rebuild only parts of them. Whether
the code to build the packages comes from a single repository or not
doesn't make a difference, so keeping one repository obviously is fine.
But it should be possible to download only parts of the whole OOo source
and build only the parts you want.

> My feeling is we should first do some work on our code base so that be
> really can benefit from a split.

Absolutely. There are a few modules where we don't need this but over
all the code base will be the limiting factor.

Ciao,
Mathias

--
Mathias Bauer (mba) - Project Lead OpenOffice.org Writer
OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS
Please don't reply to "nospamformba@...".
I use it for the OOo lists and only rarely read other mails sent to it.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Mathias Bauer :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Martin Hollmichel schrieb:

> Hi,
>
> just stumbled about that the report design extension is built during the
> regular build process, wouldn't it be better at all to create a source
> tarball include jfreereport and reportdesign modules. Where we already
> achieved modularization in the sources we should IHMO also do the right
> packaging of sources,

+1!

What already is separated shouldn't become munged with the rest. We know
how fast the separation can get lost. :-)

Ciao,
Mathias

--
Mathias Bauer (mba) - Project Lead OpenOffice.org Writer
OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS
Please don't reply to "nospamformba@...".
I use it for the OOo lists and only rarely read other mails sent to it.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Mathias Bauer :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Jan Holesovsky schrieb:

> Oh, sure!  My original mail contains my suggestion of what belongs where -
> please have a look, input is most appreciated.  I did few corrections
> afterwards (accessibility, bean, crashrep, and package moved to
> ooo-apps-extensions, jut moved to ure, rvpapi removed, it's not used any
> more).

Yes, I already planned to have a look. I wish the day had more than 24
hours ... <sigh>

Ciao,
Mathias

--
Mathias Bauer (mba) - Project Lead OpenOffice.org Writer
OpenOffice.org Engineering at Sun: http://blogs.sun.com/GullFOSS
Please don't reply to "nospamformba@...".
I use it for the OOo lists and only rarely read other mails sent to it.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Jan Holesovsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Mathias,

On Friday 12 October 2007 20:18, Mathias Bauer wrote:

> > just stumbled about that the report design extension is built during the
> > regular build process, wouldn't it be better at all to create a source
> > tarball include jfreereport and reportdesign modules. Where we already
> > achieved modularization in the sources we should IHMO also do the right
> > packaging of sources,
>
> +1!
>
> What already is separated shouldn't become munged with the rest. We know
> how fast the separation can get lost. :-)

I am a bit confused here - I thought that jfreereport was not JCA covered
[though LGPL], so bundling it together was not what would you want on the
source level?

Either way, from my point of view it is plain 3rd party stuff, so I'd like to
let it in ooo-libs-3rdparty.  To avoid reportdesign intergrowth with Base,
maybe ooo-apps-extensions would be the better option for reportdesign, what
do you think?

Regards,
Jan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Re: OOo source split

by Jan Holesovsky :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Hi Martin,

On Friday 12 October 2007 12:37, Martin Hollmichel wrote:

> > Eg. the spellchecker (hunspell) itself is in the lingucomponent which I
> > propose to put to ooo-apps-extensions (and thus to ship it together with
> > the application).  The dictionaries for it are in
> > ooo-libs-3rdparty/dictionaries - the distros have their own packages, but
> > it still must be possible to build with the internal ones.
>
> I'd prefer to treat the dictionaries as an own package and not to bundle
> them in a "super-source-package" ooo-libs-3rdparty package again. If we
> have meaningful smallest possible packages we should go with them.

Yes, this is very tricky :-(  For every module from ooo-libs-3rdparty we could
have a single package, but I'm afraid it would make the granularity too fine.

My vision is that on modern Linux distros, you won't need ooo-libs-3rdparty at
all, because you will have all the packages in the distro already, and you'll
be able to install them from there (most probably with the corresponding
-devel packages).  But for the Win32 builders, searching for the dependencies
could be too expensive - that's why I propose the 'super-source-package' ;-)  
Of course, we can go the way of ooo-3rdparty-dictionaries, ooo-3rdparty-epm,
ooo-3rdparty-hsqldb, etc. but I'm not sure if it helps in the end...

Regards,
Jan

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...


Parent Message unknown Re: OOo source split

by Martin Hollmichel - Sun Germany - ham02 - Hamburg :: Rate this Message:

Reply to Author | View Threaded | Show Only this Message

Jan Holesovsky wrote:

> Hi Mathias,
>
> On Friday 12 October 2007 20:18, Mathias Bauer wrote:
>
>>> just stumbled about that the report design extension is built during the
>>> regular build process, wouldn't it be better at all to create a source
>>> tarball include jfreereport and reportdesign modules. Where we already
>>> achieved modularization in the sources we should IHMO also do the right
>>> packaging of sources,
>> +1!
>>
>> What already is separated shouldn't become munged with the rest. We know
>> how fast the separation can get lost. :-)
>
> I am a bit confused here - I thought that jfreereport was not JCA covered
> [though LGPL], so bundling it together was not what would you want on the
> source level?
>
yes, two packages would be required.
> Either way, from my point of view it is plain 3rd party stuff, so I'd like to
> let it in ooo-libs-3rdparty.  To avoid reportdesign intergrowth with Base,
> maybe ooo-apps-extensions would be the better option for reportdesign, what
> do you think?
I don't understand why you want to create superbundles again, even if a
more fine granular packaging is possible. Why should I care about
jfreereport if I don't want to build any extension but just the core
product ?
>
> Regards,
> Jan
>
Martin

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@...
For additional commands, e-mail: dev-help@...

< Prev | 1 - 2 | Next >