speeding up buildworld/kernel

[sarek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I update my sources at least once a day and do buildworld/kernel just
as often... It seems some stuff that needs not be recompiled is on
every single run for example gcc and kerbos.   I have NO_CLEAN in
/etc/make.conf is there anything else I can do to speed stuff up...
for ref here is my /etc/make.conf:

CPUTYPE?=nocona
KERNCONF=MONSTER
NO_CLEAN=
NO_LPR=
# added by use.perl 2008-01-17 11:48:48
PERL_VER=5.8.8
PERL_VERSION=5.8.8

- --
Aryeh M. Friedman
FloSoft Systems, Java Tool Developers
Developer, not business, friendly
http://www.flosoft-systems.com

    "Free software != Free beer"

Blog:
 
 http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHmPHPQi2hk2LEXBARAofVAKD5MBoQ24Wal5CjKng5bUv8Pp2/mQCfcX6p
NfCsz8egGQjn9KFPQ0Frths=
=XeOG
-----END PGP SIGNATURE-----

_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Maxim Khitrov]

On Jan 24, 2008 3:15 PM, Aryeh M. Friedman <aryeh.friedman@...> wrote:

> I update my sources at least once a day and do buildworld/kernel just
> as often... It seems some stuff that needs not be recompiled is on
> every single run for example gcc and kerbos.   I have NO_CLEAN in
> /etc/make.conf is there anything else I can do to speed stuff up...
> for ref here is my /etc/make.conf:
>
> CPUTYPE?=nocona
> KERNCONF=MONSTER
> NO_CLEAN=
> NO_LPR=
> # added by use.perl 2008-01-17 11:48:48
> PERL_VER=5.8.8
> PERL_VERSION=5.8.8
>
> - --
> Aryeh M. Friedman
> FloSoft Systems, Java Tool Developers
> Developer, not business, friendly
> http://www.flosoft-systems.com

I might be wrong, but NO_CLEAN seems like a bad idea except in special
circumstances. Install ccache, but make sure you set
CCACHE_HASH_COMPILER environment variable to 1. That will make sure
that the cache stays valid if the compiler executable is overwritten
by an identical copy (as it would be on installworld). When the
compiler changes the cache will be repopulated on the next rebuild.

- Max
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Erik Trulsson]

On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote:

> On Jan 24, 2008 3:15 PM, Aryeh M. Friedman <aryeh.friedman@...> wrote:
> > I update my sources at least once a day and do buildworld/kernel just
> > as often... It seems some stuff that needs not be recompiled is on
> > every single run for example gcc and kerbos.   I have NO_CLEAN in
> > /etc/make.conf is there anything else I can do to speed stuff up...
> > for ref here is my /etc/make.conf:
> >
> > CPUTYPE?=nocona
> > KERNCONF=MONSTER
> > NO_CLEAN=
> > NO_LPR=
> > # added by use.perl 2008-01-17 11:48:48
> > PERL_VER=5.8.8
> > PERL_VERSION=5.8.8
> >
> > - --
> > Aryeh M. Friedman
> > FloSoft Systems, Java Tool Developers
> > Developer, not business, friendly
> > http://www.flosoft-systems.com
>
> I might be wrong, but NO_CLEAN seems like a bad idea except in special
> circumstances. Install ccache, but make sure you set
> CCACHE_HASH_COMPILER environment variable to 1. That will make sure
> that the cache stays valid if the compiler executable is overwritten
> by an identical copy (as it would be on installworld). When the
> compiler changes the cache will be repopulated on the next rebuild.

You are indeed wrong.  NO_CLEAN will work fine almost all the time - except
in special circumstances.  The few times it does not work one can always do
a 'make clean' by hand first. (Or even faster: 'rm -fr /usr/obj/*')
If you set WRKDIRPREFIX to some useful value you can do the same thing
for the ports tree.
Personally I always compile with -DNO_CLEAN and use 'rm -fr' to clean.
I have never had problems originating with this.

ccache is not very useful for buildworld, since among the first thing
buildworld does is to build the compiler and then use the newly built
compiler to compile the rest. I.e. the already installed compiler (which is
the one ccache will handle) will not be used for most of the build thus
removing almost all the advantage of ccache.
It is supposed to be possible to use ccache for buildworld as well, but
that would require a bit of hackery.


As for speeding up the build even more there a couple of things that can be
tried:

  You can add NO_PROFILE=true to make.conf if you do not need profiling
  libraries.
 
  Set CFLAGS/COPTFLAGS to -O instead of -O2. This should speed up the
  compiler a bit since it will no have to do as much work.  This will make
  programs slightly less well optimized, but since the vast majority of the
  system binaries are not really CPU-bound anyway it is unlikely that any
  performance loss will be noticed.

  If you have more than one CPU-core in your machine (and an SMP-enabled
  kernel) you can use the -j flag to tell make to run several jobs in
  parallell.  Just be aware that building with -j does get broken
  occasionaly and there is no promise that it will always be fixed quickly.
  If you do run into problems when building with -j, try without -j before
  sending any bug reports.





--
<Insert your favourite quote here.>
Erik Trulsson
ertr1013@...
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[sarek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Erik Trulsson wrote:

> On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote:
>> On Jan 24, 2008 3:15 PM, Aryeh M. Friedman
>> <aryeh.friedman@...> wrote:
>>> I update my sources at least once a day and do
>>> buildworld/kernel just as often... It seems some stuff that
>>> needs not be recompiled is on every single run for example gcc
>>> and kerbos.   I have NO_CLEAN in /etc/make.conf is there
>>> anything else I can do to speed stuff up... for ref here is my
>>> /etc/make.conf:
>>>
>>> CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by
>>> use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8 PERL_VERSION=5.8.8
>>>
>>> - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers
>>> Developer, not business, friendly
>>> http://www.flosoft-systems.com
>> I might be wrong, but NO_CLEAN seems like a bad idea except in
>> special circumstances. Install ccache, but make sure you set
>> CCACHE_HASH_COMPILER environment variable to 1. That will make
>> sure that the cache stays valid if the compiler executable is
>> overwritten by an identical copy (as it would be on
>> installworld). When the compiler changes the cache will be
>> repopulated on the next rebuild.
>
> You are indeed wrong.  NO_CLEAN will work fine almost all the time
> - except in special circumstances.  The few times it does not work
> one can always do a 'make clean' by hand first. (Or even faster:
> 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some useful value
> you can do the same thing for the ports tree. Personally I always
> compile with -DNO_CLEAN and use 'rm -fr' to clean. I have never had
> problems originating with this.
>
> ccache is not very useful for buildworld, since among the first
> thing buildworld does is to build the compiler and then use the
> newly built compiler to compile the rest. I.e. the already
> installed compiler (which is the one ccache will handle) will not
> be used for most of the build thus removing almost all the
> advantage of ccache. It is supposed to be possible to use ccache
> for buildworld as well, but that would require a bit of hackery.
>
>
> As for speeding up the build even more there a couple of things
> that can be tried:
>
> You can add NO_PROFILE=true to make.conf if you do not need
> profiling libraries.

I thought most profiled libs had been removed in current but I will
try this.  I was also looking at NO_SHARED but my gut says this would
cause a sigficant performence hit.
>
> Set CFLAGS/COPTFLAGS to -O instead of -O2. This should speed up the
>  compiler a bit since it will no have to do as much work.  This
> will make programs slightly less well optimized, but since the vast
> majority of the system binaries are not really CPU-bound anyway it
> is unlikely that any performance loss will be noticed.

Do you have any numbers on this?
>
> If you have more than one CPU-core in your machine (and an
> SMP-enabled kernel) you can use the -j flag to tell make to run
> several jobs in parallell.  Just be aware that building with -j
> does get broken occasionaly and there is no promise that it will
> always be fixed quickly. If you do run into problems when building
> with -j, try without -j before sending any bug reports.

Since I like to run it in the background (i.e. while doing stuff on a
different X screen) I usually don't use -j unless I am doing a bare
metal install and then I typically do core*4+2 for it's value.
>
>
>
>
>


- --
Aryeh M. Friedman
FloSoft Systems, Java Tool Developers
Developer, not business, friendly
http://www.flosoft-systems.com

    "Free software != Free beer"

Blog:
 
http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHmP9KQi2hk2LEXBARAu7zAJ4/sGAzpMFCZOKkZBVx/s07KTRw9gCgwF1m
6ee/hiJIvj8gyieoq/ZxIz0=
=tnVh
-----END PGP SIGNATURE-----

_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Maxim Khitrov]

On Jan 24, 2008 4:05 PM, Erik Trulsson <ertr1013@...> wrote:

>
> On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote:
> > On Jan 24, 2008 3:15 PM, Aryeh M. Friedman <aryeh.friedman@...> wrote:
> > > I update my sources at least once a day and do buildworld/kernel just
> > > as often... It seems some stuff that needs not be recompiled is on
> > > every single run for example gcc and kerbos.   I have NO_CLEAN in
> > > /etc/make.conf is there anything else I can do to speed stuff up...
> > > for ref here is my /etc/make.conf:
> > >
> > > CPUTYPE?=nocona
> > > KERNCONF=MONSTER
> > > NO_CLEAN=
> > > NO_LPR=
> > > # added by use.perl 2008-01-17 11:48:48
> > > PERL_VER=5.8.8
> > > PERL_VERSION=5.8.8
> > >
> > > - --
> > > Aryeh M. Friedman
> > > FloSoft Systems, Java Tool Developers
> > > Developer, not business, friendly
> > > http://www.flosoft-systems.com
> >
> > I might be wrong, but NO_CLEAN seems like a bad idea except in special
> > circumstances. Install ccache, but make sure you set
> > CCACHE_HASH_COMPILER environment variable to 1. That will make sure
> > that the cache stays valid if the compiler executable is overwritten
> > by an identical copy (as it would be on installworld). When the
> > compiler changes the cache will be repopulated on the next rebuild.
>
> You are indeed wrong.  NO_CLEAN will work fine almost all the time - except
> in special circumstances.  The few times it does not work one can always do
> a 'make clean' by hand first. (Or even faster: 'rm -fr /usr/obj/*')
> If you set WRKDIRPREFIX to some useful value you can do the same thing
> for the ports tree.
> Personally I always compile with -DNO_CLEAN and use 'rm -fr' to clean.
> I have never had problems originating with this.
>
> ccache is not very useful for buildworld, since among the first thing
> buildworld does is to build the compiler and then use the newly built
> compiler to compile the rest. I.e. the already installed compiler (which is
> the one ccache will handle) will not be used for most of the build thus
> removing almost all the advantage of ccache.
> It is supposed to be possible to use ccache for buildworld as well, but
> that would require a bit of hackery.

That's not true. I just ran `make buildworld buildkernel` on my
firewall. Here are ccache stats when the operation finished:

root@cerberus [/root]# ccache -s
cache directory                     /srv/.ccache
cache hit                          12056
cache miss                            38
called for link                      461
multiple source files                  1
not a C/C++ file                    1228
unsupported compiler option            7
files in cache                    117366
cache size                         679.6 Mbytes
max cache size                       2.0 Gbytes

Ccache is used through the entire build process and there is no
hackery involved. Just follow the directions for changing the compiler
to /usr/local/libexec/ccache/world-cc. On this Celeron D 1.8 GHz
machine rebuilding world and kernel takes 45 minutes and 40 seconds. I
don't recall exactly what it was without ccache, but I think it was
around 3 hours. Just make sure that you set the CCACHE_HASH_COMPILER
variable, otherwise it will assume that the compiler is different just
because its modification time has changed.

- Max
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[sarek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Maxim Khitrov wrote:

> On Jan 24, 2008 4:05 PM, Erik Trulsson <ertr1013@...>
> wrote:
>> On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote:
>>> On Jan 24, 2008 3:15 PM, Aryeh M. Friedman
>>> <aryeh.friedman@...> wrote:
>>>> I update my sources at least once a day and do
>>>> buildworld/kernel just as often... It seems some stuff that
>>>> needs not be recompiled is on every single run for example
>>>> gcc and kerbos.   I have NO_CLEAN in /etc/make.conf is there
>>>> anything else I can do to speed stuff up... for ref here is
>>>> my /etc/make.conf:
>>>>
>>>> CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by
>>>> use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8
>>>> PERL_VERSION=5.8.8
>>>>
>>>> - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers
>>>> Developer, not business, friendly
>>>> http://www.flosoft-systems.com
>>> I might be wrong, but NO_CLEAN seems like a bad idea except in
>>> special circumstances. Install ccache, but make sure you set
>>> CCACHE_HASH_COMPILER environment variable to 1. That will make
>>> sure that the cache stays valid if the compiler executable is
>>> overwritten by an identical copy (as it would be on
>>> installworld). When the compiler changes the cache will be
>>> repopulated on the next rebuild.
>> You are indeed wrong.  NO_CLEAN will work fine almost all the
>> time - except in special circumstances.  The few times it does
>> not work one can always do a 'make clean' by hand first. (Or even
>> faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some
>> useful value you can do the same thing for the ports tree.
>> Personally I always compile with -DNO_CLEAN and use 'rm -fr' to
>> clean. I have never had problems originating with this.
>>
>> ccache is not very useful for buildworld, since among the first
>> thing buildworld does is to build the compiler and then use the
>> newly built compiler to compile the rest. I.e. the already
>> installed compiler (which is the one ccache will handle) will not
>> be used for most of the build thus removing almost all the
>> advantage of ccache. It is supposed to be possible to use ccache
>> for buildworld as well, but that would require a bit of hackery.
>
> That's not true. I just ran `make buildworld buildkernel` on my
> firewall. Here are ccache stats when the operation finished:
>
> root@cerberus [/root]# ccache -s cache directory
> /srv/.ccache cache hit                          12056 cache miss
> 38 called for link                      461 multiple source files
> 1 not a C/C++ file                    1228 unsupported compiler
> option            7 files in cache                    117366 cache
> size                         679.6 Mbytes max cache size
> 2.0 Gbytes
>
> Ccache is used through the entire build process and there is no
> hackery involved. Just follow the directions for changing the
> compiler to /usr/local/libexec/ccache/world-cc. On this Celeron D
> 1.8 GHz machine rebuilding world and kernel takes 45 minutes and 40
> seconds. I don't recall exactly what it was without ccache, but I
> think it was around 3 hours. Just make sure that you set the
> CCACHE_HASH_COMPILER variable, otherwise it will assume that the
> compiler is different just because its modification time has
> changed.
>
> - Max
>

I think Erik is correct here are some times (done in the order listed):

After adding NO_PROFILE to make.conf:

flosoft# cvs -q update -dP
M lib/libc/stdlib/malloc.c
flosoft# time make buildworld buildkernel installkernel installworld
.
.
.
129.160u 49.686s 6:48.67 43.7%    1001+2748k 16259+6155io 29699pf+0w

After installing ccache (first run):

flosoft# setenv CCACHE_HASH_COMPILER 1
flosoft# set CCACHE_HASH_COMPILER=1
flosoft# time make buildworld buildkernel installkernel installworld
.
.
.
117.765u 46.502s 4:56.24 55.4%    474+2667k 674+6151io 8269pf+0w
flosoft# ccache -s
cache directory                     /root/.ccache
cache hit                              0
cache miss                             0
files in cache                         0
cache size                             0 Kbytes
max cache size                     976.6 Mbytes

Second run:

flosoft# time make buildworld buildkernel installkernel installworld
.
.
.
118.318u 46.055s 4:46.64 57.3%    475+2644k 251+6145io 6203pf+0w
flosoft# !cc
ccache -s
cache directory                     /root/.ccache
cache hit                              0
cache miss                             0
files in cache                         0
cache size                             0 Kbytes
max cache size                     976.6 Mbytes

After clearing out /usr/obj (with ccache turned off):

Note: Even though I didn't time without NO_PROFILE this time I have in
the past on the same machine and got about 1 hour 5 mins (so not a big
savings)

flosoft# rm -rf /usr/objflosoft# time make buildworld buildkernel
installkernel installworld
.
.
.

2549.561u 387.975s 58:08.69 84.2%    6352+7186k 27134+14972io 11234pf+0w

Turning ccache back on (1st run):

flosoft# rm -rf /usr/obj
flosoft# setenv CCACHE_HASH_COMPILER 1
flosoft# set CCACHE_HASH_COMPILER=1
flosoft# time make buildworld buildkernel installkernel installworld
.
.
.
2556.474u 392.932s 57:21.72 85.6%    6277+7130k 24444+15046io 6414pf+0w
flosoft# ccache -s
cache directory                     /root/.ccache
cache hit                              0
cache miss                             0
files in cache                         0
cache size                             0 Kbytes
max cache size                     976.6 Mbytes

Second run:


flosoft# rm -rf /usr/obj
flosoft# time make buildworld buildkernel installkernel installworld
.
.
.
2647.683u 406.585s 1:29:41.15 56.7%    6265+7122k 18646+14931io 7350pf+0w
flosoft# ccache -s
cache directory                     /root/.ccache
cache hit                              0
cache miss                             0
files in cache                         0
cache size                             0 Kbytes
max cache size                     976.6 Mbytes

Just for sanity a final run without cleaning out /usr/obj and not
having ccache:

flosoft# time make buildworld buildkernel installkernel installworld
.
.
.
166.330u 62.414s 17:28.56 21.8%    1830+2908k 5167+6726io 9397pf+0w

Full Discloser Section:

Hardware:

e6850 CPU (dual core 3.0GHz)
p35 chipset
ihc9 disk controller
500 GB SATA/300 drive
4GB of RAM (DDR2 667)
MSI Neo-F Mobo (Default jumper settings)
nVidia 5200 FX (PCI, 256 MB)

Not over clocked.

Base system:

FreeBSD flosoft.no-ip.biz 8.0-CURRENT FreeBSD 8.0-CURRENT #4: Thu Jan
24 14:47:29 EST 2008
aryeh@...:/usr/obj/usr/src/sys/MONSTER  amd64

Also running at various times:

xfce4 (full install with dbus and hald running also)

Allways on:
firefox
thunderbird
miro
deluge (with 1 download active)
rythmbox

Used during the 1st run of cleaning out /usr/obj and ccache
open office 2

No reboots between runs and less then  1 minute pause between runs
except between the 1st and second runs after cleaning /usr/obj (i.e.
beofre and after ccache)

No commands where issued on the terminal except the ones showed and
ccache was installed with default settings (I didn't modify any files
after doing "make install" for ccache)

BTW I think the final sanity runs proves there is somekind of mem leak
in xfce or xorg

- --
Aryeh M. Friedman
FloSoft Systems, Java Tool Developers
Developer, not business, friendly
http://www.flosoft-systems.com

    "Free software != Free beer"

Blog:
 
http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHmV8ZQi2hk2LEXBARAnyNAJ94onhHiOghmgMwI1IPBAFLSlnLuACeKTgG
UE2bfmtMYpfOcSPnqxdbasQ=
=KcWD
-----END PGP SIGNATURE-----

_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Maxim Khitrov]

On Jan 24, 2008 11:01 PM, Aryeh M. Friedman <aryeh.friedman@...> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Maxim Khitrov wrote:
> > On Jan 24, 2008 4:05 PM, Erik Trulsson <ertr1013@...>
> > wrote:
> >> On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov wrote:
> >>> On Jan 24, 2008 3:15 PM, Aryeh M. Friedman
> >>> <aryeh.friedman@...> wrote:
> >>>> I update my sources at least once a day and do
> >>>> buildworld/kernel just as often... It seems some stuff that
> >>>> needs not be recompiled is on every single run for example
> >>>> gcc and kerbos.   I have NO_CLEAN in /etc/make.conf is there
> >>>> anything else I can do to speed stuff up... for ref here is
> >>>> my /etc/make.conf:
> >>>>
> >>>> CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= # added by
> >>>> use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8
> >>>> PERL_VERSION=5.8.8
> >>>>
> >>>> - -- Aryeh M. Friedman FloSoft Systems, Java Tool Developers
> >>>> Developer, not business, friendly
> >>>> http://www.flosoft-systems.com
> >>> I might be wrong, but NO_CLEAN seems like a bad idea except in
> >>> special circumstances. Install ccache, but make sure you set
> >>> CCACHE_HASH_COMPILER environment variable to 1. That will make
> >>> sure that the cache stays valid if the compiler executable is
> >>> overwritten by an identical copy (as it would be on
> >>> installworld). When the compiler changes the cache will be
> >>> repopulated on the next rebuild.
> >> You are indeed wrong.  NO_CLEAN will work fine almost all the
> >> time - except in special circumstances.  The few times it does
> >> not work one can always do a 'make clean' by hand first. (Or even
> >> faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX to some
> >> useful value you can do the same thing for the ports tree.
> >> Personally I always compile with -DNO_CLEAN and use 'rm -fr' to
> >> clean. I have never had problems originating with this.
> >>
> >> ccache is not very useful for buildworld, since among the first
> >> thing buildworld does is to build the compiler and then use the
> >> newly built compiler to compile the rest. I.e. the already
> >> installed compiler (which is the one ccache will handle) will not
> >> be used for most of the build thus removing almost all the
> >> advantage of ccache. It is supposed to be possible to use ccache
> >> for buildworld as well, but that would require a bit of hackery.
> >
> > That's not true. I just ran `make buildworld buildkernel` on my
> > firewall. Here are ccache stats when the operation finished:
> >
> > root@cerberus [/root]# ccache -s cache directory
> > /srv/.ccache cache hit                          12056 cache miss
> > 38 called for link                      461 multiple source files
> > 1 not a C/C++ file                    1228 unsupported compiler
> > option            7 files in cache                    117366 cache
> > size                         679.6 Mbytes max cache size
> > 2.0 Gbytes
> >
> > Ccache is used through the entire build process and there is no
> > hackery involved. Just follow the directions for changing the
> > compiler to /usr/local/libexec/ccache/world-cc. On this Celeron D
> > 1.8 GHz machine rebuilding world and kernel takes 45 minutes and 40
> > seconds. I don't recall exactly what it was without ccache, but I
> > think it was around 3 hours. Just make sure that you set the
> > CCACHE_HASH_COMPILER variable, otherwise it will assume that the
> > compiler is different just because its modification time has
> > changed.
> >
> > - Max
> >
> No commands where issued on the terminal except the ones showed and
> ccache was installed with default settings (I didn't modify any files
> after doing "make install" for ccache)

It doesn't work like that. You have to read
/usr/local/share/doc/ccache/ccache-howto-freebsd.txt and configure
things properly before ccache is used for building the os.

In /etc/make.conf you need to add the following:

.if exists(/usr/local/libexec/ccache) && !defined(NOCCACHE) && \
    (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*))
CC=  /usr/local/libexec/ccache/world-cc
CXX= /usr/local/libexec/ccache/world-c++
.endif

Unless you actually want to use /root/.ccache (and have enough space
for it), I would move that directory to some other partition. In my
/etc/csh.cshrc I have this configuration:

setenv CCACHE_DIR           /srv/.ccache
setenv CCACHE_PATH          /usr/bin:/usr/local/bin
setenv CCACHE_HASH_COMPILER 1

Once you've done all of this, rebuild the world. You can run ccache -s
during that process. If the hit/miss numbers stay at 0 then ccache is
not being used. Check your configuration and try again.

The first run will be slower than normal (though not by much), because
the cache is being populated for the first time. On the second run,
however, clear ccache stats (ccache -z) and you should see the same
results as I've posted above. With the exception of only a few files,
just about everything should be obtained from the cache and not
compiled from scratch.

- Max
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[sarek]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Maxim Khitrov wrote:

> On Jan 24, 2008 11:01 PM, Aryeh M. Friedman
> <aryeh.friedman@...> wrote:
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>
>>
>> Maxim Khitrov wrote:
>>> On Jan 24, 2008 4:05 PM, Erik Trulsson <ertr1013@...>
>>>  wrote:
>>>> On Thu, Jan 24, 2008 at 03:32:18PM -0500, Maxim Khitrov
>>>> wrote:
>>>>> On Jan 24, 2008 3:15 PM, Aryeh M. Friedman
>>>>> <aryeh.friedman@...> wrote:
>>>>>> I update my sources at least once a day and do
>>>>>> buildworld/kernel just as often... It seems some stuff
>>>>>> that needs not be recompiled is on every single run for
>>>>>> example gcc and kerbos.   I have NO_CLEAN in
>>>>>> /etc/make.conf is there anything else I can do to speed
>>>>>> stuff up... for ref here is my /etc/make.conf:
>>>>>>
>>>>>> CPUTYPE?=nocona KERNCONF=MONSTER NO_CLEAN= NO_LPR= #
>>>>>> added by use.perl 2008-01-17 11:48:48 PERL_VER=5.8.8
>>>>>> PERL_VERSION=5.8.8
>>>>>>
>>>>>> - -- Aryeh M. Friedman FloSoft Systems, Java Tool
>>>>>> Developers Developer, not business, friendly
>>>>>> http://www.flosoft-systems.com
>>>>> I might be wrong, but NO_CLEAN seems like a bad idea except
>>>>> in special circumstances. Install ccache, but make sure you
>>>>> set CCACHE_HASH_COMPILER environment variable to 1. That
>>>>> will make sure that the cache stays valid if the compiler
>>>>> executable is overwritten by an identical copy (as it would
>>>>> be on installworld). When the compiler changes the cache
>>>>> will be repopulated on the next rebuild.
>>>> You are indeed wrong.  NO_CLEAN will work fine almost all the
>>>>  time - except in special circumstances.  The few times it
>>>> does not work one can always do a 'make clean' by hand first.
>>>> (Or even faster: 'rm -fr /usr/obj/*') If you set WRKDIRPREFIX
>>>> to some useful value you can do the same thing for the ports
>>>> tree. Personally I always compile with -DNO_CLEAN and use 'rm
>>>> -fr' to clean. I have never had problems originating with
>>>> this.
>>>>
>>>> ccache is not very useful for buildworld, since among the
>>>> first thing buildworld does is to build the compiler and then
>>>> use the newly built compiler to compile the rest. I.e. the
>>>> already installed compiler (which is the one ccache will
>>>> handle) will not be used for most of the build thus removing
>>>> almost all the advantage of ccache. It is supposed to be
>>>> possible to use ccache for buildworld as well, but that would
>>>> require a bit of hackery.
>>> That's not true. I just ran `make buildworld buildkernel` on my
>>>  firewall. Here are ccache stats when the operation finished:
>>>
>>> root@cerberus [/root]# ccache -s cache directory /srv/.ccache
>>> cache hit                          12056 cache miss 38 called
>>> for link                      461 multiple source files 1 not a
>>> C/C++ file                    1228 unsupported compiler option
>>> 7 files in cache                    117366 cache size
>>> 679.6 Mbytes max cache size 2.0 Gbytes
>>>
>>> Ccache is used through the entire build process and there is no
>>>  hackery involved. Just follow the directions for changing the
>>> compiler to /usr/local/libexec/ccache/world-cc. On this Celeron
>>> D 1.8 GHz machine rebuilding world and kernel takes 45 minutes
>>> and 40 seconds. I don't recall exactly what it was without
>>> ccache, but I think it was around 3 hours. Just make sure that
>>> you set the CCACHE_HASH_COMPILER variable, otherwise it will
>>> assume that the compiler is different just because its
>>> modification time has changed.
>>>
>>> - Max
>>>
>> No commands where issued on the terminal except the ones showed
>> and ccache was installed with default settings (I didn't modify
>> any files after doing "make install" for ccache)
>
> It doesn't work like that. You have to read
> /usr/local/share/doc/ccache/ccache-howto-freebsd.txt and configure
> things properly before ccache is used for building the os.
>
> In /etc/make.conf you need to add the following:
>
> .if exists(/usr/local/libexec/ccache) && !defined(NOCCACHE) && \
> (!empty(.CURDIR:M/usr/src*) || !empty(.CURDIR:M/usr/obj*)) CC=
> /usr/local/libexec/ccache/world-cc CXX=
> /usr/local/libexec/ccache/world-c++ .endif
>
> Unless you actually want to use /root/.ccache (and have enough
> space for it), I would move that directory to some other partition.
> In my /etc/csh.cshrc I have this configuration:
>
> setenv CCACHE_DIR           /srv/.ccache setenv CCACHE_PATH
> /usr/bin:/usr/local/bin setenv CCACHE_HASH_COMPILER 1
>
> Once you've done all of this, rebuild the world. You can run ccache
> -s during that process. If the hit/miss numbers stay at 0 then
> ccache is not being used. Check your configuration and try again.
>
> The first run will be slower than normal (though not by much),
> because the cache is being populated for the first time. On the
> second run, however, clear ccache stats (ccache -z) and you should
> see the same results as I've posted above. With the exception of
> only a few files, just about everything should be obtained from the
> cache and not compiled from scratch.
>
> - Max
>

Here are the results:

First run (wo/ removing /usr/obj) using the above settings:
119.025u 46.448s 8:11.19 33.6%    487+2711k 15027+6126io 28785pf+0w
flosoft# ccache -s
cache directory                     /usr/.ccache
cache hit                              1
cache miss                             3
called for link                        3
unsupported compiler option            1
files in cache                         6
cache size                           188 Kbytes
max cache size                     976.6 Mbytes

Second run:

I miscopied and pasted before I relized it was wrong and had already
started the next run but from memory:

6:?? minutes (< 6:30)
5 cache hits

I cleaned out /usr/obj /usr/.ccache (where I store the cache) for the
next run:



it crashed:

===> lib/csu/i386-elf (obj,depend,all,install)
rm -f .depend
CC='/usr/local/libexec/ccache/world-cc' mkdep -f .depend -a
- -I/usr/src/lib/csu/i386-elf/../common
- -I/usr/src/lib/csu/i386-elf/../../libc/include
/usr/src/lib/csu/i386-elf/crt1.c /usr/src/lib/csu/i386-elf/crti.S
/usr/src/lib/csu/i386-elf/crtn.S
/usr/local/libexec/ccache/world-cc -O2 -fno-strict-aliasing -pipe
- -I/usr/src/lib/csu/i386-elf/../common
- -I/usr/src/lib/csu/i386-elf/../../libc/include -Wsystem-headers
- -Werror -Wall -Wno-format-y2k -W -Wno-unused-parameter
- -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wreturn-type
- -Wcast-qual -Wwrite-strings -Wswitch -Wshadow -Wcast-align
- -Wunused-parameter -Wchar-subscripts -Winline -Wnested-externs
- -Wredundant-decls -Wno-pointer-sign -c /usr/src/lib/csu/i386-elf/crt1.c
{standard input}: Assembler messages:
{standard input}:27: Error: suffix or operands invalid for `mov'
*** Error code 1

Stop in /usr/src/lib/csu/i386-elf.
*** Error code 1

Stop in /usr/src.
*** Error code 1

Stop in /usr/src.
*** Error code 1

Stop in /usr/src.
*** Error code 1

Stop in /usr/src.
1713.072u 279.765s 58:14.38 57.0%    6179+8423k 22605+12323io 8667pf+0w

- --
Aryeh M. Friedman
FloSoft Systems, Java Tool Developers
Developer, not business, friendly
http://www.flosoft-systems.com

    "Free software != Free beer"

Blog:
 
http://www.flosoft-systems.com/flosoft_systems_community/blogs/aryeh/index.php
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4 (FreeBSD)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHmXvgQi2hk2LEXBARAohlAJ4kFVb/o7pHN2Q551n4iDMFhpprGwCbBkUe
p5hwkE6sGOkpYEP7Taq+tM0=
=llvM
-----END PGP SIGNATURE-----

_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Brian-207]

Hmm, 2 queries here.

1-wonder how much more gain would be gotten by using a speedy flash
drive for the ccache folder.

2-I'm wondering about dependencies, like a change in x requires a
recompile of y, but y doesnt look any different, is this smart enough to
rebuild based on the dependency?

Brian
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Maxim Khitrov]

On Jan 25, 2008 1:50 AM, Brian <bri@...> wrote:
> Hmm, 2 queries here.
>
> 1-wonder how much more gain would be gotten by using a speedy flash
> drive for the ccache folder.

Actually you get the opposite. Here are my results with a USB 2.0 flash drive:

cache directory                     /mnt/.ccache
cache hit                          12106
cache miss                            12
called for link                      461
multiple source files                  1
not a C/C++ file                    1228
unsupported compiler option            7
files in cache                    122144
cache size                         820.2 Mbytes
max cache size                       2.0 Gbytes

2h7m4.56s real          31m36.79s user          15m31.80s sys

For reference, I've again rebuilt world and kernel. Ccache stats were
the same as for the flash drive, but here's the time:

47m26.34s real          27m16.22s user          13m45.71s sys

Flash drive is better than nothing at all, but much worse than using a
hard drive.

> 2-I'm wondering about dependencies, like a change in x requires a
> recompile of y, but y doesnt look any different, is this smart enough to
> rebuild based on the dependency?
>
> Brian

What do you mean by "y doesn't look any different"? If recompiling y
results in the same object file, then there is no need to recompile
it. That's all that ccache does. It considers all the variables that
can possibly affect the contents of an object file. If those variables
are the same as from a previously-cached run, then it returns the
precompiled version of the file. More info is available at the ccache
website: http://ccache.samba.org/

- Max
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."

[Brian-207]

I have started using ccache as a result of this thread, the speed
benefit for old hw is nice.

p3-850 Cele
This would be 3-4 hours for a buildworld and a make kernel, now it is
3234.877u 2318.444s 1:44:56.27 88.1%    2580+1569k 79615+15476io 10553pf+0w

K6-2 450
This would be 8-9 hours for a buildworld and a make kernel, now it is
5835.095u 3103.667s 3:29:42.86 71.0%    -2168+1636k 140556+17495io
15489pf+0w

This definitely makes it more feasible to use older hardware longer.

Brian
_______________________________________________
freebsd-questions@... mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@..."