Build Safety

[Dalton, Tom]

Hi,

I think I know the answer to this question, but I want a second opinion!

In the past, I have used the Clearcase version control system, which has
it's own make system, clearmake. Clearmake provides some build caching
functionality (which it terms wink-ins), that allows it to 'wink-in'
build objects from other users views. I have experienced a lot of
problems with this system, because clearcase wouldn't detect that two
users were building the same file in a different environment (e.g.
external libraries at different versions. This caused me all sorts of
issues.

Is it possible for inconsistent environments between builds to cause
ccache to incorrectly use a cached object instead of recompiling? For
example, different builds of gcc, different external libraries etc? What
circumstances could cause this?

Secondly, if there is not much that can break ccache, is it possible to
use a shared cache (e.g. on an nfs-shared disk) to allow many users on
many hosts to cache build results across several versions of a single
product? (which will mean different gcc versions, libraries, even
kernels.)

Thanks for your help - ccache sounds really useful, but I want to make
sure there's no 'traps'!

Thanks,

Tom
_______________________________________________
ccache mailing list
ccache@...
https://lists.samba.org/mailman/listinfo/ccache

[yince]

Hi, and welcome to ccache!

> Date: Fri, 25 Sep 2009 10:22:14 +0200
> From: tom.dalton@...
> To: ccache@...
> Subject: [ccache] Build Safety
>
> Hi,
>
> I think I know the answer to this question, but I want a second opinion!
>
> In the past, I have used the Clearcase version control system, which has
> it's own make system, clearmake. Clearmake provides some build caching
> functionality (which it terms wink-ins), that allows it to 'wink-in'
> build objects from other users views. I have experienced a lot of
> problems with this system, because clearcase wouldn't detect that two
> users were building the same file in a different environment (e.g.
> external libraries at different versions. This caused me all sorts of
> issues.

That's why it is not very used.

> Is it possible for inconsistent environments between builds to cause
> ccache to incorrectly use a cached object instead of recompiling? For
> example, different builds of gcc, different external libraries etc? What
> circumstances could cause this?

for each compile ccache makes a key with the following information:1. the pre-processor output from running the compiler with -E
2. the command line options
3. the real compilers size and modification time
4. any stderr output generated by the compilerif something changes then it recompiles and recache the new key/.o
so if you use a different external librairies, #1 and maybe #2 can change.

> Secondly, if there is not much that can break ccache, is it possible to
> use a shared cache (e.g. on an nfs-shared disk) to allow many users on
> many hosts to cache build results across several versions of a single
> product? (which will mean different gcc versions, libraries, even
> kernels.)

if each users use different gcc versions, libraries, even kernels, then
sharing the cache is not useful. It's better to use it locally. Try to compile
your product from scratch witch ccache and recompile (clean before) and
compare the time. (ie: time make all). Here is my current stats at this moment:

cache directory                     /home/vince/projects/raaf/ccache
cache hit                          16462
cache miss                         11307
called for link                       70
compile failed                        63
preprocessor error                     4
not a C/C++ file                    1198
files in cache                      8779
cache size                         793.5 Mbytes
max cache size                     976.6 Mbytes

like you see, I skip compilation of 16462 files locally.

> Thanks for your help - ccache sounds really useful, but I want to make
> sure there's no 'traps'!
>
> Thanks,
>
> Tom
> _______________________________________________
> ccache mailing list
> ccache@...
> https://lists.samba.org/mailman/listinfo/ccache

     
_________________________________________________________________
Internet explorer 8 aide à protéger la vie privée.
http://go.microsoft.com/?linkid=9655573
_______________________________________________
ccache mailing list
ccache@...
https://lists.samba.org/mailman/listinfo/ccache

[Dalton, Tom]

Thanks Vincent, that is very useful. I have one more clarification on
why I think using a shared cache will be useful (if it works!)
 
We have several sub-projects, A, B, C, D.
Our developers (and their development/build machines) can move around
between the sub-projects.
 
 
Subproject A and B 1 use gcc3, subproject C and D use gcc4.
Subproject A uses libraryX v1, subprojects B,C,D use libraryX v2.
 
So effectively there are 3 configurations, (gcc3 libX1), (gcc3 libX2),
(gcc4 libX2).
 
Now if each developer has a local cache, then each developer needs to do
each compile at least once, before the cache is populated. But if each
subproject has 10 developers, and the cache is shared, then only 1
developer needs to build the subproject, and then all the other
developers will get lots of cache hits when they compile.
 
From my point of view, it is easier to configure a single, large shared
cache across all developers than to set up a single cache for each
subproject team. Also, in the above example, subprojects C and D have
the same configuration and so could actually use the same cache.
 
My issue is that the real scenario is much more complex, with each
subproject being comprised many components, which in turn are components
etc. So identifying the configurations is not trivial. I suspect that in
reality, each project has some compilation in common with some of the
other projects, but it's hard to tell which. So if ccache can cope with
using a network-shared cache, then having one single (large) cache would
give maximum efficiency across all the projects, as any
cache-partitioning I impose (per-project or per-developer or
per-machine).
 
I hope that makes sense, but I don't know how ccache actually reads the
cache. In effect, is it safe to use the same shared cache on multiple
machines? If not, then is it safe to run a parallel build (e.g. make -j
2) on a local machine with a local cache, as presumably that has the
same implications wrt. multiple concurrent cache updates.
 
Thanks for your help!
 
Tom


________________________________

        From: Vincent Dupuis [mailto:vincedupuis@...]
        Sent: 25 September 2009 13:18
        To: Dalton, Tom
        Cc: ccache@...
        Subject: RE: [ccache] Build Safety
       
       
        Hi, and welcome to ccache!
       
        > Date: Fri, 25 Sep 2009 10:22:14 +0200
        > From: tom.dalton@...
        > To: ccache@...
        > Subject: [ccache] Build Safety
        >
        > Hi,
        >
        > I think I know the answer to this question, but I want a
second opinion!
        >
        > In the past, I have used the Clearcase version control system,
which has
        > it's own make system, clearmake. Clearmake provides some build
caching
        > functionality (which it terms wink-ins), that allows it to
'wink-in'
        > build objects from other users views. I have experienced a lot
of
        > problems with this system, because clearcase wouldn't detect
that two
        > users were building the same file in a different environment
(e.g.
        > external libraries at different versions. This caused me all
sorts of
        > issues.
       
        That's why it is not very used.
       
        > Is it possible for inconsistent environments between builds to
cause
        > ccache to incorrectly use a cached object instead of
recompiling? For
        > example, different builds of gcc, different external libraries
etc? What
        > circumstances could cause this?
       
        for each compile ccache makes a key with the following
information:

                1. the pre-processor output from running the compiler
with -E
                2. the command line options
                3. the real compilers size and modification time
                4. any stderr output generated by the compiler

        if something changes then it recompiles and recache the new
key/.o
        so if you use a different external librairies, #1 and maybe #2
can change.
       
        > Secondly, if there is not much that can break ccache, is it
possible to
        > use a shared cache (e.g. on an nfs-shared disk) to allow many
users on
        > many hosts to cache build results across several versions of a
single
        > product? (which will mean different gcc versions, libraries,
even
        > kernels.)
       
        if each users use different gcc versions, libraries, even
kernels, then
        sharing the cache is not useful. It's better to use it locally.
Try to compile
        your product from scratch witch ccache and recompile (clean
before) and
        compare the time. (ie: time make all). Here is my current stats
at this moment:
       
        cache directory
/home/vince/projects/raaf/ccache
        cache hit                          16462
        cache miss                         11307
        called for link                       70
        compile failed                        63
        preprocessor error                     4
        not a C/C++ file                    1198
        files in cache                      8779
        cache size                         793.5 Mbytes
        max cache size                     976.6 Mbytes
       
        like you see, I skip compilation of 16462 files locally.
       
       
        > Thanks for your help - ccache sounds really useful, but I want
to make
        > sure there's no 'traps'!
        >
        > Thanks,
        >
        > Tom
        > _______________________________________________
        > ccache mailing list
        > ccache@...
        > https://lists.samba.org/mailman/listinfo/ccache
       
       
________________________________

        Naviguez plus vite avec Internet Explorer 8 Ici.
<http://go.microsoft.com/?linkid=9655263>  

_______________________________________________
ccache mailing list
ccache@...
https://lists.samba.org/mailman/listinfo/ccache

[Dalton, Tom]

Thanks!
 
I also realised that each developer builds their own compiler from source, so the modification date will also be different. However, I can think of ways that this can be solved, like using relative paths for the includes. I think that initially though, we will get a nice improvement from using a local cache for each machine :-)
 
Cheers for your help!
 
Tom


________________________________

        From: Vincent Dupuis [mailto:vincedupuis@...]
        Sent: 25 September 2009 14:50
        To: Dalton, Tom
        Subject: RE: [ccache] Build Safety
       
       
        Tom,
       
        you are right. ccache is independent of your project/component setup.
        it is source file (compile) based.
       
        for each file compiled
        build a key (see below 1-4)
        if the key is found return the .o
        else add the new key and the new .o in the cache
       
        Remember that if the file is exactly the same but the compiler option
        are not (#2) like the -I option (include path). It will not retrieve the cached file.
        So your makefile needs to use relative path or all user need to use the same include paths.
       
        V
       
       
        > Date: Fri, 25 Sep 2009 14:56:48 +0200
        > From: tom.dalton@...
        > To: ccache@...
        > Subject: Re: [ccache] Build Safety
        >
        > Thanks Vincent, that is very useful. I have one more clarification on
        > why I think using a shared cache will be useful (if it works!)
        >
        > We have several sub-projects, A, B, C, D.
        > Our developers (and their development/build machines) can move around
        > between the sub-projects.
        >
        >
        > Subproject A and B 1 use gcc3, subproject C and D use gcc4.
        > Subproject A uses libraryX v1, subprojects B,C,D use libraryX v2.
        >
        > So effectively there are 3 configurations, (gcc3 libX1), (gcc3 libX2),
        > (gcc4 libX2).
        >
        > Now if each developer has a local cache, then each developer needs to do
        > each compile at least once, before the cache is populated. But if each
        > subproject has 10 developers, and the cache is shared, then only 1
        > developer needs to build the subproject, and then all the other
        > developers will get lots of cache hits when they compile.
        >
        > From my point of view, it is easier to configure a single, large shared
        > cache across all developers than to set up a single cache for each
        > subproject team. Also, in the above example, subprojects C and D have
        > the same configuration and so could actually use the same cache.
        >
        > My issue is that the real scenario is much more complex, with each
        > subproject being comprised many components, which in turn are components
        > etc. So identifying the configurations is not trivial. I suspect that in
        > reality, each project has some compilation in common with some of the
        > other projects, but it's hard to tell which. So if ccache can cope with
        > using a network-shared cache, then having one single (large) cache would
        > give maximum efficiency across all the projects, as any
        > cache-partitioning I impose (per-project or per-developer or
        > per-machine).
        >
        > I hope that makes sense, but I don't know how ccache actually reads the
        > cache. In effect, is it safe to use the same shared cache on multiple
        > machines? If not, then is it safe to run a parallel build (e.g. make -j
        > 2) on a local machine with a local cache, as presumably that has the
        > same implications wrt. multiple concurrent cache updates.
        >
        > Thanks for your help!
        >
        > Tom
        >
        >
        > ________________________________
        >
        > From: Vincent Dupuis [mailto:vincedupuis@...]
        > Sent: 25 September 2009 13:18
        > To: Dalton, Tom
        > Cc: ccache@...
        > Subject: RE: [ccache] Build Safety
        >
        >
        > Hi, and welcome to ccache!
        >
        > > Date: Fri, 25 Sep 2009 10:22:14 +0200
        > > From: tom.dalton@...
        > > To: ccache@...
        > > Subject: [ccache] Build Safety
        > >
        > > Hi,
        > >
        > > I think I know the answer to this question, but I want a
        > second opinion!
        > >
        > > In the past, I have used the Clearcase version control system,
        > which has
        > > it's own make system, clearmake. Clearmake provides some build
        > caching
        > > functionality (which it terms wink-ins), that allows it to
        > 'wink-in'
        > > build objects from other users views. I have experienced a lot
        > of
        > > problems with this system, because clearcase wouldn't detect
        > that two
        > > users were building the same file in a different environment
        > (e.g.
        > > external libraries at different versions. This caused me all
        > sorts of
        > > issues.
        >
        > That's why it is not very used.
        >
        > > Is it possible for inconsistent environments between builds to
        > cause
        > > ccache to incorrectly use a cached object instead of
        > recompiling? For
        > > example, different builds of gcc, different external libraries
        > etc? What
        > > circumstances could cause this?
        >
        > for each compile ccache makes a key with the following
        > information:
        >
        > 1. the pre-processor output from running the compiler
        > with -E
        > 2. the command line options
        > 3. the real compilers size and modification time
        > 4. any stderr output generated by the compiler
        >
        > if something changes then it recompiles and recache the new
        > key/.o
        > so if you use a different external librairies, #1 and maybe #2
        > can change.
        >
        > > Secondly, if there is not much that can break ccache, is it
        > possible to
        > > use a shared cache (e.g. on an nfs-shared disk) to allow many
        > users on
        > > many hosts to cache build results across several versions of a
        > single
        > > product? (which will mean different gcc versions, libraries,
        > even
        > > kernels.)
        >
        > if each users use different gcc versions, libraries, even
        > kernels, then
        > sharing the cache is not useful. It's better to use it locally.
        > Try to compile
        > your product from scratch witch ccache and recompile (clean
        > before) and
        > compare the time. (ie: time make all). Here is my current stats
        > at this moment:
        >
        > cache directory
        > /home/vince/projects/raaf/ccache
        > cache hit 16462
        > cache miss 11307
        > called for link 70
        > compile failed 63
        > preprocessor error 4
        > not a C/C++ file 1198
        > files in cache 8779
        > cache size 793.5 Mbytes
        > max cache size 976.6 Mbytes
        >
        > like you see, I skip compilation of 16462 files locally.
        >
        >
        > > Thanks for your help - ccache sounds really useful, but I want
        > to make
        > > sure there's no 'traps'!
        > >
        > > Thanks,
        > >
        > > Tom
        > > _______________________________________________
        > > ccache mailing list
        > > ccache@...
        > > https://lists.samba.org/mailman/listinfo/ccache
        >
        >
        > ________________________________
        >
        > Naviguez plus vite avec Internet Explorer 8 Ici.
        > <http://go.microsoft.com/?linkid=9655263>
        >
        > _______________________________________________
        > ccache mailing list
        > ccache@...
        > https://lists.samba.org/mailman/listinfo/ccache
       
       
________________________________

        Avec Windows Live, vous gardez le contact avec tous vos amis au même endroit. <http://go.microsoft.com/?linkid=9660828>  

_______________________________________________
ccache mailing list
ccache@...
https://lists.samba.org/mailman/listinfo/ccache