|
»
»
how to include dependencies only if includes changed
|
View:
New views
7 Messages
—
Rating Filter:
Alert me
|
 |
|
|
|
Re: how to include dependencies only if includes changed
by Pete Johnson
::
Rate this Message:
Mark,
See http://make.paulandlesley.org/autodep.html for a good way to handle auto dependencies in make. Using the methods described in the article, you will only build dependencies when they are needed. -Pete On 11/8/09, Mark Galeck (CW) <mgaleck@...> wrote: Hello, _______________________________________________ Help-make mailing list Help-make@... http://lists.gnu.org/mailman/listinfo/help-make |
|
|
RE: how to include dependencies only if includes changed
by Mark Galeck (CW)
::
Rate this Message:
> See http://make.paulandlesley.org/autodep.html for a good way to handle auto dependencies in make. Using the methods described in the article, you will only build dependencies when they are needed.
Oh yes, Pete, I am very much aware of this article. In fact, I combined ideas from that article, with the auto-dep section from GNU make manual, to implement the auto-dependencies. The problem is, and I believe neither that article nor make manual address it, is that I don't want to "-include" all the dependencies unless necessary. This is because I am dealing with a code base of over 8 million lines, more than 1000 .c files, each depending on around 500 .h files. The dependencies, all of them, take about 1 minute to "-include", even if you are just rebuilding one .o file. It is OK with me, but other developers would not be happy about it. I did finally come up with some solution, let me write it here, maybe someone can comment if there is a better way. The target chooseIncl, is OS dependent: it decides whether any included files have changed since last time, and based on that invokes $(MAKE) incl or $(MAKE) noIncl -------------------- Main makefile (relevant fragments) ------------------------ -include included .PHONY: all incl noIncl all: mainProgram incl noIncl: all #goals that do not need any .d makefiles GOALS_REQ_NO_INCLUDE = noIncl lastIncl clean #goals that require some .d makefiles GOALS_REQ_INCLUDE = $(filter-out $(GOALS_REQ_NO_INCLUDE), $(MAKECMDGOALS)) #goals that require all .d makefiles GOALS_REQ_ALL_INCLUDE = $(filter-out %.o, $( GOALS_REQ_INCLUDE)) ifneq (, $( GOALS_REQ_ALL_INCLUDE)) -include $(OBJECTS:.o=.d) else -include $(GOALS_REQ_INCLUDE:.o=.d) Endif -------------------------------- "included" makefile,generated automatically by a Perl script when mainProgram is rebuilt (relevant fragments) ------------------------------- chooseIncl: ($(MAKE) -q lastIncl && ( $(MAKE) noIncl & touch lastIncl)) || ($(MAKE) incl & touch lastIncl) lastIncl:\ foobar.h\ foobar1.h\ (... all the header files) _______________________________________________ Help-make mailing list Help-make@... http://lists.gnu.org/mailman/listinfo/help-make |
|
|
Re: how to include dependencies only if includes changed
by Allan Odgaard-8
::
Rate this Message:
On 9 Nov 2009, at 19:25, Mark Galeck (CW) wrote:
>> [...] I am dealing with a code base of over 8 million lines, more >> than 1000 .c files, each depending on around 500 .h files. The >> dependencies, all of them, take about 1 minute to "-include", even >> if you are just rebuilding one .o file. The time should be relative to number of total .h files (and I guess number of included files). I have a much smaller project (~200 sources + ~250 headers) but I noticed that the implicit rules of make can really slow things down: % time make make: Nothing to be done for `all'. real 0m0.318s % time make -r make: Nothing to be done for `all'. real 0m0.090s Don’t know if you rely on the implicit rules, if not, definitely test how much time they eat. If you do use them, you may be able to just copy those you use and disable the rest. _______________________________________________ Help-make mailing list Help-make@... http://lists.gnu.org/mailman/listinfo/help-make |
|
|
RE: how to include dependencies only if includes changed
by Mark Galeck (CW)
::
Rate this Message:
>The time should be relative to number of total .h files (and I guess
number of included files). Hmm, from what I see, what takes a long time, is all the lines from all the .o files, to read in all this stuff, is more that a million lines, then I guess make checks if any of those lines have anything to do with your foobar.o target, and finds out most of them they do not. Anyway, it seems most of the time is just reading all the files. >I have a much smaller project (~200 sources + ~250 headers) but I noticed that the implicit rules of make can really slow things down: Oh thank you very much, I definitely will take a look and follow your suggestion if I can! _______________________________________________ Help-make mailing list Help-make@... http://lists.gnu.org/mailman/listinfo/help-make |
|
|
Re: how to include dependencies only if includes changed
by Bart Robinson-2
::
Rate this Message:
On Mon, Nov 9, 2009 at 10:02 AM, Pete Johnson <pete@...> wrote:
> Mark, > > See http://make.paulandlesley.org/autodep.html for a good way to handle auto > dependencies in make. Using the methods described in the article, you will > only build dependencies when they are needed. > > -Pete To save time for anyone looking to do this (the auto-dependencies, not Mark's auto-include-auto-dependencies), I just wanted to comment that the mechanisms in that article are somewhat dated and it is a little easier to do this now with the gcc -MP flag, which does what I think is the equivalent of the hairy sed transforms. If using fairly recent gcc (i can vouch for 3.4.6+), skip to the -MD part (I suggest -MMD, which omits stdio.h and the like) and also use -MP: srcs := src/foo.c stuff/bar.c ... %.o : %.c $(CC) -o $@ -c -MMD -MP -MF $*.d $(CFLAGS) $< depfiles := $(patsubst %.c,%.d,$(sort $(srcs))) ifneq ($(depfiles),) $(depfiles): ; -include $(depfiles) endif The -MF <file> part is only for older distcc (they symptom indicating this is needed is if all your .d files end up at top-level, e.g. ./foo.d rather than src/foo.d, and the include doesn't find them). Newer distcc (somewhere in 3.x) has reportedly fixed this. The sort is only to remove duplicates, which I remember having errors for years ago. Might not be necessary anymore (so my example may also be somewhat dated :-). The $(depfiles): ; is to speed up make by avoiding the implicit rule search for these. This puts them alongside the .o files, which may be annoying if building within the srcdir. If that is bothersome then (for several other reasons as well) I would suggest judicious use of vpath and a $(srcdir) notion to enable building in a separate dir. -- bart > On 11/8/09, Mark Galeck (CW) <mgaleck@...> wrote: >> >> Hello, >> >> Suppose for each .c file such as foobar.c I have generated a makefile >> foobar.d file, that lists dependencies of foobar.o on files included in >> foobar.c. >> >> There are many .d files and each of them has many dependencies, so if I >> just blindly do >> >> -include $(OBJS:.o=.d) >> >> then it takes a lot of time for make to just get started. What I want, is >> to only include all of those *.d when needed, that is, when some include >> file has been modified. I want this whenever a user calls >> >> >make target >> >> for any target. How to do this? >> -------------------------- >> >> What follows below is my solution to this problem, but it only works for >> >make >> (default target) >> and can be extended to any fixed number of other targets, but not to >> general target such as >> >make foobar.o >> where foobar.c is any .c file >> >> >> The solution, >> simplified somewhat to show you just the gist of it, >> is to generate another makefile called "included" like this: >> >> chooseIncl: >> ($(MAKE) -q lastIncl && ( $(MAKE) noIncl & touch lastIncl)) || >> ($(MAKE) incl & touch lastIncl) >> >> lastIncl:\ >> foobar.h\ >> foobar1.h\ >> (...) >> >> >> (lastIncl depends on all the included files). >> If you don't know DOS, the way the rule for chooseIncl works, is that if >> any included files have changed, then >> $(MAKE) incl >> is called, otherwise >> $(MAKE) noIncl >> is called >> >> >> then include "included" at the top of the main makefile, >> and put in the main makefile (where "all" is the original default target): >> >> incl noIncl: all >> >> ifeq (incl, $(MAKECMDGOALS)) >> -include $(ALL_OBJS:.o=.d) >> endif >> >> >> >> >> >> _______________________________________________ >> Help-make mailing list >> Help-make@... >> http://lists.gnu.org/mailman/listinfo/help-make > > > _______________________________________________ > Help-make mailing list > Help-make@... > http://lists.gnu.org/mailman/listinfo/help-make > > _______________________________________________ Help-make mailing list Help-make@... http://lists.gnu.org/mailman/listinfo/help-make |
|
|
Re: how to include dependencies only if includes changed
by Bart Robinson-2
::
Rate this Message:
On Mon, Nov 9, 2009 at 7:23 PM, Mark Galeck (CW) <mgaleck@...> wrote:
>> The time should be relative to number of total .h files (and I >> guess number of included files). > > Hmm, from what I see, what takes a long time, is all the lines from > all the .o files, to read in all this stuff, is more that a million > lines, then I guess make checks if any of those lines have anything > to do with your foobar.o target, and finds out most of them they do > not. Anyway, it seems most of the time is just reading all the > files. > >>I have a much smaller project (~200 sources + ~250 headers) but I >>noticed that the implicit rules of make can really slow things down: > > Oh thank you very much, I definitely will take a look and follow > your suggestion if I can! FWIW, I recently went thru this slimming exercise and here is what I'm using to get rid of most of the implicit rules (using make -r is not a option, and of course it isn't possible to set the -r'ness in a makefile itself, since the implicits are loaded before reading your makefile): 1. make your dependencies have empty commands, to avoid searching implicit rules $(all-your-dependency-.d-files): ; 2. add this to your makefile to neuter most of the built-ins .SUFFIXES: %:: %,v %:: RCS/% %:: RCS/%,v %:: s.% %:: SCCS/s.% %.c: %.w %.ch -- bart _______________________________________________ Help-make mailing list Help-make@... http://lists.gnu.org/mailman/listinfo/help-make |
| Free embeddable forum powered by Nabble | Forum Help |
