When we refactored our word counting program to use header files, a thorny little problem snuck up on us. We added the dependency between the object files and the C header files to the makefile by hand. It was easy to do in this case, but in normal programs (not toy examples) this can be tedious and error-prone . In fact, in most programs it is virtually impossible because most header files include other header files forming a complex tree. For instance, on my system, the single header file stdio.h (the most commonly referenced header file in C) expands to include 15 other header files. Resolving these relationships by hand is a hopeless task. But failing to recompile files can lead to hours of debugging headaches or worse , bugs in the resulting program. So what do we do?
Well, computers are pretty good at searching and pattern matching. Let's use a program to identify the relationships between files and maybe even have this program write out these dependencies in makefile syntax. As you have probably guessed, such a program already exists ”at least for C/C++. There is a option to gcc and many other C/C++ compilers that will read the source and write makefile dependencies. For instance, here is how I found the dependencies for stdio.h :
$ echo "#include <stdio.h>" > stdio.c $ gcc -M stdio.c stdio.o: stdio.c /usr/include/stdio.h /usr/include/_ansi.h \ /usr/include/newlib.h /usr/include/sys/config.h \ /usr/include/machine/ieeefp.h /usr/include/cygwin/config.h \ /usr/lib/gcc-lib/i686-pc-cygwin/3.2/include/stddef.h \ /usr/lib/gcc-lib/i686-pc-cygwin/3.2/include/stdarg.h \ /usr/include/sys/reent.h /usr/include/sys/_types.h \ /usr/include/sys/types.h /usr/include/machine/types.h \ /usr/include/sys/features.h /usr/include/cygwin/types.h \ /usr/include/sys/sysmacros.h /usr/include/stdint.h \ /usr/include/sys/stdio.h
"Fine." I hear you cry, "Now I need to run gcc and use an editor to paste the results of -M into my makefile s. What a pain." And you'd be right if this was the whole answer. There are two traditional methods for including automatically generated dependencies into makefile s. The first and oldest is to add a line such as:
# Automatically generated dependencies follow - Do Not Edit
to the end of the makefile and then write a shell script to update these generated lines. This is certainly better than updating them by hand, but it's also very ugly. The second method is to add an include directive to the make program. By now most versions of make have the include directive and GNU make most certainly does.
So, the trick is to write a makefile target whose action runs gcc over all your source with the -M option, saves the results in a dependency file, and then re-runs make including the generated dependency file in the makefile so it can trigger the updates we need. Before GNU make , this is exactly what was done and the rule looked like:
depend: count_words.c lexer.c counter.c $(CC) -M $(CPPFLAGS) $^ > $@ include depend
Before running make to build the program, you would first execute make depend to generate the dependencies. This was good as far as it went, but often people would add or remove dependencies from their source without regenerating the depend file. Then source wouldn't get recompiled and the whole mess started again.
GNU make solved this last niggling problem with a cool feature and a simple algorithm. First, the algorithm. If we generated each source file's dependencies into its own dependency file with, say, a .d suffix and added the .d file itself as a target to this dependency rule, then make could know that the .d needed to be updated (along with the object file) when the source file changed:
counter.o counter.d: src/counter.c include/counter.h include/lexer.h
Generating this rule can be accomplished with a pattern rule and a ( fairly ugly) command script (this is taken directly from the GNU make manual): 
 This is an impressive little command script, but I think it requires some explanation. First, we use the C compiler with the -M option to create a temporary file containing the dependencies for this target. The temporary filename is created from the target, $@ , with a unique numeric suffix added, $$$$ . In the shell, the variable $$ returns the process number of the currently running shell. Since process numbers are unique, this produces a unique filename. We then use sed to add the .d file as a target to the rule. The sed expression consists of a search part, \($*\)\.o[ :]* , and a replacement part, \1.o $@ :, separated by commas. The search expression begins with the target stem, $* , enclosed in a regular expression (RE) group , \(\) , followed by the file suffix, \.o . After the target filename, there come zero or more spaces or colons, [ :]* . The replacement portion restores the original target by referencing the first RE group and appending the suffix, \1.o , then adding the dependency file target, $@ .
%.d: %.c $(CC) -M $(CPPFLAGS) $< > $@.$$$$; \ sed 's,\($*\)\.o[ :]*,.o $@ : ,g' < $@.$$$$ > $@; \ rm -f $@.$$$$
Now, for the cool feature. make will treat any file named in an include directive as a target to be updated. So, when we mention the .d files we want to include, make will automatically try to create these files as it reads the makefile . Here is our makefile with the addition of automatic dependency generation:
VPATH = src include CPPFLAGS = -I include SOURCES = count_words.c \ lexer.c \ counter.c count_words: counter.o lexer.o -lfl count_words.o: counter.h counter.o: counter.h lexer.h lexer.o: lexer.h include $(subst .c,.d,$(SOURCES)) %.d: %.c $(CC) -M $(CPPFLAGS) $< > $@.$$$$; \ sed 's,\($*\)\.o[ :]*,.o $@ : ,g' < $@.$$$$ > $@; \ rm -f $@.$$$$
The include directive should always be placed after the hand-written dependencies so that the default goal is not hijacked by some dependency file. The include directive takes a list of files (whose names can include wildcards). Here we use a make function, subst , to transform the list of source files into a list of dependency filenames. (We'll discuss subst in detail in Section 4.2.1 in Chapter 4.) For now, just note that this use replaces the string .c with .d in each of the words in $(SOURCES) .
When we run this makefile with the ”just-print option, we get:
$ make --just-print Makefile:13: count_words.d: No such file or directory Makefile:13: lexer.d: No such file or directory Makefile:13: counter.d: No such file or directory gcc -M -I include src/counter.c > counter.d.$$; \ sed 's,\(counter\)\.o[ :]*,.o counter.d : ,g' < counter.d.$$ > counter.d; \ rm -f counter.d.$$ flex -t src/lexer.l > lexer.c gcc -M -I include lexer.c > lexer.d.$$; \ sed 's,\(lexer\)\.o[ :]*,.o lexer.d : ,g' < lexer.d.$$ > lexer.d; \ rm -f lexer.d.$$ gcc -M -I include src/count_words.c > count_words.d.$$; \ sed 's,\(count_words\)\.o[ :]*,.o count_words.d : ,g' < count_words.d. $$ count_words.d; \ rm -f count_words.d.$$ rm lexer.c gcc -I include -c -o count_words.o src/count_words.c gcc -I include -c -o counter.o src/counter.c gcc -I include -c -o lexer.o lexer.c gcc count_words.o counter.o lexer.o /lib/libfl.a -o count_words
At first the response by make is a little alarming ”it looks like a make error message. But not to worry, this is just a warning. make looks for the include files and doesn't find them, so it issues the No such file or directory warning before searching for a rule to create these files. This warning can be suppressed by preceding the include directive with a hyphen ( - ). The lines following the warnings show make invoking gcc with the -M option, then running the sed command. Notice that make must invoke flex to create lexer.c , then it deletes the temporary lexer.c before beginning to satisfy the default goal.
This gives you a taste of automatic dependency generation. There's lots more to say, such as how do you generate dependencies for other languages or build tree layouts. We'll return to this topic in more depth in Part II of this book.