| Multidirectory projects can also be managed without recursive make s. The difference here is that the source manipulated by the makefile lives in more than one directory. To accommodate this, references to files in subdirectories must include the path to the file ”either absolute or relative. Often, the makefile managing a large project has many targets, one for each module in the project. For our mp3 player example, we would need targets for each of the libraries and each of the applications. It can also be useful to add phony targets for collections of modules such as the collection of all libraries. The default goal would typically build all of these targets. Often the default goal builds documentation and runs a testing procedure as well. The most straightforward use of nonrecursive make includes targets, object file references, and dependencies in a single makefile . This is often unsatisfying to developers familiar with recursive make because information about the files in a directory is centralized in a single file while the source files themselves are distributed in the filesystem. To address this issue, the Miller paper on nonrecursive make suggests using one make include file for each directory containing file lists and module-specific rules. The top-level makefile includes these sub- makefile s. Example 6-1 shows a makefile for our mp3 player that includes a module-level makefile from each subdirectory. Example 6-2 shows one of the module-level include files. Example 6-1. A nonrecursive makefile # Collect information from each module in these four variables. # Initialize them here as simple variables. programs := sources := libraries := extra_clean := objects = $(subst .c,.o,$(sources)) dependencies = $(subst .c,.d,$(sources)) include_dirs := lib include CPPFLAGS += $(addprefix -I ,$(include_dirs)) vpath %.h $(include_dirs) MV := mv -f RM := rm -f SED := sed all: include lib/codec/module.mk include lib/db/module.mk include lib/ui/module.mk include app/player/module.mk .PHONY: all all: $(programs) .PHONY: libraries libraries: $(libraries) .PHONY: clean clean: $(RM) $(objects) $(programs) $(libraries) \ $(dependencies) $(extra_clean) ifneq "$(MAKECMDGOALS)" "clean" include $(dependencies) endif %.c %.h: %.y $(YACC.y) --defines $< $(MV) y.tab.c $*.c $(MV) y.tab.h $*.h %.d: %.c $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -M $< \ $(SED) 's,\($(notdir $*)\.o\) *:,$(dir $@) $@: ,' > $@.tmp $(MV) $@.tmp $@
Example 6-2. The lib/codec include file for a nonrecursive makefile local_dir := lib/codec local_lib := $(local_dir)/libcodec.a local_src := $(addprefix $(local_dir)/,codec.c) local_objs := $(subst .c,.o,$(local_src)) libraries += $(local_lib) sources += $(local_src) $(local_lib): $(local_objs) $(AR) $(ARFLAGS) $@ $^
Thus, all the information specific to a module is contained in an include file in the module directory itself. The top-level makefile contains only a list of modules and include directives. Let's examine the makefile and module.mk in detail. Each module.mk include file appends the local library name to the variable libraries and the local sources to sources . The local_ variables are used to hold constant values or to avoid duplicating a computed value. Note that each include file reuses these same local_ variable names . Therefore, it uses simple variables (those assigned with := ) rather than recursive ones so that builds combining multiple makefile s hold no risk of infecting the variables in each makefile . The library name and source file lists use a relative path as discussed earlier. Finally, the include file defines a rule for updating the local library. There is no problem with using the local_ variables in this rule because the target and prerequisite parts of a rule are immediately evaluated. In the top-level makefile , the first four lines define the variables that accumulate each module's specific file information. These variables must be simple variables because each module will append to them using the same local variable name: local_src := $(addprefix $(local_dir)/,codec.c) ... sources += $(local_src)
If a recursive variable were used for sources , for instance, the final value would simply be the last value of local_src repeated over and over. An explicit assignment is required to initialize these simple variables, even though they are assigned null values, since variables are recursive by default. The next section computes the object file list, objects , and dependency file list from the sources variable. These variables are recursive because at this point in the makefile the sources variable is empty. It will not be populated until later when the include files are read. In this makefile , it is perfectly reasonable to move the definition of these variables after the includes and change their type to simple variables, but keeping the basic file lists (e.g., sources , libraries , objects ) together simplifies understanding the makefile and is generally good practice. Also, in other makefile situations, mutual references between variables require the use of recursive variables. Next, we handle C language include files by setting CPPFLAGS . This allows the compiler to find the headers. We append to the CPPFLAGS variable because we don't know if the variable is really empty; command-line options, environment variables, or other make constructs may have set it. The vpath directive allows make to find the headers stored in other directories. The include_dirs variable is used to avoid duplicating the include directory list. Variables for mv , rm , and sed are defined to avoid hard coding programs into the makefile . Notice the case of variables. We are following the conventions suggested in the make manual. Variables that are internal to the makefile are lowercased; variables that might be set from the command line are uppercased. In the next section of the makefile, things get more interesting. We would like to begin the explicit rules with the default target, all . Unfortunately, the prerequisite for all is the variable programs . This variable is evaluated immediately, but is set by reading the module include files. So, we must read the include files before the all target is defined. Unfortunately again, the include modules contain targets, the first of which will be considered the default goal. To work through this dilemma, we can specify the all target with no prerequisites, source the include files, then add the prerequisites to all later. The remainder of the makefile is already familiar from previous examples, but how make applies implicit rules is worth noting. Our source files now reside in subdirectories. When make tries to apply the standard %.o: %.c rule, the prerequisite will be a file with a relative path, say lib/ui/ui.c . make will automatically propagate that relative path to the target file and attempt to update lib/ui/ui.o . Thus, make automagically does the Right Thing. There is one final glitch. Although make is handling paths correctly, not all the tools used by the makefile are. In particular, when using gcc , the generated dependency file does not include the relative path to the target object file. That is, the output of gcc -M is: ui.o: lib/ui/ui.c include/ui/ui.h lib/db/playlist.h
rather than what we expect: lib/ui/ui.o: lib/ui/ui.c include/ui/ui.h lib/db/playlist.h
This disrupts the handling of header file prerequisites. To fix this problem we can alter the sed command to add relative path information: $(SED) 's,\($(notdir $*)\.o\) *:,$(dir $@) $@: ,'
Tweaking the makefile to handle the quirks of various tools is a normal part of using make . Portable makefile s are often very complex due to vagarities of the diverse set of tools they are forced to rely upon. We now have a decent nonrecursive makefile , but there are maintenance problems. The module.mk include files are largely similar. A change to one will likely involve a change to all of them. For small projects like our mp3 player it is annoying. For large projects with several hundred include files it can be fatal. By using consistent variable names and regularizing the contents of the include files, we position ourselves nicely to cure these ills. Here is the lib/codec include file after refactoring: local_src := $(wildcard $(subdirectory)/*.c) $(eval $(call make-library, $(subdirectory)/libcodec.a, $(local_src)))
Instead of specifying source files by name, we assume we want to rebuild all .c files in the directory. The make-library function now performs the bulk of the tasks for an include file. This function is defined at the top of our project makefile as: # $(call make-library, library-name, source-file-list) define make-library libraries += sources += : $(call source-to-object,) $(AR) $(ARFLAGS) $$@ $$^ endef
The function appends the library and sources to their respective variables, then defines the explicit rule to build the library. Notice how the automatic variables use two dollar signs to defer actual evaluation of the $@ and $^ until the rule is fired . The source-to-object function translates a list of source files to their corresponding object files: source-to-object = $(subst .c,.o,$(filter %.c,)) \ $(subst .y,.o,$(filter %.y,)) \ $(subst .l,.o,$(filter %.l,))
In our previous version of the makefile , we glossed over the fact that the actual parser and scanner source files are playlist.y and scanner.l . Instead, we listed the source files as the generated .c versions. This forced us to list them explicitly and to include an extra variable, extra_clean . We've fixed that issue here by allowing the sources variable to include .y and .l files directly and letting the source-to-object function do the work of translating them. In addition to modifying source-to-object , we need another function to compute the yacc and lex output files so the clean target can perform proper clean up. The generated-source function simply accepts a list of sources and produces a list of intermediate files as output: # $(call generated-source, source-file-list) generated-source = $(subst .y,.c,$(filter %.y,)) \ $(subst .y,.h,$(filter %.y,)) \ $(subst .l,.c,$(filter %.l,))
Our other helper function, subdirectory , allows us to omit the variable local_dir . subdirectory = $(patsubst %/makefile,%, \ $(word \ $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)))
As noted in Section 4.2.1 in Chapter 4, we can retrieve the name of the current makefile from MAKEFILE_LIST . Using a simple patsubst , we can extract the relative path from the top-level makefile . This eliminates another variable and reduces the differences between include files. Our final optimization (at least for this example), uses wildcard to acquire the source file list. This works well in most environments where the source tree is kept clean. However, I have worked on projects where this is not the case. Old code was kept in the source tree "just in case." This entailed real costs in terms of programmer time and anguish since old, dead code was maintained when it was found by global search and replace and new programmers (or old ones not familiar with a module) attempted to compile or debug code that was never used. If you are using a modern source code control system, such as CVS, keeping dead code in the source tree is unnecessary (since it resides in the repository) and using wildcard becomes feasible . The include directives can also be optimzed: modules := lib/codec lib/db lib/ui app/player . . . include $(addsuffix /module.mk,$(modules))
For larger projects, even this can be a maintenance problem as the list of modules grows to the hundreds or thousands. Under these circumstances, it might be preferable to define modules as a find command: modules := $(subst /module.mk,,$(shell find . -name module.mk)) . . . include $(addsuffix /module.mk,$(modules))
We strip the filename from the find output so the modules variable is more generally useful as the list of modules. If that isn't necessary, then, of course, we would omit the subst and addsuffix and simply save the output of find in modules . Example 6-3 shows the final makefile . Example 6-3. A nonrecursive makefile, version 2 # $(call source-to-object, source-file-list) source-to-object = $(subst .c,.o,$(filter %.c,)) \ $(subst .y,.o,$(filter %.y,)) \ $(subst .l,.o,$(filter %.l,)) # $(subdirectory) subdirectory = $(patsubst %/module.mk,%, \ $(word \ $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))) # $(call make-library, library-name, source-file-list) define make-library libraries += sources += : $(call source-to-object,) $(AR) $(ARFLAGS) $$@ $$^ endef # $(call generated-source, source-file-list) generated-source = $(subst .y,.c,$(filter %.y,)) \ $(subst .y,.h,$(filter %.y,)) \ $(subst .l,.c,$(filter %.l,)) # Collect information from each module in these four variables. # Initialize them here as simple variables. modules := lib/codec lib/db lib/ui app/player programs := libraries := sources := objects = $(call source-to-object,$(sources)) dependencies = $(subst .o,.d,$(objects)) include_dirs := lib include CPPFLAGS += $(addprefix -I ,$(include_dirs)) vpath %.h $(include_dirs) MV := mv -f RM := rm -f SED := sed all: include $(addsuffix /module.mk,$(modules)) .PHONY: all all: $(programs) .PHONY: libraries libraries: $(libraries) .PHONY: clean clean: $(RM) $(objects) $(programs) $(libraries) $(dependencies) \ $(call generated-source, $(sources)) ifneq "$(MAKECMDGOALS)" "clean" include $(dependencies) endif %.c %.h: %.y $(YACC.y) --defines $< $(MV) y.tab.c $*.c $(MV) y.tab.h $*.h %.d: %.c $(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -M $< \ $(SED) 's,\($(notdir $*)\.o\) *:,$(dir $@) $@: ,' > $@.tmp $(MV) $@.tmp $@
Using one include file per module is quite workable and has some advantages, but I'm not convinced it is worth doing. My own experience with a large Java project indicates that a single top-level makefile , effectively inserting all the module.mk files directly into the makefile , provides a reasonable solution. This project included 997 separate modules, about two dozen libraries, and half a dozen applications. There were several makefile s for disjoint sets of code. These makefile s were roughly 2,500 lines long. A common include file containing global variables, user -defined functions, and pattern rules was another 2,500 lines. Whether you choose a single makefile or break out module information into include files, the nonrecursive make solution is a viable approach to building large projects. It also solves many traditional problems found in the recursive make approach. The only drawback I'm aware of is the paradigm shift required for developers used to recursive make . | 