The Linux kernel makefile is an excellent example of using make in a complex build environment. While it is beyond the scope of this book to explain how the Linux kernel is structured and built, we can examine several interesting uses of make employed by the kernel build system. See http://macarchive.linuxsymposium.org/ols2003/Proceedings/All-Reprints/Reprint-Germaschewski-OLS2003.pdf for a more complete discussion of the 2.5/2.6 kernel build process and its evolution from the 2.4 approach.
Since the makefile has so many facets, we will discuss just a few features that are applicable to a variety of applications. First, we'll look at how single-letter make variables are used to simulate single-letter command-line options. We'll see how the source and binary trees are separated in a way that allows users to invoke make from the source tree. Next , we'll examine the way the makefile controls the verboseness of the output. Then we'll review the most interesting user -defined functions and see how they reduce code duplication, improve readability, and provide encapsulation. Finally, we'll look at the way the makefile implements a simple help facility.
The Linux kernel build follows the familiar configure, build, install pattern used by my most free software. While many free and open software packages use a separate configure script (typically built by autoconf ), the Linux kernel makefile implements configuration with make , invoking scripts and helper programs indirectly.
When the configuration phase is complete, a simple make or make all will build the bare kernel, all the modules, and produce a compressed kernel image (these are the vmlinux , modules , and bzImage targets, respectively). Each kernel build is given a unique version number in the file version.o linked into the kernel. This number (and the version.o file) are updated by the makefile itself.
Some makefile features you might want to adapt to your own makefile are: the handling of command line options, analyzing command-line goals, saving build status between builds, and managing the output of make .
11.2.1 Command-Line Options
The first part of the makefile contains code for setting common build options from the command line. Here is an excerpt that controls the verbose flag:
# To put more focus on warnings, be less verbose as default # Use 'make V=1' to see the full commands ifdef V ifeq ("$(origin V)", "command line") KBUILD_VERBOSE = $(V) endif endif ifndef KBUILD_VERBOSE KBUILD_VERBOSE = 0 endif
The nested ifdef / ifeq pair ensures that the KBUILD_VERBOSE variable is set only if V is set on the command line. Setting V in the environment or makefile has no effect. The following ifndef conditional will then turn off the verbose option if KBUILD_VERBOSE has not yet been set. To set the verbose option from either the environment or makefile , you must set KBUILD_VERBOSE and not V .
Notice, however, that setting KBUILD_VERBOSE directly on the command line is allowed and works as expected. This can be useful when writing shell scripts (or aliases) to invoke the makefile . These scripts would then be more self-documenting , similar to using GNU long options.
The other command-line options, sparse checking ( C ) and external modules ( M ), both use the same careful checking to avoid accidentally setting them from within the makefile .
The next section of the makefile handles the output directory option ( O ). This is a fairly involved piece of code. To highlight its structure, we've replaced some parts of this excerpt with ellipses:
# kbuild supports saving output files in a separate directory. # To locate output files in a separate directory two syntax'es are supported. # In both cases the working directory must be the root of the kernel src. # 1) O= # Use "make O=dir/to/store/output/files/" # # 2) Set KBUILD_OUTPUT # Set the environment variable KBUILD_OUTPUT to point to the directory # where the output files shall be placed. # export KBUILD_OUTPUT=dir/to/store/output/files/ # make # # The O= assigment takes precedence over the KBUILD_OUTPUT environment variable. # KBUILD_SRC is set on invocation of make in OBJ directory # KBUILD_SRC is not intended to be used by the regular user (for now) ifeq ($(KBUILD_SRC),) # OK, Make called in directory where kernel src resides # Do we want to locate output files in a separate directory? ifdef O ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif endif ... ifneq ($(KBUILD_OUTPUT),) ... .PHONY: $(MAKECMDGOALS) $(filter-out _all,$(MAKECMDGOALS)) _all: $(if $(KBUILD_VERBOSE:1=),@)$(MAKE) -C $(KBUILD_OUTPUT) \ KBUILD_SRC=$(CURDIR) KBUILD_VERBOSE=$(KBUILD_VERBOSE) \ KBUILD_CHECK=$(KBUILD_CHECK) KBUILD_EXTMOD="$(KBUILD_EXTMOD)" \ -f $(CURDIR)/Makefile $@ # Leave processing to above invocation of make skip-makefile := 1 endif # ifneq ($(KBUILD_OUTPUT),) endif # ifeq ($(KBUILD_SRC),) # We process the rest of the Makefile if this is the final invocation of make ifeq ($(skip-makefile),) ... the rest of the makefile here ... endif # skip-makefile
Essentially, this says that if KBUILD_OUTPUT is set, invoke make recursively in the output directory defined by KBUILD_OUTPUT . Set KBUILD_SRC to the directory where make was originally executed, and grab the makefile from there as well. The rest of the makefile will not be seen by make , since skip-makefile will be set. The recursive make will reread this same makefile again, only this time KBUILD_SRC will be set, so skip-makefile will be undefined, and the rest of the makefile will be read and processed .
This concludes the processing of command-line options. The bulk of the makefile follows in the ifeq ($(skip-makefile),) section.
11.2.2 Configuration Versus Building
The makefile contains configuration targets and build targets. The configuration targets have the form menuconfig , defconfig , etc. Maintenance targets like clean are treated as configuration targets as well. Other targets such as all , vmlinux , and modules are build targets. The primary result of invoking a configuration target is two files: .config and .config.cmd . These two files are included by the makefile for build targets but are not included for configuration targets (since the configuration target creates them). It is also possible to mix both configuration targets and build targets on a single make invocation, such as:
$ make oldconfig all
In this case, the makefile invokes itself recursively handling each target individually, thus handling configuration targets separately from build targets.
The code controlling configuration, build, and mixed targets begins with:
# To make sure we do not include .config for any of the *config targets # catch them early, and hand them over to scripts/kconfig/Makefile # It is allowed to specify more targets when calling make, including # mixing *config targets and build targets. # For example 'make oldconfig all'. # Detect when mixed targets is specified, and make a second invocation # of make so .config is not included in this case either (for *config). no-dot-config-targets := clean mrproper distclean \ cscope TAGS tags help %docs check% config-targets := 0 mixed-targets := 0 dot-config := 1
The variable no-dot-config-targets lists additional targets that do not require a .config file. The code then initializes the config-targets , mixed-targets , and dot-config variables. The config-targets variable is 1 if there are any configuration targets on the command line. The dot-config variable is 1 if there are build targets on the command line. Finally, mixed-targets is 1 if there are both configuration and build targets.
The code to set dot-config is:
ifneq ($(filter $(no-dot-config-targets), $(MAKECMDGOALS)),) ifeq ($(filter-out $(no-dot-config-targets), $(MAKECMDGOALS)),) dot-config := 0 endif endif
The filter expression is non-empty if there are configuration targets in MAKECMDGOALS . The ifneq part is true if the filter expression is not empty. The code is hard to follow partly because it contains a double negative. The ifeq expression is true if MAKECMDGOALS contains only configuration targets. So, dot-config will be set to 0 if there are configuration targets and only configuration targets in MAKECMDGOALS . A more verbose implementation might make the meaning of these two conditionals more clear:
config-target-list := clean mrproper distclean \ cscope TAGS tags help %docs check% config-target-goal := $(filter $(config-target-list), $(MAKECMDGOALS)) build-target-goal := $(filter-out $(config-target-list), $(MAKECMDGOALS)) ifdef config-target-goal ifndef build-target-goal dot-config := 0 endif endif
The ifdef form can be used instead of ifneq , because empty variables are treated as undefined, but care must be taken to ensure a variable does not contain merely a string of blanks (which would cause it to be defined).
The config-targets and mixed-targets variables are set in the next code block:
ifeq ($(KBUILD_EXTMOD),) ifneq ($(filter config %config,$(MAKECMDGOALS)),) config-targets := 1 ifneq ($(filter-out config %config,$(MAKECMDGOALS)),) mixed-targets := 1 endif endif endif
KBUILD_EXTMOD will be non-empty when external modules are being built, but not during normal builds. The first ifneq will be true when MAKECMDGOALS contains a goal with the config suffix. The second ifneq will be true when MAKECMDGOALS contains nonconfig targets, too.
Once the variables are set, they are used in an if - else chain with four branches. The code has been condensed and indented to highlight its structure:
ifeq ($(mixed-targets),1) # We're called with mixed targets (*config and build targets). # Handle them one by one. %:: FORCE $(Q)$(MAKE) -C $(srctree) KBUILD_SRC= $@ else ifeq ($(config-targets),1) # *config targets only - make sure prerequisites are updated, and descend # in scripts/kconfig to make the *config target %config: scripts_basic FORCE $(Q)$(MAKE) $(build)=scripts/kconfig $@ else # Build targets only - this includes vmlinux, arch specific targets, clean # targets and others. In general all targets except *config targets. ... ifeq ($(dot-config),1) # In this section, we need .config # Read in dependencies to all Kconfig* files, make sure to run # oldconfig if changes are detected. -include .config.cmd include .config # If .config needs to be updated, it will be done via the dependency # that autoconf has on .config. # To avoid any implicit rule to kick in, define an empty command .config: ; # If .config is newer than include/linux/autoconf.h, someone tinkered # with it and forgot to run make oldconfig include/linux/autoconf.h: .config $(Q)$(MAKE) -f $(srctree)/Makefile silentoldconfig else # Dummy target needed, because used as prerequisite include/linux/autoconf.h: ; endif include $(srctree)/arch/$(ARCH)/Makefile ... lots more make code ... endif #ifeq ($(config-targets),1) endif #ifeq ($(mixed-targets),1)
The first branch, ifeq ($(mixed-targets),1) , handles mixed command-line arguments. The only target in this branch is a completely generic pattern rule. Since there are no specific rules to handle targets (those rules are in another conditional branch), each target invokes the pattern rule once. This is how a command line with both configuration targets and build targets is separated into a simpler command line. The command script for the generic pattern rule invokes make recursively for each target, causing this same logic to be applied, only this time with no mixed command-line targets. The FORCE prerequisite is used instead of .PHONY , because pattern rules like:
cannot be declared .PHONY . So it seems reasonable to use FORCE consistently everywhere.
The second branch of the if - else chain, ifeq ($(config-targets),1) , is invoked when there are only configuration targets on the command line. Here the primary target in the branch is the pattern rule %config (other targets have been omitted). The command script invokes make recursively in the scripts/kconfig subdirectory and passes along the target. The curious $(build) construct is defined at the end of the makefile :
# Shorthand for $(Q)$(MAKE) -f scripts/Makefile.build obj=dir # Usage: # $(Q)$(MAKE) $(build)=dir build := -f $(if $(KBUILD_SRC),$(srctree)/)scripts/Makefile.build obj
If KBUILD_SRC is set, the -f option is given a full path to the scripts makefile , otherwise a simple relative path is used. Next, the obj variable is set to the righthand side of the equals sign.
The third branch, ifeq ($(dot-config),1) , handles build targets that require including the two generated configuration files, .config and .config.cmd . The final branch merely includes a dummy target for autoconf.h to allow it to be used as a prerequisite, even if it doesn't exist.
Most of the remainder of the makefile follows the third and fourth branches. It contains the code for building the kernel and modules.
11.2.3 Managing Command Echo
The kernel makefile s use a novel technique for managing the level of detail echoed by commands. Each significant task is represented in both a verbose and a quiet version. The verbose version is simply the command to be executed in its natural form and is stored in a variable named cmd_ action . The brief version is a short message describing the action and is stored in a variable named quiet_cmd_ action . For example, the command to produce emacs tags is:
quiet_cmd_TAGS = MAKE $@ cmd_TAGS = $(all-sources) etags -
A command is executed by calling the cmd function:
# If quiet is set, only print short version of command cmd = @$(if $($(quiet)cmd_$(1)),\ echo ' $($(quiet)cmd_$(1))' &&) $(cmd_$(1))
To invoke the code for building emacs tags, the makefile would contain:
TAGS: $(call cmd,TAGS)
Notice the cmd function begins with an @ , so the only text echoed by the function is text from the echo command. In normal mode, the variable quiet is empty, and the test in the if , $($(quiet)cmd_$(1)) , expands to $(cmd_TAGS) . Since this variable is not empty, the entire function expands to:
echo ' $(all-sources) etags -' && $(all-sources) etags -
If the quiet version is desired, the variable quiet contains the value quiet_ and the function expands to:
echo ' MAKE $@' && $(all-sources) etags -
The variable can also be set to silent_ . Since there is no command silent_cmd_TAGS , this value causes the cmd function to echo nothing at all.
Echoing the command sometimes becomes more complex, particularly if commands contain single quotes. In these cases, the makefile contains this code:
$(if $($(quiet)cmd_$(1)),echo ' $(subst ','\'',$($(quiet)cmd_$(1)))';)
Here the echo command contains a substitution that replaces single quotes with escaped single quotes to allow them to be properly echoed.
Minor commands that do not warrant the trouble of writing cmd_ and quiet_cmd_ variables are prefixed with $(Q) , which contains either nothing or @ :
ifeq ($(KBUILD_VERBOSE),1) quiet = Q = else quiet=quiet_ Q = @ endif # If the user is running make -s (silent mode), suppress echoing of # commands ifneq ($(findstring s,$(MAKEFLAGS)),) quiet=silent_ endif
11.2.4 User-Defined Functions
The kernel makefile defines a number of functions. Here we cover the most interesting ones. The code has been reformatted to improve readability.
The check_gcc function is used to select a gcc command-line option.
# $(call check_gcc,preferred-option,alternate-option) check_gcc = \ $(shell if $(CC) $(CFLAGS) $(1) -S -o /dev/null \ -xc /dev/null > /dev/null 2>&1; \ then \ echo "$(1)"; \ else \ echo "$(2)"; \ fi ;)
The function works by invoking gcc on a null input file with the preferred command-line option. The output file, standard output, and standard error files are discarded. If the gcc command succeeds, it means the preferred command-line option is valid for this architecture and is returned by the function. Otherwise, the option is invalid and the alternate option is returned. An example use can be found in arch/i386/Makefile :
# prevent gcc from keeping the stack 16 byte aligned CFLAGS += $(call check_gcc,-mpreferred-stack-boundary=2,)
The if_changed_dep function generates dependency information using a remarkable technique.
# execute the command and also postprocess generated # .d dependencies file if_changed_dep = \ $(if \ $(strip $? \ $(filter-out FORCE $(wildcard $^),$^) \ $(filter-out $(cmd_$(1)),$(cmd_$@)) \ $(filter-out $(cmd_$@),$(cmd_$(1)))), \ @set -e; \ $(if $($(quiet)cmd_$(1)), \ echo ' $(subst ','\'',$($(quiet)cmd_$(1)))';) \ $(cmd_$(1)); \ scripts/basic/fixdep \ $(depfile) \ $@ \ '$(subst $$,$$$$,$(subst ','\'',$(cmd_$(1))))' \ > $(@D)/.$(@F).tmp; \ rm -f $(depfile); \ mv -f $(@D)/.$(@F).tmp $(@D)/.$(@F).cmd)
The function consists of a single if clause. The details of the test are pretty obscure, but it is clear the intent is to be non-empty if the dependency file should be regenerated. Normal dependency information is concerned with the modification timestamps on files. The kernel build system adds another wrinkle to this task. The kernel build uses a wide variety of compiler options to control the construction and behavior of components . To ensure that command-line options are properly accounted for during a build, the makefile is implemented so that if command-line options used for a particular target change, the file is recompiled. Let's see how this is accomplished.
In essence, the command used to compile each file in the kernel is saved in a .cmd file. When a subsequent build is executed, make reads the .cmd files and compares the current compile command with the last command. If they are different, the .cmd dependency file is regenerated causing the object file to be rebuilt. The .cmd file usually contains two items: the dependencies that represent actual files for the target file and a single variable recording the command-line options. For example, the file arch/i386/kernel/cpu/mtrr/if.c yields this (abbreviated) file:
cmd_arch/i386/kernel/cpu/mtrr/if.o := gcc -Wp,-MD ...; if.c deps_arch/i386/kernel/cpu/mtrr/if.o := \ arch/i386/kernel/cpu/mtrr/if.c \ ... arch/i386/kernel/cpu/mtrr/if.o: $(deps_arch/i386/kernel/cpu/mtrr/if.o) $(deps_arch/i386/kernel/cpu/mtrr/if.o):
Getting back to the if_changed_dep function, the first argument to the strip is simply the prerequisites that are newer than the target, if any. The second argument to strip is all the prerequisites other than files and the empty target FORCE . The really obscure bit is the last two filter-out calls:
$(filter-out $(cmd_$(1)),$(cmd_$@)) $(filter-out $(cmd_$@),$(cmd_$(1)))
One or both of these calls will expand to a non-empty string if the command-line options have changed. The macro $(cmd_$(1)) is the current command and $(cmd_$@) will be the previous command, for instance the variable cmd_arch/i386/kernel/cpu/mtrr/if.o just shown. If the new command contains additional options, the first filter-out will be empty, and the second will expand to the new options. If the new command contains fewer options, the first command will contain the deleted options and the second will be empty. Interestingly, since filter-out accepts a list of words (each treated as an independent pattern), the order of options can change and the filter-out will still accurately identify added or removed options. Pretty nifty.
The first statement in the command script sets a shell option to exit immediately on error. This prevents the multiline script from corrupting files in the event of problems. For simple scripts another way to achieve this effect is to connect statements with && rather than semicolons.
The next statement is an echo command written using the techniques described in Section 11.2.3 earlier in this chapter, followed by the dependency generating command itself. The command writes $(depfile) , which is then transformed by scripts/basic/fixdep . The nested subst function in the fixdep command line first escapes single quotes, then escapes occurrences of $$ (the current process number in shell syntax).
Finally, if no errors have occurred, the intermediate file $(depfile) is removed and the generated dependency file (with its .cmd suffix) is moved into place.
The next function, if_changed_rule , uses the same comparison technique as if_changed_dep to control the execution of a command:
# Usage: $(call if_changed_rule,foo) # will check if $(cmd_foo) changed, or any of the prequisites changed, # and if so will execute $(rule_foo) if_changed_rule = \ $(if $(strip $? \ $(filter-out $(cmd_$(1)),$(cmd_$(@F))) \ $(filter-out $(cmd_$(@F)),$(cmd_$(1)))), \ @$(rule_$(1)))
In the topmost makefile , this function is used to link the kernel with these macros:
# This is a bit tricky: If we need to relink vmlinux, we want # the version number incremented, which means recompile init/version.o # and relink init/init.o. However, we cannot do this during the # normal descending-into-subdirs phase, since at that time # we cannot yet know if we will need to relink vmlinux. # So we descend into init/ inside the rule for vmlinux again. ... quiet_cmd_vmlinux_ _ = LD $@ define cmd_vmlinux_ _ $(LD) $(LDFLAGS) $(LDFLAGS_vmlinux) \ ... endef # set -e makes the rule exit immediately on error define rule_vmlinux_ _ +set -e; \ $(if $(filter .tmp_kallsyms%,$^),, \ echo ' GEN .version'; \ . $(srctree)/scripts/mkversion > .tmp_version; \ mv -f .tmp_version .version; \ $(MAKE) $(build)=init;) \ $(if $($(quiet)cmd_vmlinux_ _), \ echo ' $($(quiet)cmd_vmlinux_ _)' &&) \ $(cmd_vmlinux_ _); \ echo 'cmd_$@ := $(cmd_vmlinux_ _)' > $(@D)/.$(@F).cmd endef define rule_vmlinux $(rule_vmlinux_ _); \ $(NM) $@ \ grep -v '\(compiled\)\...' \ sort > System.map endef
The if_changed_rule function is used to invoke rule_vmlinux , which performs the link and builds the final System.map . As the comment in the makefile notes, the rule_vmlinux_ _ function must regenerate the kernel version file and relink init.o before relinking vmlinux . This is controlled by the first if in rule_vmlinux_ _ . The second if controls the echoing of the link command, $(cmd_vmlinux_ _) . After the link command, the actual command executed is recorded in a .cmd file for comparison in the next build.