sys.dependfile.mk

This makefile is read via sys.mk, and looks in the current directory (.CURDIR) to find the first entry in .MAKE.DEPENDFILE_PREFERENCE that exists. The default list is:

# All depend file names should start with this
.MAKE.DEPENDFILE_PREFIX ?= Makefile.depend

# The order of preference: we will use the first one of these we find
# It usually makes sense to order from most specific to least.
.MAKE.DEPENDFILE_PREFERENCE ?= \
        ${.CURDIR}/${.MAKE.DEPENDFILE_PREFIX}.${MACHINE} \
        ${.CURDIR}/${.MAKE.DEPENDFILE_PREFIX}

# Normally the 1st entry is our default choice
# Another useful default is ${.MAKE.DEPENDFILE_PREFIX}
.MAKE.DEPENDFILE_DEFAULT ?= ${.MAKE.DEPENDFILE_PREFERENCE:[1]}

That is if a Makefile.depend qualified with ${MACHINE} exists that will be used in preference to the more generic name. A more complex preference list might be:

.MAKE.DEPENDFILE_PREFERENCE ?= \
        ${.CURDIR}/${.MAKE.DEPENDFILE_PREFIX}.auto.${MACHINE} \
        ${.CURDIR}/${.MAKE.DEPENDFILE_PREFIX}.manual.${MACHINE} \
        ${.CURDIR}/${.MAKE.DEPENDFILE_PREFIX}.auto \
        ${.CURDIR}/${.MAKE.DEPENDFILE_PREFIX}.manual \
        ${.CURDIR}/${.MAKE.DEPENDFILE_PREFIX}

which could be useful if SCM triggers are to allow Makefile.depend.auto* to be committed with less restriction than manually edited ones.

In the FreeBSD build, a plain Makefile.depend suffices for the vast majority of the tree - with lib/${CSU_DIR} hiding the most common machine specific dependency.

For a small number of directories though, either DIRDEPS or the local dependencies are quite machine specific, and for these a Makefile.depend.${MACHINE} is more appropriate.

Sometimes we want to build some of the tree for both the target machine and the build host itself. We use the pseudo machine host to represent the build host, and invariably use a qualified file Makefile.depend.host because the DIRDEPS needed is usually a small subset of that needed when building for the target.

Note: when building for host I use an objdir named for HOST_TARGET a variable derrived from the OS name and architecture such as netbsd5-i386, freebsd10-amd64 or linux3-x86_64. This makes it safe to simultaneously build a tree shared via NFS on different hosts. It is also intended to not be confused with any objdir derived from TARGET_SPEC_VARS.

example

For those that need concrete examples, we can look at the dirdeps.cache for bin/cat (on NetBSD):

# Autogenerated - do NOT edit!

BUILD_DIRDEPS=no

.include <dirdeps.mk>

# bin/cat.i386
dirdeps: \
        ${SRCTOP}/bin/cat.i386 \
        ${SRCTOP}/gnu/lib/crtstuff4.i386 \
        ${SRCTOP}/gnu/lib/libgcc4/libgcc.i386 \
        ${SRCTOP}/gnu/lib/libgcc4/libgcc_eh.i386 \
        ${SRCTOP}/include.i386 \
        ${SRCTOP}/lib/csu/i386_elf.i386 \
        ${SRCTOP}/lib/libc.i386 \
        ${SRCTOP}/lib/libpthread.i386 \
        ${SRCTOP}/sys/arch/i386/include.i386 \
        ${SRCTOP}/sys/arch/x86/include.i386 \
        ${SRCTOP}/sys/sys.i386

${SRCTOP}/bin/cat.i386: _DIRDEP_USE
${SRCTOP}/gnu/lib/crtstuff4.i386: _DIRDEP_USE
${SRCTOP}/gnu/lib/libgcc4/libgcc.i386: _DIRDEP_USE
${SRCTOP}/gnu/lib/libgcc4/libgcc_eh.i386: _DIRDEP_USE
${SRCTOP}/include.i386: _DIRDEP_USE
${SRCTOP}/lib/csu/i386_elf.i386: _DIRDEP_USE
${SRCTOP}/lib/libc.i386: _DIRDEP_USE
${SRCTOP}/lib/libpthread.i386: _DIRDEP_USE
${SRCTOP}/sys/arch/i386/include.i386: _DIRDEP_USE
${SRCTOP}/sys/arch/x86/include.i386: _DIRDEP_USE
${SRCTOP}/sys/sys.i386: _DIRDEP_USE


${SRCTOP}/bin/cat.i386: \
        ${SRCTOP}/gnu/lib/crtstuff4.i386 \
        ${SRCTOP}/gnu/lib/libgcc4/libgcc.i386 \
        ${SRCTOP}/gnu/lib/libgcc4/libgcc_eh.i386 \
        ${SRCTOP}/include.i386 \
        ${SRCTOP}/lib/csu/i386_elf.i386 \
        ${SRCTOP}/lib/libc.i386 \
        ${SRCTOP}/lib/libpthread.i386 \
        ${SRCTOP}/sys/arch/i386/include.i386 \
        ${SRCTOP}/sys/arch/x86/include.i386 \
        ${SRCTOP}/sys/sys.i386

skip a bit ....

# lib/libc.i386
dirdeps: \
        ${SRCTOP}/common/include/prop.i386 \
        ${SRCTOP}/include.i386 \
        ${SRCTOP}/include/rpc.i386 \
        ${SRCTOP}/lib/libpthread.i386 \
        ${SRCTOP}/sys/arch/i386/include.i386 \
        ${SRCTOP}/sys/arch/x86/include.i386 \
        ${SRCTOP}/sys/sys.i386

${SRCTOP}/include/rpc.i386: _DIRDEP_USE


${SRCTOP}/lib/libc.i386: \
        ${SRCTOP}/common/include/prop.i386 \
        ${SRCTOP}/include.i386 \
        ${SRCTOP}/include/rpc.i386 \
        ${SRCTOP}/lib/libpthread.i386 \
        ${SRCTOP}/sys/arch/i386/include.i386 \
        ${SRCTOP}/sys/arch/x86/include.i386 \
        ${SRCTOP}/sys/sys.i386

.info ${.newline}${TRACER}Makefiles read: total=41 depend=14 dirdeps=14

we can see that ${SRCTOP}/include.${MACHINE} would need to be built before ${SRCTOP}/lib/libc.${MACHINE} which is built before ${SRCTOP}/bin/cat.${MACHINE}.

DIRDEPS_CACHE

The time it takes for the level 0 bmake to compute the tree graph is directly proportional to the number of Makefile.depend files to be read. It can be as little as a second, or as much as several minutes (dirdeps.mk outputs stats so we can track this):

Makefiles read: total=38 depend=11 dirdeps=16 seconds=0
Makefiles read: total=11609 depend=11573 dirdeps=13560 seconds=200

If we enable using DIRDEPS_CACHE, then the computed graph is saved and can be re-used see example above.

Naturally there is a .meta files so if any of the Makefile.depend files are updated we have to re-compute the graph, but since saving the cache adds only a few seconds overhead and can save you several minutes when it is up to date, and as a bonus it can be invaluable for diagnosing why things you didn't expect to be built, were.

WITH_SOME_KNOB=1        # we need to enable SOME_KNOB
DEP_EXPORT_VARS+= WITH_SOME_KNOB
# incase we have old version
.export ${DEP_EXPORT_VARS}

_DIRDEP_USE

This target macro is associated with each expanded dirdep - absolute path plus .${TARGET_SPEC} extension.

The target's extension describes the .${TARGET_SPEC} it should be built for. We can thus set MACHINE and TARGET_SPEC to the extension of the target before running bmake in the appropriate directory.

Also since the target will never exist (because of the .${TARGET_SPEC} extension) it is always out of date and will always be visited. Which is exactly what we want.

TARGET_SPEC_VARS

As noted above, we normally only care about setting ${MACHINE} for each sub-make. The default TARGET_SPEC_VARS is just MACHINE, but other variables can be added such that TARGET_SPEC is set to a tuple for each sub-make, though we expect that MACHINE is always the first element of that tuple.

In this case sys.mk is expected to decompose ${TARGET_SPEC} and set the component variables appropriately. The coordination is not hard since dirdeps.mk is using the TARGET_SPEC_VARS list set by sys.mk.

While this makes dirdeps.mk more complex, it adds great flexibility. For example building for multiple target operating systems at the same time (and multiple machines for each TARGET_OS).

MK_DIRDEPS_CACHE

If MK_DIRDEPS_CACHE is "yes", and NO_DIRDEPS is not defined then dirdeps.mk uses a sub-make to process all the Makefile.depend* files and capture the result in DIRDEPS_CACHE which defaults to ${OBJTOP}/dirdeps-cache.${.TARGETS} (suitably transformed).

We rely on meta mode for this to work well since the .meta file allows us to know if the cache is out-of-date, and thus regenerate it.

For a large target that might take several minutes to compute the tree dependencies, an up-to-date cache allows the build to get going in only a few seconds.

local.dirdeps.mk

Each time dirdeps.mk is read, it sets DEP_RELDIR and DEP_* for TARGET_SPEC_VARS (just MACHINE by default), in all but the 1st case, these are passed in via _DEP_TARGET_SPEC set by the previous dirdeps.mk.

It then reads local.dirdeps.mk which provides an opportunity to tweak DIRDEPS for the currently considered directory. This can make it feasible to share one set of Makefile.depend files for different architectures.

The main goal of local.dirdeps.mk is to avoid any temptation to touch dirdeps.mk.

Like dirdeps.mk, local.dirdeps.mk can use a test for the target _DIRDEP_USE to know wether we are being read for the first time or not.

We can set generic things like DIRDEPS_FILTER or we can set things like DIRDEPS_FILTER.host which will only be applicable when DEP_MACHINE is "host" (ie. we are building something for the build host itself).

For example:

.if !target(_DIRDEP_USE)
# first time here, do one off initialization

# never build these for "host"
SKIP_DIR.host = \
        Nlib/csu* \
        Nlib/libc

# CSU_DIR.i386 etc are set in *sys.mk if necessary
# and CSU_DIR = ${CSU_DIR.${MACHINE_ARCH}}
# but here we need it to be selected by DEP_MACHINE_ARCH
# these allow us to use ${CSU_DIR} in DIRDEPS
DEP_MACHINE_ARCH = ${MACHINE_ARCH.${DEP_MACHINE}}
CSU_DIR.${DEP_MACHINE_ARCH} ?= csu/${DEP_MACHINE_ARCH}
CSU_DIR = ${CSU_DIR.${DEP_MACHINE_ARCH}}

As above, variables used by dirdeps.mk are qualified with DEP_* which are reset for the current DEP_TARGET_SPEC we are processing.

If we were doing something really complicated like building for multiple TARGET_OS at the same time, we might want to have local.dirdeps.mk include something like local.dirdeps.${DEP_TARGET_OS}.mk to reset any complex filters needed:

.endif  # !target(_DIRDEP_USE)

# this section re-processed for each DEP_TARGET_SPEC

# pick up per TARGET_OS filters etc
.-include <local.dirdeps.${DEP_TARGET_OS}.mk>

Each time we read dirdeps.mk we reset DEP_DIRDEPS_FILTER and DEP_SKIP_DIR to include any variables qualified with DEP_*. For example:

DEP_SKIP_DIR = ${SKIP_DIR} \
        ${SKIP_DIR.${DEP_TARGET_SPEC}:U} \
        ${TARGET_SPEC_VARS:@v@${SKIP_DIR.${DEP_$v}:U}@} \
        ${SKIP_DIRDEPS.${DEP_TARGET_SPEC}:U} \
        ${TARGET_SPEC_VARS:@v@${SKIP_DIRDEPS.${DEP_$v}:U}@}