bmake

Introduction

Many years ago, when building large software projects, I've had to ensure that GNU make (if not my own patched version of it), was installed and had developed a set of macros to greatly simplify delveloping complex build trees.

Since the early 90's my main development machines, run BSD (NetBSD to be precise), and the BSD source tree is an excellent example of managing a large software project. In following NetBSD's -current delevlopment, I learned to use the new BSD Makefiles and make(1), I did an autoconf version of it that I'll refer to as bmake(1) from here on. Since then all my new projects and many of my old ones use bmake. Since bmake.tar.gz uses GNU's autoconf and is quite portable, I've skipped the effort of making the rest of my distributions support GNU make.

Bmake is derrived from NetBSD's make(1), its goal is to be a portable version of same, so new features are added via imports of NetBSD's make (I'm one of the contributors to NetBSD). Thus bmake is kept in sync with NetBSD's make.

Since about 2003, the bmake version typically represents the date of import from NetBSD, and I avoid any code changes which are not strictly related to portability.

It may be insteresting to note that I have projects where much of the tree uses the same simple Makefile:

PROG=	${.CURDIR:T}

SRCS=	${PROG}.c

.include "prog.mk"

macros

The example Makefile above, perhaps suggests why I like bmake(1). The important magic is in the line:
.include "prog.mk";
Makefiles for libraries inlcude lib.mk btw. Anyway, apart from reading a bunch of rules from the system macros (/usr/share/mk/* by default) it reads "../Makefile.inc" thus provinding a hook for much magic. My Makefile.inc files typically look for Makefile.${MACHINE}.inc as well. Though I'm converting to using Makefile.${HOST_TARGET}.inc and having HOST_TARGET set in sys.mk, as being more flexible.

Note that I used to use the bsd.*.mk macros (intended for building /usr/src), but attempts to get simple portability improvments committed into these, usually failed, so the only viable solution was to maintain a parallel set.

obj

Another cool feature of bmake(1) is that if the directory obj exists in the same directory as the Makefile, then bmake will chdir into it before doing anything else. This helps keep the build tree clean, and if the obj directories are symlinks off to a separate file system, eg.
/usr/src/bin/cat/obj -> /usr/obj/bin/cat
then suddenly building for multiple architectures is very easy.

Even better if MAKEOBJDIRPREFIX is set in the environment (see below) then it is trival to export a source tree read-only.

MAKEOBJDIRPREFIX

If MAKEOBJDIRPREFIX is set in the environment and the directory $MAKEOBJDIRPREFIX${.CURDIR} (.CURDIR is set by getcwd()) exists, make(1) will chdir into it rather than look for ./obj as described above. This simple feature allows true read-only source trees. This entire build tree has been built using MAKEOBJDIRPREFIX. Note that MAKEOBJDIRPREFIX is actioned before any Makefiles are read which is why it must be an environment variable.

Note that the latest versions of bmake can use the target .OBJDIR: to avoid having to have MAKEOBJDIRPREFIX in the environment, but I like to tightly control the build environment anyway so setting MAKEOBJDIRPREFIX is not an issue (for me).

While bmake normally overrides getcwd() with getenv("PWD") so that parallel makes work correctly with an automounter, this is dissabled (in bmake 3.x) when MAKEOBJDIRPREFIX is set to prevent $MAKEOBJDIRPREFIX${.CURDIR} having different values depending on how bmake is invoked.

Recent versions of bmake, also allow variable substitutions on MAKEOBJDIR so for a neater result you can use something like:

export MAKEOBJDIR="\${.CURDIR:S,$SRCTOP,$OBJTOP,}"

MACHINE and MACHINE_ARCH

The variables MACHINE and MACHINE_ARCH are built into bmake. The script bmake/machine.sh is used to set it it from the output of uname(1). You can override this value via configure's --with-machine=MACHINE. Do configure --help for a list.

Building bmake

All I do to build new version of bmake (on a non-NetBSD box) is:

$ tar zxf ~/bmake.tar.gz
$ tar zxf ~/mk.tar.gz
$ ./bmake/boot-strap
Note that I have the following content in ~/.bmake-boot-strap.rc

:
echo "Building for $HOST_TARGET"
case "$HOST_TARGET" in
netbsd*) sys_share_mk=:/usr/share/mk;;
*) sys_share_mk=
FORCE_BSD_MK=1
FORCE_SYS_MK=Generic
export FORCE_BSD_MK FORCE_SYS_MK
;;
esac
objdir=./$HOST_TARGET
case "$prefix" in
*/$HOST_TARGET) 
        CONFIGURE_ARGS="$CONFIGURE_ARGS --with-default-sys-path=$prefix/share/mk:`dirname $prefix`/share/mk$sys_share_mk"
        ;;
esac
For building bmake on NetBSD I use the system bsd.*.mk so the following is sufficient:

$ tar zxf ~/bmake.tar.gz
$ ./bmake/boot-strap -m none
The example below is building bmake-20091007 on Darwin. Note that the build isn't considered successful unless the unit-tests pass. With the above content in ~/.bmake-boot-strap.rc, boot-strap will print the commands to install into /homes/sjg/darwin9-i386/bin/ etc.

Also note that the src directories are left untouched and a host-target specific objdir is used. Thus I can use the exact same command sequence to build bmake for all the platforms available.

$ mkdir ~/tmp/x
$ cd ~/tmp/x
$ tar zxf ../bmake-20091007.tar.gz
$ tar zxf ../mk-20090908.tar.gz
./bmake/boot-strap
NOTE: reading /homes/sjg/.bmake-boot-strap.rc
Building for darwin9-i386
checking for gcc... gcc
checking for C compiler default output file name... a.out
...
checking if diff -u works... yes
checking for MACHINE & MACHINE_ARCH...
defaults: MACHINE=darwin9, MACHINE_ARCH=i386
Using: MACHINE=darwin9, MACHINE_ARCH=i386
Using: MKSRC=/homes/sjg/tmp/x/mk
configure: creating ./config.status
config.status: creating Makefile
config.status: creating makefile.boot
config.status: creating lst.lib/makefile.boot
config.status: creating unit-tests/Makefile
config.status: creating config.h
...
rm -f bmake *.[ado] */*.[ado] .*.done .depend
gcc -g -O2 -I. -I/homes/sjg/tmp/x/bmake -DHAVE_CONFIG_H	-D_PATH_DEFSYSPATH=\"/homes/sjg/darwin9-i386/share/mk:/homes/sjg/share/mk\"   -c -o arch.o /homes/sjg/tmp/x/bmake/arch.c
gcc -g -O2 -I. -I/homes/sjg/tmp/x/bmake -DHAVE_CONFIG_H	-D_PATH_DEFSYSPATH=\"/homes/sjg/darwin9-i386/share/mk:/homes/sjg/share/mk\"   -c -o buf.o /homes/sjg/tmp/x/bmake/buf.c
...
gcc *.o lst.lib/*.o -o bmake.boot
rm -f *.[ado] */*.[ado]
mkdir -p /homes/sjg/tmp/x/darwin9-i386/mk
cp -f autodep.mk auto.obj.mk dep.mk doc.mk dpadd.mk host-target.mk init.mk final.mk java.mk lib.mk libs.mk links.mk man.mk nls.mk obj.mk own.mk prlist.mk prog.mk progs.mk scripts.mk subdir.mk target-flags.mk warnings.mk yacc.mk AIX.sys.mk HP-UX.sys.mk Linux.sys.mk NetBSD.sys.mk OSF1.sys.mk SunOS.4.sys.mk SunOS.5.sys.mk Darwin.sys.mk IRIX.sys.mk OpenBSD.sys.mk Generic.sys.mk UnixWare.sys.mk /homes/sjg/tmp/x/darwin9-i386/mk
rm -f /homes/sjg/tmp/x/darwin9-i386/mk/sys.mk
ln -s Generic.sys.mk /homes/sjg/tmp/x/darwin9-i386/mk/sys.mk
chmod 644 autodep.mk auto.obj.mk dep.mk doc.mk dpadd.mk host-target.mk init.mk final.mk java.mk lib.mk libs.mk links.mk man.mk nls.mk obj.mk own.mk prlist.mk prog.mk progs.mk scripts.mk subdir.mk target-flags.mk warnings.mk yacc.mk
ln -s dep.mk bsd.dep.mk
ln -s doc.mk bsd.doc.mk
...
ln -s prog.mk bsd.prog.mk
ln -s subdir.mk bsd.subdir.mk
CC="gcc" LIBC= MAKEFLAGS= MAKESYSPATH=`pwd`/mk:/homes/sjg/darwin9-i386/share/mk:/homes/sjg/tmp/x/mk:/homes/sjg/darwin9-i386/share/mk:/homes/sjg/share/mk:/usr/share/mk:/usr/local/share/mk:/opt/share/mk ./bmake.boot -f Makefile
gcc -O2 -D_PATH_DEFSYSPATH=\"/homes/sjg/darwin9-i386/share/mk:/homes/sjg/share/mk\" -I. -I/homes/sjg/tmp/x/bmake -DHAVE_CONFIG_H	 -DMAKE_NATIVE	-DSIGNAL_FLAGS=SA_RESTART  -MD	 -c /homes/sjg/tmp/x/bmake/arch.c
...
gcc   -o bmake	arch.o buf.o compat.o cond.o dir.o for.o hash.o job.o main.o make.o parse.o str.o suff.o targ.o trace.o var.o util.o strlist.o make_malloc.o lstAppend.o lstAtEnd.o lstAtFront.o lstClose.o lstConcat.o lstDatum.o lstDeQueue.o lstDestroy.o lstDupl.o lstEnQueue.o lstFind.o lstFindFrom.o lstFirst.o lstForEach.o lstForEachFrom.o lstInit.o lstInsert.o lstIsAtEnd.o lstIsEmpty.o lstLast.o lstMember.o lstNext.o lstOpen.o lstRemove.o lstReplace.o lstSucc.o lstPrev.o sigcompat.o   -L.
CC="gcc" LIBC= MAKEFLAGS= MAKESYSPATH=`pwd`/mk:/homes/sjg/darwin9-i386/share/mk:/homes/sjg/tmp/x/mk:/homes/sjg/darwin9-i386/share/mk:/homes/sjg/share/mk:/usr/share/mk:/usr/local/share/mk:/opt/share/mk ./bmake.boot -f Makefile bmake.1
making bmake.1
CC="gcc" LIBC= MAKEFLAGS= MAKESYSPATH=`pwd`/mk:/homes/sjg/darwin9-i386/share/mk:/homes/sjg/tmp/x/mk:/homes/sjg/darwin9-i386/share/mk:/homes/sjg/share/mk:/usr/share/mk:/usr/local/share/mk:/opt/share/mk `pwd`/bmake -f Makefile test
cd /homes/sjg/tmp/x/darwin9-i386/unit-tests && /homes/sjg/tmp/x/darwin9-i386/bmake TEST_MAKE=/homes/sjg/tmp/x/darwin9-i386/bmake test
/homes/sjg/tmp/x/darwin9-i386/bmake -f Makefile > test.out 2>&1
diff -u /homes/sjg/tmp/x/bmake/unit-tests/test.exp test.out

Commands to install into /homes/sjg/darwin9-i386/

mkdir -p /homes/sjg/darwin9-i386/bin
cp ./darwin9-i386/bmake /homes/sjg/darwin9-i386/bin/bmake-20091007
rm -f /homes/sjg/darwin9-i386/bin/bmake
ln -s bmake-20091007 /homes/sjg/darwin9-i386/bin/bmake
mkdir -p /homes/sjg/share/man/cat1
cp /homes/sjg/tmp/x/bmake/bmake.cat1 /homes/sjg/share/man/cat1/bmake.1
FORCE_BSD_MK=1 FORCE_SYS_MK=Generic sh /homes/sjg/tmp/x/mk/install-mk /homes/sjg/share/mk
$
I just cut/paste those last few lines to install.


$Id: bmake.html,v 1.17 2009/10/29 18:38:29 sjg Exp $
Copyright © 1997-2001 CRUFTY.NET