Sandbox tools
*************

As far as I know, the concept of a sandbox and the tools ``mksb`` and
``mk`` come from the OSF Development Environment (ODE).

The scripts described here implement the idea but not necessarily in a
compatible manner. 


Introduction
============

A *sandbox* is simply a software source tree with a special
file ``.sandbox-env`` which serves both to mark the location of
the sandbox (the variable ``SB`` is set to the directory where
``.sandbox-env`` was found) and provide a set of environment variables
for it.

Below we describe some of the tools that work within this ecosystem.
I've been using this setup for over 15 years and find it very useful.

mk
--

This is the most commonly used tool.
In a nutshell its job is to condition the environment and run
``${REAL_MAKE:-make}``.

In this implementation it is sourced by both 
mksb_ and workon_ to handle environment
setup in a consistent manner, each sets ``MYNAME`` to its canonical
name.

The following files are read (if they exist) in this order::

	$Mydir/sb-env.rc
	$Mydir/$MYNAME.rc
	$HOME/.sandboxrc

Any of the above can contribute hook functions called via run_hooks_
to control the behavior.
For example::

	. $Mydir/os.sh
	. $Mydir/test_opt.h

	# set test_L to -L or -h
	test_opt L -h

	add_hooks mk_target_machine_hooks mk_target_machine_xtra

	mk_target_machine_xtra() {
	    case "$MACHINE" in
	    host32) OBJTOP=$OBJROOT/$HOST_TARGET32;;
	    host)  OBJTOP=$OBJROOT/$HOST_TARGET;;
	    esac
	    :
	}

	add_hooks ${MYNAME}_init_hooks sb_project

	# canonicalize SB_PROJECT
	sb_project() {
	    _p=${1:-$SB_PROJECT}

	    case "$_p" in
	    netbsd*|NetBSD*)
	        SB_PROJECT=NetBSD
		# other setup....
	        ;;
	    esac
	    :
	}


results in each app calling ``sb_project`` early
and for ``mk`` calling ``mk_target_machine_xtra``
from ``mk_target_machine``.

The last ``:`` just ensures a happy return code (``return 0`` would
work too), since by default ``run_hooks`` stops if any hook returns
non-zero.
This is handy when using ``run_hooks`` to allow customizing
command line options, if a hook consumes an argument it returns non-zero
and the search stops.

After the global rc files have been read we run::

	run_hooks ${MYNAME}_begin_hooks

Next ``mk`` searches upwards from the current directory for the file 
``.sandbox-env``.
If found, its location defines ``SB``.
If not found ``mk`` throws an error.

Once ``SB`` is known, it reads (if they exist)::

	$SB/../.sandboxrc
	$SB/.sandboxrc

We then run::

	run_hooks ${MYNAME}_init_hooks

and then read::

	$SB/.sandbox-env

After all the rc files have been read, we run more hook functions::

	run_hooks ${MYNAME}_setup_hooks
	run_hooks ${MYNAME}_finish_hooks

Some apps run others as well.
For example ``mk`` runs ``mk_cmd_hooks`` after ``mk_finish_hooks``

Finally ``mk`` runs ``mk_run_make`` which runs ``${REAL_MAKE:-make}``
passing it ``$MK_MAKEFLAGS`` and any command line args.

You can think of ``mk_run_make`` as the last member of
``mk_cmd_hooks``, it will not be reached if one of those hooks execs
something.

options
~~~~~~~

With the following exceptions, anything on the ``mk`` command line
is expected to be consumed by ``$REAL_MAKE``.

If any of these options are used, they *must* appear before anything else:

``--doc``
	Calls the ``__doc`` function.  The default implementation in
	``sb-funcs.sh`` simply displays the documentation embedded in
	the script.

``--help[=topic]``

	Calls the ``__help`` function. The default implementation runs
	any ``${MYNAME}_help_hooks``, if none found it calls 
	``${MYNAME}_help`` if it exists and if all else
	fails calls ``__doc``.

	If an argument is provided it is treated as a *topic* and refines the
	functions we look for.

	See ``mk --doc`` for details.

``--machine [MAKELEVEL,]MACHINE[,...]``

	Calls ``mk_target_machine`` to set ``MACHINE`` to the value
	provided.

	If the tupple provided starts with a numeric it is used to set
	``MAKELEVEL`` and the remainer used for ``MACHINE``, and if it
	is also a tupple we set ``TARGET_SPEC`` to the full tupple
	while ``MACHINE`` is set to just the first element.

	It also sets ``REQUESTED_MACHINE`` so that the build can tell
	the difference between this case and a default value of ``MACHINE``.

	The same result happens if ``mk`` is invoked as ``mk-$MACHINE``.


mksb
----

The ``ev`` (``.sandbox-env``) file can be created manually of course.
Or it can be created by a tool; ``mksb`` which can also handle initial
check-out of the source tree.

Using ``mksb`` we can hide details like the SCM used by a given
project, or the URL of the repository, etc.
This allows those to be changed over time, without the need for users
(or robots) to change their operations.
When you have 1000's of developers and dozens of projects that's
very handy.

Note that ``mksb`` itself is just an engine, it is told the
``SB_PROJECT`` that a sandbox is being created for and a set of site
specific `hook functions`_ do the actual work.

Apart from running lots of hooks, ``mksb`` provides a number of
functions to help populate ``.sandbox-env``:

``expShellVars VAR ...``

	Simply adds ``VAR=value; export VAR``
	
``expShellDefVars VAR ...``

	Like ``expShellVars`` but adds ``VAR=${VAR:-value}; export VAR``

``expShellLiteralVars VAR ...``

	Like ``expShellVars`` but uses single quotes.
	This is useful for cases like:
	``MAKEOBJDIR='${.CURDIR:S,${SRCTOP},${OBJTOP},}'``
	which is to be interpreted by 'bmake' rather than
	the shell.

The hooks are run in several stages so that earlier ones can
influence later ones::

	run_hooks mksb_begin_hooks
	run_hooks mksb_pre_create_hooks

now we actually create ``$SB``, then::

	run_hooks mksb_init_hooks
	run_hooks mksb_setup_hooks

and initialize ``.sandbox-env``, then::

	run_hooks mksb_env_init_hooks
	run_hooks mksb_env_setup_hooks
	run_hooks mksb_env_hooks
	run_hooks mksb_env_finish_hooks

at this point ``.sandbox-env`` is generally complete and read
by ``mksb``, then we run::

	run_hooks mksb_checkout_init_hooks
	run_hooks mksb_checkout_setup_hooks
	run_hooks mksb_checkout_hooks
	run_hooks mksb_checkout_finish_hooks
	run_hooks mksb_finish_hooks

run_hooks
~~~~~~~~~

.. _`hook functions`:

The file ``hooks.sh`` provides and api for defining lists of hook functions
and running them at appropriate times.

``add_hooks list`` *function* [...]

	Adds *function* to ``$list``

``run_hooks list [LIFO]`` [*args*]

	Run each of the functions in ``$list`` passing them any *args*.

	Processing stops if a function returns !0

	If the first arg is ``LIFO`` the list is run in reverse order.

``run_hooks_all list [LIFO]`` [*args*]

	As for ``run_hooks`` but all functions are run regardless of
	return values.

options
~~~~~~~

Because ``mksb`` is a much more complicated script than say ``mk``
it uses ``eval_args.sh`` to process its command line.

Apart from supporting both long and short options, ``eval_args.sh``
also handles ``VAR=VALUE`` assignments - with special handling if needed.
	
``--expShellVars=VAR[=value] --expShellDefVars=VAR[=value] --expShellLiteralVars=VAR[=value]``

	As described above for ``expShellVars`` etc.
	If no *value* is provided, it is assumed to have been set
	already on the command line.

sb-opt.sh
~~~~~~~~~

This provides support for options like ``--sb-opt-KNOB={yes,no}``.

It creates a file ``sbopt-KNOB.inc`` and arranges for
``$SB/.sandboxrc`` to include it.

If the arg is ``yes`` then ``sbopt-KNOB.inc`` will contain ``export
WITH_KNOB=1`` and ``export WITHOUT_KNOB=1`` for ``no``.

This works in conjunction with our ``options.mk`` or FreeBSD's
``bsd.mkopt.mk`` to set ``MK_KNOB={yes,no}``

``sb-opt.sh`` can be used at any time to update ``sbopt-*.inc``.


workon
------

This is also an optional tool, though I find it very handy.
::

	$ workon NetBSD/current

will find the sandbox called ``NetBSD/current`` chdir into it,
set the environment up (as per mk_) and run my shell.
I can then make use of variables like ``$SB`` and others setup by
``.sandbox-env``.

I have ``mksb`` export the sandbox name in a variable that will be
included in my shell prompt::

	tty_tag() {
        	# I use this in the xterm title bar (among other things)
        	TTY_TAG="[\$SB_NAME]"
        	expShellVars TTY_TAG
	}

	add_hooks mksb_env_finish_hooks tty_tag

You can also use it as a one command thing::

	$ workon FreeBSD/current command

will do as above, but instead of running my shell, it will run
``command`` and exit when it does.


Sandbox environment
===================

There is only one variable that every sandbox has; ``SB`` which is set
by tools like ``mk`` when they find the magic ``.sandbox-env`` marker.
Everything else is totally up to you.
Ok we also set ``SB_NAME`` to the basename of ``SB``.

For projects that use bmake_, I typically have::

	export OBJROOT=$SB/obj/
	export SRCTOP=$SB/src
	export OBJTOP='${OBJROOT}${MACHINE}'
	export MAKEOBJDIR='${.CURDIR:S,${SRCTOP},${OBJTOP},}'
	export MAKESYSPATH=$SRCTOP/share/mk

Note that ``OBJTOP`` and ``MAKEOBJDIR`` are single quoted so that 
they are seen like that by ``bmake``.

This allows us to ensure that each build uses the correct
``share/mk/*.mk`` files and we get nice neat object dirs outside of
the src tree without the ugliness that ``MAKEOBJDIRPREFIX`` results
in.

Such a setup even works well when shared via NFS.
Though in that case we might make ``SB_OBJROOT`` point to a local
filesystem for better performance.

For a project that used ``gmake`` I might have::

	export SRCTOP=$SB/src
	export GMKSYSDIR=$SRCTOP/gmk
	export MAKEFILES=$GMKSYSDIR/sys.gmk
	REAL_MAKE=gmake

assuming it used my http://www.crufty.net/ftp/pub/sjg/gmk.tar.gz
(see `gmake-dirdeps`_)

Download
========

An implementation of the above can be found at 

http://www.crufty.net/ftp/pub/sjg/sb-tools.tar.gz

------

.. _bmake: /help/sjg/bmake.htm
.. _`gmake-dirdeps`: gmake-dirdeps.htm

:Author: sjg@crufty.net
:Revision: $Id: sb-tools.txt,v 22f4b2154e88 2020-09-12 21:26:21Z sjg $
:Copyright: Crufty.NET
