Here's a script I use to speed up compilation at work. Since I work at Collabora, I call it collaboratively. It's a prefix to a command (used in the same way as sudo or nice) which sets up icecc (Icecream) and ccache to work nicely together.

#!/bin/sh
# ~/bin/collaboratively
# Typical usage: collaboratively make check
PATH=/usr/lib/ccache:/usr/lib/icecc/bin$(echo :$PATH | sed -e s@:/usr/lib/ccache@@g)
export PATH
MAKEFLAGS='-j -l3'
export MAKEFLAGS
exec "$@"

Some more explanation of what it does:

• First, I want the gcc wrappers (actually symlinks to ccache) in /usr/lib/ccache to be invoked. This uses ccache to make sure I don't actually recompile source code that hasn't changed.

• Next, I want to use icecc, which distributes builds around the office. We run an icecc scheduler on the office server, which also contributes two CPU cores and lots of RAM to the compilation effort.

• To prevent infinite recursion between icecc and ccache, I have to make sure ccache isn't in my $PATH for a second time! My bash and zsh dotfiles automatically set up ccache, so in practice it does need editing out.

• When distributing builds, make should be parallelizing as many compiles as feasible - I use the flags -j -l3 to run an unlimited number of parallel compilation processes, but stop when the load average reaches 3 (the number of CPU cores I have, plus 1, seems a reasonable maximum load-average).

apt-get install icecc-monitor and run icemon to see whether this is all working - if it is, you should see compile jobs going off to other hosts.