Blog

Making websites with Make: a comparison of build tools


JavaScript task runners / build tools have never appealed to us. Streams (piping) can be done with shell scripts with more flexibility, and the declarative approach of nested JSON configuration is horrible to maintain. We had been using a mixture of NPM scripts and Bash scripts, which worked ok but build times were slow, until watching this presentation by Rob Ashton and discovering that Make is really rather lovely.

Now build times are fast thanks to Make only building files that have changed and compiling branches of the dependency tree in parallel. Make's declarative approach combined with pattern matching just feels right. Most of all, Make is fun. It's not often that you get to write code like this:

@cat $^ > $@

Make is a small language so the esoteric syntax isn't really hard to grasp.

Good Makefile tutorials covering general principles are hard to find, most being too short and focussed on C/C++ details, but Rob has an accompanying blog post and git repo which are a great place to start. The GNU make manual is also rather good.

The basic process is to first gather your input file paths and work out your output file paths. For example:

HTML_INPUTS = $(wildcard src/*.html)
HTML_OUTPUTS = $(patsubst src/%,dist/%,$(HTML_INPUTS))

The above stores a list of HTML input file paths in the HTML_INPUTS variable, then creates a list of HTML output file paths by pattern matching; replacing 'src/' with 'dist/' in the list of inputs.

Then work backwards, starting with your outputs, write rules to create them from their dependencies, and their dependencies from their dependencies' dependencies and so on. A rule consists of target(s) (outputs) followed by a colon and their prerequisites (dependencies) on one line. Subsequent tab indented lines define a recipe to make the targets from the prerequisites.

dist/%.html: src/%.html
    @cp $< $@

This states that the HTML files in dist/ depend on the HTML files in src/, and the recipe simply copies files from one to the other.

Caveat: we don't claim this to be a particularly good or idiomatic example of a Makefile, but for what it's worth here's roughly how we use Make to build JavaScript, SASS, HTML, images and fonts.

########## CONFIGURATION ##########

PATH := ./node_modules/.bin:$(PATH)
BUILD := development

# Directories

INPUT_DIR := src
BUILD_PARENT_DIR := build
BUILD_DIR = $(BUILD_PARENT_DIR)/$(BUILD)
BUILD_DIRS = $(BUILD_DIR) $(SCSS_BUILD_DIR) $(JS_BUILD_DIR)
OUTPUT_DIR := dist
OUTPUT_DIRS = $(OUTPUT_DIR) $(IMG_OUTPUT_DIR) $(FONT_OUTPUT_DIR)
TARGET_DIRS = $(BUILD_DIRS) $(OUTPUT_DIRS)
INSTALL_DIR :=
SCSS_INPUT_DIR = $(INPUT_DIR)/scss
SCSS_BUILD_DIR = $(BUILD_DIR)/css
JS_MODULE_INPUT_DIR = $(INPUT_DIR)/js/node_modules/modules
JS_LIBRARY_INPUT_DIR = $(INPUT_DIR)/js/node_modules/lib
JS_BUILD_DIR = $(BUILD_DIR)/js
HTML_INPUT_DIR = $(INPUT_DIR)/html
FONT_INPUT_DIR = $(INPUT_DIR)/fonts
FONT_OUTPUT_DIR = $(OUTPUT_DIR)/fonts
IMG_INPUT_DIR = $(INPUT_DIR)/img
IMG_OUTPUT_DIR = $(OUTPUT_DIR)/img

# Map SCSS inputs to outputs

SCSS_INPUTS = $(wildcard $(SCSS_INPUT_DIR)/*.scss)
COMPILED_SCSS = $(patsubst $(SCSS_INPUT_DIR)/%.scss,$(SCSS_BUILD_DIR)/%.css,$(SCSS_INPUTS))
CSS_OUTPUTS = $(OUTPUT_DIR)/app.css

# Map Javascript inputs to outputs

JS_INPUTS = $(wildcard $(JS_MODULE_INPUT_DIR)/*/index.js)
COMPILED_JS = $(patsubst $(JS_MODULE_INPUT_DIR)/%,$(JS_BUILD_DIR)/%,$(JS_INPUTS))
JS_OUTPUTS = $(OUTPUT_DIR)/app.js

# Map HTML inputs to outputs

HTML_INPUTS = $(wildcard $(HTML_INPUT_DIR)/*.html)
HTML_OUTPUTS = $(patsubst $(HTML_INPUT_DIR)/%,$(OUTPUT_DIR)/%,$(HTML_INPUTS))

# Map font inputs to outputs

FONT_INPUTS = $(wildcard $(FONT_INPUT_DIR)/*)
FONT_OUTPUTS = $(patsubst $(FONT_INPUT_DIR)/%,$(FONT_OUTPUT_DIR)/%,$(FONT_INPUTS))

# Map image inputs to outputs

IMG_INPUTS = $(wildcard $(IMG_INPUT_DIR)/*)
IMG_OUTPUTS = $(patsubst $(IMG_INPUT_DIR)/%,$(IMG_OUTPUT_DIR)/%,$(IMG_INPUTS))

# Assemble outputs

OUTPUTS = node_modules $(JS_OUTPUTS) $(HTML_OUTPUTS) $(CSS_OUTPUTS) $(FONT_OUTPUTS) $(IMG_OUTPUTS)

# Helper functions

COMPILE_MSG = @printf "Compiling $^\n--> $@\n\n"

# Compilation commands

ifeq ($(BUILD),development)
define BROWSERIFY_CMD =
@browserify --standalone $$(basename $$(dirname $<)) --debug $< > $@
endef
NODESASS_CMD = @node-sass $< | postcss -u autoprefixer > $@
else ifeq ($(BUILD),production)
BROWSERIFY_CMD = @browserify --standalone $$(basename $$(dirname $<)) -g uglifyify $< | uglifyjs -c warnings=false -o $@
NODESASS_CMD = @node-sass $< | postcss -u autoprefixer | cleancss --skip-rebase -o $@
else
$(error Unrecognised build type $(BUILD))
endif

########## RULES ##########

# Top level

.PHONY: all install clean

install: all
    @if [ -z $(INSTALL_DIR) ]; then printf "Missing argument: INSTALL_DIR\n"; exit 1; fi
    @mkdir -p $(INSTALL_DIR)
    rsync -avc --delete $(OUTPUT_DIR)/ $(INSTALL_DIR)/

clean:
    @printf "Deleting directories: $(OUTPUT_DIR) $(BUILD_PARENT_DIR)\n\n"
    @rm -rf $(OUTPUT_DIR) $(BUILD_PARENT_DIR)

all: $(TARGET_DIRS) $(OUTPUTS)
    @echo > /dev/null

$(TARGET_DIRS):
    @printf "Creating directories: $(TARGET_DIRS)\n\n"
    @mkdir -p $(TARGET_DIRS)

# Node modules

node_modules: package.json
    npm install

# CSS/SCSS

$(OUTPUT_DIR)/app.css: $(COMPILED_SCSS)
    $(COMPILE_MSG)
    @cat $^ > $@

$(SCSS_BUILD_DIR)/%.css: $(SCSS_INPUT_DIR)/%.scss
    $(COMPILE_MSG)
    $(NODESASS_CMD)

# Javascript

$(OUTPUT_DIR)/app.js: $(COMPILED_JS)
    $(COMPILE_MSG)
    @cat $^ > $@

$(JS_BUILD_DIR)/widgetA/index.js : $(JS_MODULE_INPUT_DIR)/widgetA/index.js  $(JS_LIBRARY_INPUT_DIR)/domready/index.js node_modules
    $(COMPILE_MSG)
    @mkdir -p $(@D)
    $(BROWSERIFY_CMD)

$(JS_BUILD_DIR)/widgetB/index.js : $(JS_MODULE_INPUT_DIR)/widgetB/index.js  $(JS_LIBRARY_INPUT_DIR)/domready/index.js node_modules
    $(COMPILE_MSG)
    @mkdir -p $(@D)
    $(BROWSERIFY_CMD)

# HTML

$(OUTPUT_DIR)/%.html: $(HTML_INPUT_DIR)/%.html
    $(COMPILE_MSG)
    @cp $< $@

# Fonts

$(FONT_OUTPUT_DIR)/% : $(FONT_INPUT_DIR)/%
    $(COMPILE_MSG)
    @cp $< $@

# Images

$(IMG_OUTPUT_DIR)/% : $(IMG_INPUT_DIR)/%
    $(COMPILE_MSG)
    @cp $< $@

With regard to JavaScript, there's a tradeoff between using CommonJS / AMD / ES6 modules and a bundler to compile dependencies and using your Makefile. For example, if module A depends on libraries B and C, and B changes, if you're just feeding A into your bundler then A, B, and C will be recompiled even though C doesn't need to (shorter configuration, longer compilation). On the other hand your Makefile could have recipes for building all three and concatenate the results, in which case just A and B will be recompiled (longer configuration, shorter compilation).

Advantages of using Make over JavaScript task runners/build tools

  1. Make is a general purpose, unopinionated build tool which can be used to build everything, easily, which is great if your project includes Python/PHP/whatever.
  2. Make is just a build tool, there's no conflation with task running.
  3. No JavaScript tooling fatigue. People have been building things with Make for over 40 years. The same will not be said of Grunt -> Gulp -> WebPack -> Brunch | Rollup -> the next hotness. Trying to keep up with JavaScript tooling is a huge time sink.
  4. No breaking API changes. Your Makefile will still work ten years from now, we have no confidence that this will be the case for your JavaScript build tool configuration. We like stability when it comes to build tools and server management tools.

Common complaints

But Make can't inline your SVGs and pass your PNGs through a Gaussian filter constructed from atmospheric noise whilst making you a cup of tea.

Nor can your JavaScript build tool / task runner. It depends on external NPM packages to actually do stuff. A Makefile would use the same NPM packages, with the advantage that it wouldn't need to wrap them with a plugin and be tied to a particular version.

Make isn't portable, QED.

To state the obvious, if your team doesn't use Windows then this doesn't actually matter. If they do, then it's the usual tradeoff between greater software choice/quality and greater compatibility. Or run Linux in Windows, of which there are many ways.

UPDATE 06/07/2018: See our followup on how to watch for file system changes.


By Spritely Design