Engineering · Part 8 · refuse to start rather than start wrong

Fail-closed configuration

platformsecurity

Reference documentation for a configuration discipline that refuses to start rather than start wrong: every required setting read from the environment with no default, no .env file, one reader, three sources, and a single trick for expressing "explicitly off" in a store that cannot hold an empty string.

Most configuration bugs are silent. A setting is missing, a default fills in, and the app runs — against the wrong database, with debug on, with an insecure key — until something downstream reveals the mistake, often in production, often expensively. The cure is to make a missing setting loud: refuse to boot. This document is the whole discipline; it is short because the idea is small and its value is in applying it without exception.

The reader: required means required

# settings.py
def require(key):
    val = os.environ.get(key)
    if not val:
        raise ImproperlyConfigured(
            f'{key} must be set — config is fail-closed with no defaults. '
            f'Locally: the task-runner env block. In prod: the parameter store.')
    return val

SECRET_KEY   = require('DJANGO_SECRET_KEY')
DEBUG        = require_bool('DJANGO_DEBUG')     # explicit EVERYWHERE — dev sets 1, prod sets 0
DATABASE_URL = require('DATABASE_URL')

Read every required setting through require(). There is no default, not even in development — that is the point. A defaulted SECRET_KEY is the canonical disaster: it works on every laptop and silently ships a known key to production. require() makes that impossible; a missing value is an ImproperlyConfigured at startup with a message that says where the value should come from. Note that even DEBUG is required and explicit — dev environments set it to 1, production sets 0, and neither inherits a framework default that could flip the wrong way.

No .env — three sources, one reader

A .env file is a fourth source of truth that drifts, gets committed by accident, and differs subtly between machines. Do without it. The same variables are stated in three places, each appropriate to its context, and all read by the one fail-closed reader:

flowchart LR
    M["local dev:
task-runner env block
(committed, dev-insecure)"]:::s --> R
    C["CI:
workflow env / secrets"]:::s --> R
    P["prod / staging:
parameter store → boot"]:::s --> R
    R["require() — one fail-closed reader
missing ⇒ refuse to start"]:::r
    classDef s fill:#eff6ff,stroke:#1d4ed8;
    classDef r fill:#fffbeb,stroke:#b45309;

• Local development states every variable in a block in the task runner (a Makefile), prefixed onto each command that boots the app. The values are deliberately dev-insecure and committed — a fake secret key, a local database URL — so a fresh checkout runs with zero setup, and nobody is tempted to invent a .env.
• CI states them in the workflow's environment and secrets — the same names, so a test that boots the app sees exactly the shape production will.
• Production and staging read them from a parameter store, materialised into the box's environment once at boot (Part 5). The box does no translation; what is in the store is what the app sees.

One reader, three sources, no file. The discipline that keeps it honest: when you add a required setting, you update all three at once — the task runner, the CI env, and the parameter store — and the app tells you if you forgot, because it will not start. A missing setting is never a mystery; it is an ImproperlyConfigured naming the exact key.

Tuning knobs are the deliberate exception

Not everything is required. Genuinely optional values — a feature that is off by default, a performance knob with a sensible value — read through a different helper, so the two categories never blur:

def tuning(key, default=None):                 # optional: a default that MEANS something
    return os.environ.get(key, default)

def tuning_bool(key, default=False):
    return tuning(key, str(int(default))).lower() in ('1', 'true', 'yes', 'on')

The rule is a clean dichotomy: if the app cannot run correctly without a value, require() it (no default); if the app has a correct behaviour when the value is absent, tuning() it (with a default that is that behaviour — usually "feature off"). Reading the settings file, you can see at a glance which settings are load-bearing and which are adjustable, because they call different functions. There is no third, ambiguous category.

The empty-string problem, and the sentinel

Fail-closed config collides with a mundane reality: many parameter stores cannot store an empty string, and yet "this optional feature is deliberately off" is exactly an empty value. If require() rejects empty and the store rejects empty, how do you express "present, but intentionally blank" — an off feature flag, an unset optional integration — for a setting you nonetheless want to be explicit rather than absent?

Reserve a sentinel: a literal string, say blank, that means "intentionally empty". The provisioning layer writes blank into the store for a deliberately-off value; the box's environment carries it verbatim; and the settings layer interprets it back to an empty string in exactly one place:

BLANK_SENTINEL = 'blank'

def deblank(val):
    """Map the explicit blank sentinel back to '' (a real blank is left as-is)."""
    return '' if val == BLANK_SENTINEL else val

STRIPE_SECRET_KEY = deblank(require('STRIPE_SECRET_KEY'))   # present, maybe 'blank' ⇒ off

Now the two properties coexist that otherwise conflict. require() still fails closed — the key must be present, so you cannot forget it. And "off" is expressible and explicit — the store holds the literal blank, which reads in review as "yes, we meant this off", not as an accident of a missing row.

Where the sentinel leads: values that can say "off"

The sentinel works, but it is a patch on a deeper mismatch: flat string variables have no way to say "nothing". Fourteen of our keys existed only to be blank in most environments — three environments times fourteen assignments carrying zero information — and the sentinel literal had to be kept in step across the settings layer and the provisioning scripts. The real fix is to stop using values too small to express the choice. This codebase now groups each feature into one JSON-valued variable, validated against a JSON Schema at boot:

DJANGO_BILLING_CONFIG='{"provider":"none"}'          # off — typed, validated, no sentinel
DJANGO_TOOLCHAT_CONFIG='{"backend":{"kind":"bedrock","model_id":"…"},
                         "budget":{"global_monthly_usd":100}}'

JSON can say "none", null, or {} natively, so "off" is a legal value rather than a reserved string; the store never needs to hold an empty field. The schema buys more than the sentinel ever did: conditional requiredness becomes structural (a lambda render backend without its queue URL is invalid as data — no hand-rolled if guards), and additionalProperties: false turns a typo'd tunable from a silently-ignored string into a boot failure. require() survives underneath — every variable is still mandatory, still no defaults — it just requires a document now, not a word.

Tests read the same settings

The test suite boots the real settings module — that is a feature, because a change that would break production config (a new require() key) breaks the tests first. So a new required setting is a three-place update plus the test environment, and the failing suite is the reminder. The one thing the test path overrides is deliberately weak, not absent: a fast password hasher for speed (Part 3). It never relaxes require() — tests prove the fail-closed reader works, rather than routing around it.

The shape to copy

1. Every required setting through require() — no default, not even in dev; a missing value refuses to start with a message naming the key.
2. No .env: the same names in the task runner (committed, dev-insecure), in CI, and in a parameter store read at boot — one reader for all three.
3. Optional values through a separate tuning() helper whose default is the absent-behaviour; the two helpers keep required and optional visibly distinct.
4. A reserved blank sentinel, interpreted in exactly one place, so "explicitly off" survives a store that cannot hold an empty string without weakening fail-closed.
5. Tests boot the real settings, so a config regression fails the suite before it reaches production.

Part 9 turns to the front end: rich, live UI from server-rendered HTML and htmx — no client framework, no build step, and the effects stream from Part 1 driving live updates.

← All posts