OneBug All write-ups

Python ·

Python dict assignment doesn't copy - it aliases the dict

config = defaults binds a second name to the same dict, so an update() writes into your defaults. How to spot aliasing and copy a dict properly.

Test yourself before reading - the write-up below gives it away.

Spot the bug Python
1
# Build a per-environment config on top of the defaults.
2
def with_overrides(base, overrides):
3
    merged = base
4
    merged.update(overrides)
5
    return merged
6
 
7
DEFAULTS = {"retries": 3, "debug": False}
8
prod = with_overrides(DEFAULTS, {"debug": True})
9
print(DEFAULTS["debug"])
10
# expected: False
11
# actual:   True  (!)

The debug flag that turned itself on everywhere

One deploy later, every environment was logging at debug level - including the two that never asked for it.

# Build a per-environment config on top of the defaults.
def with_overrides(base, overrides):
    merged = base
    merged.update(overrides)
    return merged

DEFAULTS = {"retries": 3, "debug": False}
prod = with_overrides(DEFAULTS, {"debug": True})
print(DEFAULTS["debug"])
# expected: False
# actual:   True  (!)

Why does changing the "copy" change the original dict?

Because there is no copy. merged = base binds a second name to the same dict object - assignment in Python never duplicates anything. merged.update(overrides) therefore writes straight into DEFAULTS, and every config built afterwards inherits debug: True. The fix: make the copy explicit - merged = dict(base) - before mutating.

One dict, two names

Python variables are names bound to objects, never boxes holding values - Ned Batchelder's Facts and myths about Python names and values is the canonical walkthrough. After line 3 both names refer to one object, which id() will confirm:

>>> merged = base
>>> id(merged) == id(base)
True

Line 4, the update, is the line everyone stares at, and it would be correct on a private copy - the defect is one line up, where the private copy was supposed to be made and a mere rebinding happened instead. The copy module docs open with this exact distinction: "Assignment statements in Python do not copy objects, they create bindings between a target and an object."

What makes it nasty is that the first call returns the right answer. prod has debug: True, as requested. The damage lands on whoever reads DEFAULTS after this call - so the symptom shows up far from the cause, in code that did nothing wrong. A test that calls with_overrides once and checks the return value passes. A suite that happens to run the defaults-consuming test first also passes. Reorder the tests and it fails - the classic signature of shared mutable state, and worth remembering next time a suite goes red only under pytest -p no:randomly.

The staging outage that was really a dict

A team runs build_config("staging") in a request handler. Under the hood, each call layers overrides onto module-level defaults exactly like the snippet. The first staging request after boot works. From then on the defaults carry staging's database URL, so the next environment to boot from the same module - a worker, a cron job - quietly connects to staging's database. The postmortem takes a day because everyone reads build_config and sees a function that "returns a new config". It returns the old one, renamed.

The fix - and how deep it needs to go

def with_overrides(base, overrides):
    merged = dict(base)
    merged.update(overrides)
    return merged

dict(base), base.copy(), or on Python 3.9+ the merge operator, which builds the new dict in one step (PEP 584):

return base | overrides

The half-fix to watch for: a shallow copy duplicates only the top level. If your defaults contain a nested dict - {"db": {"host": ..., "pool": 5}} - then dict(base) gives you a new outer dict whose "db" key still points at the one shared inner dict, and merged["db"]["pool"] = 50 is this bug all over again, one level down. For nested config, reach for copy.deepcopy(base) and pay the copy cost once, at startup.

How to find dict aliasing in your codebase

  • Grep for functions that mutate their parameters: grep -rnE '\b(def [a-z_]+\([^)]*\))' -A 5 --include='*.py' . | grep -E '\.(update|pop|setdefault)\(' is crude but surfaces candidates fast.
  • The debugging tool is id(): when a value changes "by itself", print id() of the two dicts you believed were independent.
  • Module-level mutable constants (DEFAULTS = {...}) deserve a comment or a MappingProxyType wrapper - types.MappingProxyType(DEFAULTS) makes accidental writes raise a TypeError instead of corrupting state.

Related bugs

Aliasing has a whole family here: a list shared through a class attribute, a list shared through a mutable default argument, and in Go, two slices sharing one backing array. Hub: bug write-ups.

with_overrides was never a bad name. It just needed one honest dict() call to live up to it.