OneBug All write-ups

Python ·

Python lambda in loop returns the same value (late binding)

Three handlers built in a loop all format the 503 label instead of 404, 500, 503. Python closures read the loop variable at call time. Here is the fix.

Test yourself before reading - the write-up below gives it away.

Spot the bug Python
1
# One label formatter per HTTP status.
2
codes = [404, 500, 503]
3
labels = {404: "not found", 500: "server error", 503: "unavailable"}
4
handlers = []
5
for code in codes:
6
    handlers.append(lambda: f"{code} {labels[code]}")
7
print([h() for h in handlers])
8
# expected: ['404 not found', '500 server error', '503 unavailable']
9
# actual:   ['503 unavailable', '503 unavailable', '503 unavailable']  (!)

A monitoring dashboard needed one message formatter per error code, so someone built the three of them in a loop and stored them in a list. The unit test called each formatter and compared the strings, and it went green. On the live dashboard every panel, the 404 one and the 500 one included, rendered "503 unavailable".

Here is the loop that built the handlers:

# One label formatter per HTTP status.
codes = [404, 500, 503]
labels = {404: "not found", 500: "server error", 503: "unavailable"}
handlers = []
for code in codes:
    handlers.append(lambda: f"{code} {labels[code]}")
print([h() for h in handlers])
# expected: ['404 not found', '500 server error', '503 unavailable']
# actual:   ['503 unavailable', '503 unavailable', '503 unavailable']  (!)

Why does a lambda in a loop return the same value every time?

Line 6 is the bug. The lambda closes over the name code, not the number code held at that moment. Python resolves code when the lambda runs, not when it is created, and by the time you call the handlers the loop has finished with code bound to its last value, 503. All three share one variable, so all three read 503 and format the same label. The busy labels dict on line 3 is the natural thing to blame and it is correct; bind the value per iteration instead with lambda code=code: f"{code} {labels[code]}".

Python looks up free variables at call time, not definition time

A lambda that mentions a name it does not define has a free variable, and the closure keeps a reference to the enclosing scope, not a snapshot of it. When h() runs, the interpreter walks out to that scope and reads code right then. The three lambdas on line 6 all point at the same code in the same function, and the Python execution model spells out why they must: a name refers to one binding in its block, and every reference resolves to whatever that binding holds at the time of the reference.

The loud line is the labels dict on line 3, the one that looks like it could map a code to the wrong string, and it is perfectly correct. The loop on line 5 rebinds code on each pass exactly as it should. The trap is quiet: the lambda saved on line 6 borrows that one shared name and never takes a copy.

Here is the same late binding in a one-liner, so the shared reference is easy to see:

funcs = [lambda: code for code in [404, 500, 503]]
print([f() for f in funcs])   # [503, 503, 503] - each closure read the final code
# print(code) here would raise NameError: the comprehension keeps code in its own scope

It stays hidden because the natural test calls each lambda while the loop is still on that iteration. Write for code in codes: assert make(code)() == f"{code} {labels[code]}" and it passes, because you invoke the closure before code moves on. Storing the lambdas and calling them later is a different timeline, and only the second one is buggy. Fixtures that exercise the factory in place will never reproduce what production does.

One dashboard that followed 503 all the way down

Trace it forward from the panel. The dashboard maps each error code to a colored tile with a click-through to the matching runbook. Because every formatter returned "503 unavailable", every tile linked to the 503 runbook, so an on-call engineer paged for a 500 opened the wrong page, found nothing that matched, and assumed the alert was noise. The 404 tile did the same. The codes on the tiles were correct; those came from a static config. Only the human-readable line under each tile ran through the loop, so the mismatch looked like a copy tweak gone wrong rather than a scoping bug, and nobody grepped the closure for a week.

The fix: bind the value as a default argument

Give the lambda its own parameter with a default, and capture the current code there:

for code in codes:
    handlers.append(lambda code=code: f"{code} {labels[code]}")

Default arguments are evaluated once, at the moment def (or lambda) runs, so code=code copies this iteration's number into the function's own local before the loop moves on. Each handler now carries its own value instead of sharing the loop's. functools.partial(lambda code: f"{code} {labels[code]}", code) does the same by freezing the argument, if you prefer it read as an explicit bind.

The tempting wrong fix is to swap the lambda for a named nested function, on the hunch that a real def captures more eagerly:

for code in codes:
    def handler():
        return f"{code} {labels[code]}"     # still late-binds; same bug
    handlers.append(handler)

That changes nothing. def and lambda build closures the same way, so handler still looks code up at call time and still reads 503. The other red herring is blaming Python 3's comprehension scope. A list comprehension does get its own scope, which is why the one-liner above does not leak in the way a bare loop would, but a closure created inside a comprehension still late-binds its free variables. Scope decides where the name lives; it does not decide when the name is read.

How to spot late-binding closures in your codebase

Find lambdas first, then check each for a loop variable used without a default:

grep -rnE 'lambda[^:]*:' --include=*.py .

For every hit, ask whether the lambda mentions a name that a surrounding for rebinds, and whether the default-argument slot is empty. A closure that reads the loop variable and stores itself for later is the shape that bites; a closure called immediately inside the loop is fine. Nested defs inside loops deserve the same read, since grep for lambda will skip them. Comprehensions get their own scope, so they will not leak the name outward, but any function you build inside one is still on the hook for the same late lookup.

Related bugs

This is the loop not doing what the eye reads, the same family as changing a list while you iterate over it, where the loop and the data disagree mid-pass. It is also a scoping surprise cut from the same cloth as a mutable class attribute shared across instances, another case of one object standing in for many. The exact same closure trap shows up one language over in Go closures capturing a shared loop variable. More live on the Python bug hub, with the rest at bug write-ups.

Tomorrow, grep your handlers for a lambda inside a for, and read the line after the colon: if it names the loop variable and there is no = binding it, that closure is holding a reference, not a number.