Remove items from a list while iterating in Python (fix)

The cleanup that left a pending task behind

The cleanup function reported success. The queue it was told to empty still held a pending task. Two tasks were pending on the way in; one came back out untouched, as though it had never matched the filter at all.

# Drop every pending task from the queue.
def clear_pending(tasks):
    for t in tasks:
        if t["status"] == "pending":
            tasks.remove(t)
    return tasks

queue = [
    {"id": 1, "status": "done"},
    {"id": 2, "status": "pending"},
    {"id": 3, "status": "pending"},
    {"id": 4, "status": "done"},
]
print([t["id"] for t in clear_pending(queue)])
# expected: [1, 4]
# actual:   [1, 3, 4]  (!)

Task 3 was pending. It walked out with the survivors.

Why does removing items from a list while iterating skip elements?

Line 3, for t in tasks, iterates the same list that line 5 is deleting from. A for loop tracks its position with an internal integer index, and list.remove shifts every later element one slot to the left. The element sitting after a removed one slides into an index the loop already passed, so it never gets checked. Iterate a copy instead: for t in list(tasks).

The loop counts by index; remove renumbers the list underneath it

The guilty line is line 3. A Python for over a list holds an iterator that remembers one thing: how far along it is, as a plain counter starting at 0. It does not remember object identities, and it does not get told when the list changes shape. Each step, it hands you tasks[i] and bumps i by one.

Watch the counter and the list move together:

i = 0 reads tasks[0], id 1, status done. Nothing happens. i becomes 1.
i = 1 reads tasks[1], id 2, status pending. remove deletes it. The list is now [id1, id3, id4] and id 3 has slid down into index 1. i becomes 2.
i = 2 reads tasks[2], which is now id 4. Id 3, living at index 1, sits behind the counter and is never visited. i becomes 3, past the end, and the loop stops.

Id 3 is skipped because deleting id 2 moved it into a seat the loop had already left. The real output is [1, 3, 4], and the Python tutorial warns about this class of mistake directly: "Code that modifies a collection while iterating over that same collection can be tricky to get right" (section 4.2, for Statements).

Line 5, tasks.remove(t), is the line that draws the eye, and it is innocent. It removes precisely the task it was handed. The fault is not what it removes but the list it removes from - the one the loop is still counting through.

That renumbering is also why the bug hides. A fixture with a single pending task passes: there is nothing behind it to skip over. A fixture where the pending tasks are separated by done tasks passes too, because each removal only skips the harmless done item that follows. The failure needs two matching items back to back, so that deleting the first hops the counter clean over the second. Realistic sample data often has exactly one flagged row, and the test stays green for months.

The retry queue that keeps a job it swore it dropped

A worker drains a retry_queue, deleting jobs it has decided to abandon, using this same loop shape. For a long time it behaves. Then a downstream service has a bad minute and returns two failures in a row, which land as adjacent pending entries. The drain deletes the first, skips the second, and the second stays in the queue. On the next tick it gets picked up and processed again - a charge captured twice, an email sent twice. Nobody reaches for the loop, because a loop removes what it iterates and this one ran fine every day up to now. The bug only ever shows its face when two doomed jobs arrive as neighbours.

The fix: iterate a copy, or build a new list

Give the loop something stable to count through while you mutate the original:

def clear_pending(tasks):
    for t in list(tasks):          # iterate a snapshot
        if t["status"] == "pending":
            tasks.remove(t)
    return tasks

list(tasks) (or the slice tasks[:]) copies the elements into a throwaway list; the loop counts through that copy while remove edits the real one, so no live index gets renumbered. Cleaner still, stop mutating in place and keep what you want:

def clear_pending(tasks):
    return [t for t in tasks if t["status"] != "pending"]

The tempting wrong fix is to reach for a manual while loop and index by hand:

def clear_pending(tasks):
    i = 0
    while i < len(tasks):
        if tasks[i]["status"] == "pending":
            del tasks[i]
        i += 1                     # bug: advances even after a delete
    return tasks

This rebuilds the original bug with more typing. After del tasks[i] the next task shifts down into index i, and i += 1 steps right over it. The counter has to stay put on a delete and only advance when nothing was removed - put the i += 1 in an else, or the manual loop skips exactly the way the for loop did.

How to catch list mutation during iteration in your codebase

The shape is a .remove(, .pop(, or del whose target is the very list the enclosing loop is walking. A first-pass grep to eyeball:

grep -rnE '\.(remove|pop)\(|^\s*del \w+\[' --include='*.py' .

It over-matches, so the read is manual: for each hit, check whether the mutated name is the collection the surrounding for iterates.

Pylint catches this one automatically. modified-iterating-list (W4701) fires when a list is added to or removed from inside a loop over that list, with siblings modified-iterating-dict and modified-iterating-set for the other containers. As of mid-2026 Ruff ships no equivalent rule, so if Ruff is your only linter the reliable defence is a habit rather than a check: when a loop needs to delete from what it reads, iterate list(x) by default and let the copy absorb the churn.

Related mutation bugs in Python

This is the same family as a list that lives once on the class and is shared by every instance, and the default argument list that survives between calls - all three are one mutable object being touched from a place you did not expect. The loop misleads a different way when a lambda built inside it reads the loop variable only at call time, long after the loop has moved on. More in the Python bug write-ups, or the full bug index.

Hand the loop a copy and run the queue again: task 3 leaves with the other pending job, and the cleanup finally does what its name promised.