guardlist

GuardedList is a drop-in replacement for the built-in list that raises an error the moment it is modified while a for-loop is iterating over it. Python already does this for dicts and sets; this package brings the same protection to lists, which are the one common container where it is missing.

The problem in Python

If you change the size of a dict or a set while looping over it, Python notices and raises an error immediately:

d = {"a": 1, "b": 2}
for key in d:
    del d[key]

RuntimeError: dictionary changed size during iteration

This is a deliberate safety feature. It exists because changing a container while you are in the middle of iterating over it can silently corrupt the iteration: elements can be skipped, repeated, or the loop can even run forever, all without printing any error at all. Sets behave the same way as dicts.

Lists do not have this protection. If you remove, add, or reorder items in a plain list while a for-loop is iterating over it, nothing warns you. The loop just produces the wrong answer.

How the flaw shows up with plain lists

a. Silent skipping

The most common version of the bug: removing items from a list while looping over it skips the item right after the one you removed.

numbers = [1, 2, 4, 6, 7]
for n in numbers:
    if n % 2 == 0:
        numbers.remove(n)
print(numbers)

[1, 4, 7]

The 4 should have been removed, since it is even, but it survives. When 2 is removed, everything after it shifts one position to the left, but the for-loop's internal position counter keeps counting up as if nothing moved, so it steps over 4 without ever looking at it.

b. The cleanup bug

The same shift happens with any remove() call, not just numbers. This is a typical "strip empty entries" cleanup that beginners write, and it does not fully work:

words = ["", "", "hello", ""]
for w in words:
    if w == "":
        words.remove(w)
print(words)

['hello', '']

Three empty strings existed, but only two were removed. After the first empty string at index 0 is removed, the second empty string shifts into its place, and the loop's position counter skips past it, so it is never even checked for emptiness.

c. Infinite loop

Adding to a list while iterating over it is worse than removing from it: it can make the loop never finish, because you keep creating more items for the loop to reach.

numbers = [1, 2, 3]
for n in numbers:
    numbers.append(n)

This never stops on its own: every item that gets visited immediately adds a copy of itself to the end of the list, so there is always one more item waiting. Running it with a safety counter shows the list still growing without end after a thousand iterations:

numbers = [1, 2, 3]
safety_counter = 0
for n in numbers:
    numbers.append(n)
    safety_counter += 1
    if safety_counter > 1000:
        print("stopped manually after", safety_counter, "iterations; list length is now", len(numbers))
        break

stopped manually after 1001 iterations; list length is now 1004

d. Insert during loop

Inserting near the front of the list while iterating causes items to be revisited, because everything after the insertion point shifts forward into positions the loop has not reached yet:

numbers = [1, 2, 3]
for index, n in enumerate(numbers):
    print(n)
    if index == 0:
        numbers.insert(0, 99)
print("final:", numbers)

1
1
2
3
final: [99, 1, 2, 3]

The value 1 prints twice. On the first pass the loop sees 1 at position 0 and inserts 99 at position 0. That pushes the original 1 into position 1, which is exactly the next position the loop looks at, so it sees 1 a second time.

e. Sorting or reversing during loop

Reordering the list mid-loop mixes up which elements get visited, and can cause some to repeat while others are skipped entirely:

numbers = [5, 3, 1, 4, 2]
seen = []
for n in numbers:
    seen.append(n)
    if n == 3:
        numbers.sort()
print("seen:", seen)

seen: [5, 3, 3, 4, 5]

The loop's position counter just walks through index 0, 1, 2, 3, 4 in order, with no idea that the list underneath it has been rearranged. Once sort() runs, position 2 no longer holds the value it used to, so 3 gets seen twice, 5 gets seen twice, and 1 and 2 are never seen at all.

f. Wrong-looking fix

A beginner who gets burned by remove() inside a for-loop often tries switching to an index-based loop with range(len(...)), expecting that to avoid the problem. It does not; it just fails in a different, more confusing way:

numbers = [1, 2, 3, 4, 5]
for i in range(len(numbers)):
    if numbers[i] % 2 == 0:
        numbers.remove(numbers[i])

IndexError: list index out of range

range(len(numbers)) is computed once, up front, using the original length of the list. As remove() shrinks the list, the later index values from that original range stop existing, and the loop eventually asks for an index that is no longer there.

The fix: GuardedList

pip install guardlist

GuardedList behaves exactly like a normal list, except that it notices when you try to mutate it during an active iteration and raises an error right away, instead of letting the loop quietly produce the wrong answer:

from guardlist import GuardedList

numbers = GuardedList([1, 2, 4, 6, 7])
for n in numbers:
    if n % 2 == 0:
        numbers.remove(n)

guardlist.IterationMutationError: GuardedList was modified by 'remove' while a for-loop was iterating over it. Iterate over a copy instead, for example: for item in list(my_list):

The error names the exact method that caused the problem and tells you how to fix it. The fix is to iterate over a copy of the list, so the mutations happen to the original list while the loop walks over an unchanging snapshot. Wrapping the loop in list(...) does this:

from guardlist import GuardedList

numbers = GuardedList([1, 2, 4, 6, 7])
for n in list(numbers):
    if n % 2 == 0:
        numbers.remove(n)
print(numbers)

[1, 7]

A list comprehension reaches the same correct result without needing a loop at all:

from guardlist import GuardedList

numbers = GuardedList([1, 2, 4, 6, 7])
numbers = GuardedList([n for n in numbers if n % 2 != 0])
print(numbers)

[1, 7]

Every operation GuardedList protects

Each of the following raises IterationMutationError if it is called while a loop over the same GuardedList is active. The message always names the method or operation that was attempted.

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers.append(4)

GuardedList was modified by 'append' while a for-loop was iterating over it. Iterate over a copy instead, for example: for item in list(my_list):

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers.extend([4, 5])

IterationMutationError: ... modified by 'extend' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers.insert(0, 9)

IterationMutationError: ... modified by 'insert' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers.remove(2)

IterationMutationError: ... modified by 'remove' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers.pop()

IterationMutationError: ... modified by 'pop' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers.clear()

IterationMutationError: ... modified by 'clear' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers.sort()

IterationMutationError: ... modified by 'sort' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers.reverse()

IterationMutationError: ... modified by 'reverse' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers[0] = 99

IterationMutationError: ... modified by '__setitem__' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    del numbers[0]

IterationMutationError: ... modified by '__delitem__' ...

numbers = GuardedList([1, 2, 3, 4])
for n in numbers:
    numbers[1:3] = [8, 9]

IterationMutationError: ... modified by '__setitem__' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers += [4]

IterationMutationError: ... modified by '__iadd__' ...

numbers = GuardedList([1, 2, 3])
for n in numbers:
    numbers *= 2

IterationMutationError: ... modified by '__imul__' ...

Slice assignment is reported as __setitem__, since that is the single method Python calls for both single-item and slice assignment.

What is still allowed during a loop

Anything that only reads the list, without changing it, is always allowed, even while a loop over it is active:

numbers = GuardedList([1, 2, 3, 4, 5])
for n in numbers:
    _ = numbers[0]
    _ = numbers[1:3]
    _ = len(numbers)
    _ = 3 in numbers
    _ = numbers.count(2)
    _ = numbers.index(4)
    _ = numbers == [1, 2, 3, 4, 5]
    _ = numbers + [6]
    _ = sorted(numbers, reverse=True)
print("all read-only operations succeeded during iteration")

all read-only operations succeeded during iteration

numbers + [6] and sorted(numbers, ...) are allowed because they build and return a brand new list, leaving the original GuardedList untouched.

Nested loops, break, and exceptions

Two loops over the same GuardedList at the same time are fine, since each loop's iterator adds its own count to the guard, and reading is always safe:

numbers = GuardedList([1, 2, 3])
pairs = []
for a in numbers:
    for b in numbers:
        pairs.append((a, b))
print(pairs)

[(1, 1), (1, 2), (1, 3), (2, 1), (2, 2), (2, 3), (3, 1), (3, 2), (3, 3)]

Leaving a loop early with break releases the guard, so the list can be mutated again right after:

numbers = GuardedList([1, 2, 3])
for n in numbers:
    break
numbers.append(4)
print(list(numbers))

[1, 2, 3, 4]

The same is true if an exception is raised inside the loop and caught outside it: the guard is released as the loop is abandoned, so the list is mutable again once the exception has been handled.

numbers = GuardedList([1, 2, 3])
try:
    for n in numbers:
        raise ValueError("boom")
except ValueError:
    pass
numbers.append(4)
print(list(numbers))

[1, 2, 3, 4]

Slices and copies stay guarded

Slicing a GuardedList and calling copy() on one both return a new GuardedList, so the protection travels with the data. Converting to a plain list() does not carry the protection over, which is exactly what you want when you need an unguarded snapshot to iterate over while mutating the original.

numbers = GuardedList([1, 2, 3, 4, 5])
print(type(numbers[1:3]).__name__)
print(type(numbers.copy()).__name__)
print(type(list(numbers)).__name__)

GuardedList
GuardedList
list

What it does not catch

GuardedList only guards the list itself, the container. It cannot see changes made through other routes:

Changing an object that is stored inside the list, rather than changing the list's own structure, is not something GuardedList can detect. If an element is itself a mutable object such as a list, modifying that inner object from within a loop is not blocked:

numbers = GuardedList([[1], [2], [3]])
for inner in numbers:
    inner.append(99)
print(numbers)

[[1, 99], [2, 99], [3, 99]]

A common attempt at writing the loop by hand, using an index and pop(), also slips past the guard, because it never calls iter() on the list at all:

numbers = GuardedList([10, 20, 20, 30, 40])
i = 0
while i < len(numbers):
    if numbers[i] % 20 == 0:
        numbers.pop(i)
    i += 1
print(numbers)

[10, 20, 30]

Every multiple of 20 should be gone, leaving [10, 30], but one 20 survives, with no error raised. This variant uses a while loop with a manually managed index instead of a for loop, so no iterator over the list is ever created and GuardedList has no active count to check against. The fix is the same as elsewhere: iterate over a copy, for example with list(numbers), or build the result with a list comprehension instead of mutating the list in place.

Calling the underlying list method directly on the class, instead of through the instance, also bypasses the guard, since it never goes through GuardedList's own overridden method:

numbers = GuardedList([1, 2, 3])
it = iter(numbers)
next(it)
list.append(numbers, 4)
print(numbers)

[1, 2, 3, 4]

When to use it

GuardedList is meant to be used where it earns its keep, not everywhere:

• Teaching: it turns a silent, confusing bug into an error message that explains exactly what went wrong and how to fix it, which makes it a good tool for anyone learning how iteration works in Python.
• Shared lists across many functions: if a list is passed around and mutated by code far away from the loop that iterates over it, the extra check catches mistakes that would otherwise be very hard to track down.
• Temporary debugging: swap a suspect list for a GuardedList, run the code to find exactly where the mutation happens, then swap it back to a plain list once the bug is fixed.
• Tests: assert that code under test does not mutate a list while iterating over it, by passing it a GuardedList instead of a list.

How it works

GuardedList keeps an internal counter of how many iterators over it are currently active. Calling iter() on it, which happens automatically at the start of every for-loop and also when you call reversed(), increments the counter. The counter is decremented again once that iterator is exhausted, explicitly closed, or garbage collected, so break and exceptions inside a loop correctly release the guard. Every method that mutates the list in place checks that counter first, and raises IterationMutationError if it is greater than zero, instead of performing the mutation. On CPython the guard is released immediately on break or an exception, because of reference counting; on other interpreters such as PyPy the release may be delayed until garbage collection runs.

API

• GuardedList: a subclass of list that raises IterationMutationError when mutated while it is being iterated over, and otherwise behaves exactly like a built-in list.
• IterationMutationError: a subclass of RuntimeError raised by GuardedList; it carries a method_name attribute holding the name of the method that triggered the error.

License

MIT