lazysequence 0.1.0

lazysequence

tl;dr A lazy sequence makes an iterator look like a tuple.

from lazysequence import LazySequence

def load_records():
    yield from [1, 2, 3, 4, 5, 6]  # pretend each iteration is expensive

records = LazySequence(load_records())
if not records:
    raise SystemExit("no records found")

first, second = records[:2]

print("The first record is", first)
print("The second record is", second)

for record in records.release():  # do not cache all records in memory
    print("record", record)

Sometimes you need to peek ahead at items returned by an iterator. How do you do that?

If the iterator does not need to be used later, just consume the items from the iterator. If later code needs to see all the items from the iterator, there are various options:

1. You could pass the consumed items to the surrounding code separately. This can get messy, though.

2. You could copy the items into a sequence beforehand. This is an option if the copy does not take a lot of space or time.

3. You could duplicate the iterator using itertools.tee, or write your own custom itertool. Consumed items are buffered internally. There are some good examples of this approach on SO, by Alex Martelli, Raymond Hettinger, and Ned Batchelder.

A lazy sequence combines advantages from option 2 and option 3. It is constructed from an iterable, and implements collections.abc.Sequence, providing the full set of immutable sequence operations on the iterable. Consumed items are cached internally, so the lookahead can happen transparently, and remains invisible to later code. Unlike a full copy (option 2), but like a duplicated iterator (option 3), items are only consumed and stored in memory as far as required for any given operation.

Caveats:

• The lazy sequence will eventually store all items in memory. If this is a problem, use s.release() to obtain an iterator over the sequence items without further caching. After calling this function, the sequence should no longer be used.

• Explicit is better than implicit. Clients may be better off being passed an iterator and dealing with its limitations. For example, clients may not expect len(s) to incur the cost of consuming the iterator to its end.

Installation

You can install lazysequence via pip from PyPI:

$ pip install lazysequence

Contributing

Contributions are very welcome. To learn more, see the Contributor Guide.

License

Distributed under the terms of the MIT license, lazysequence is free and open source software.

Issues

If you encounter any problems, please file an issue along with a detailed description.

Credits

This project was generated from @cjolowicz’s Hypermodern Python Cookiecutter template.