more-itertools 7.0.0

More routines for operating on iterables, beyond itertools

Python’s itertools library is a gem - you can compose elegant solutions for a variety of problems with the functions it provides. In more-itertools we collect additional building blocks, recipes, and routines for working with Python iterables.

---

Grouping	chunked, sliced, distribute, divide, split_at, split_before, split_after, split_into, bucket, unzip, grouper, partition
Lookahead and lookback	spy, peekable, seekable
Windowing	windowed, substrings, substrings_indexes, stagger, pairwise
Augmenting	count_cycle, intersperse, padded, adjacent, groupby_transform, padnone, ncycles
Combining	collapse, sort_together, interleave, interleave_longest, collate, zip_offset, dotproduct, flatten, roundrobin, prepend
Summarizing	ilen, first, last, one, unique_to_each, locate, rlocate, consecutive_groups, exactly_n, run_length, map_reduce, all_equal, first_true, nth, quantify
Selecting	islice_extended, strip, lstrip, rstrip, take, tail, unique_everseen, unique_justseen
Combinatorics	distinct_permutations, circular_shifts, partitions, powerset, random_product, random_permutation, random_combination, random_combination_with_replacement, nth_combination
Wrapping	always_iterable, consumer, with_iter, iter_except
Others	replace, numeric_range, always_reversible, side_effect, iterate, difference, make_decorator, SequenceView, time_limited, consume, tabulate, repeatfunc

Getting started

To get started, install the library with pip:

pip install more-itertools

The recipes from the itertools docs are included in the top-level package:

>>> from more_itertools import flatten
>>> iterable = [(0, 1), (2, 3)]
>>> list(flatten(iterable))
[0, 1, 2, 3]

Several new recipes are available as well:

>>> from more_itertools import chunked
>>> iterable = [0, 1, 2, 3, 4, 5, 6, 7, 8]
>>> list(chunked(iterable, 3))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

>>> from more_itertools import spy
>>> iterable = (x * x for x in range(1, 6))
>>> head, iterable = spy(iterable, n=3)
>>> list(head)
[1, 4, 9]
>>> list(iterable)
[1, 4, 9, 16, 25]

For the full listing of functions, see the API documentation.

Development

more-itertools is maintained by @erikrose and @bbayles, with help from many others. If you have a problem or suggestion, please file a bug or pull request in this repository. Thanks for contributing!

Version History

7.0.0

• New itertools:

  • time_limited

  • partitions (thanks to rominf and Saluev)

  • substrings_indexes (thanks to rominf)

• Changes to existing itertools:

  • collapse now treats bytes objects the same as str objects. (thanks to Sweenpet)

The major version update is due to the change in the default behavior of collapse. It now treats bytes objects the same as str objects. This aligns its behavior with always_iterable.

>>> from more_itertools import collapse
>>> iterable = [[1, 2], b'345', [6]]
>>> print(list(collapse(iterable)))
[1, 2, b'345', 6]

6.0.0

• Major chances:

  • Python 2.7 is no longer supported. The 5.0.0 release will be the last version targeting Python 2.7.

  • All future releases will target the active versions of Python 3. As of 2019, those are Python 3.4 and above.

  • The six library is no longer a dependency.

  • The accumulate function is no longer part of this library. You may import a better version from the standard itertools module.

• Changes to existing itertools:

  • The order of the parameters in grouper have changed to match the latest recipe in the itertools documentation. Use of the old order will be supported in this release, but emit a DeprecationWarning. The legacy behavior will be dropped in a future release. (thanks to jaraco)

  • distinct_permutations was improved (thanks to jferard - see also permutations with unique values at StackOverflow.)

  • An unused parameter was removed from substrings. (thanks to pylang)

• Other changes:

  • The docs for unique_everseen were improved. (thanks to jferard and MSeifert04)

  • Several Python 2-isms were removed. (thanks to jaraco, MSeifert04, and hugovk)

5.0.0

• New itertools:

  • split_into (thanks to rovyko)

  • unzip (thanks to bmintz)

  • substrings (thanks to pylang)

• Changes to existing itertools:

  • ilen was optimized a bit (thanks to MSeifert04, achampion, and bmintz)

  • first_true now returns None by default. This is the reason for the major version bump - see below. (thanks to sk and OJFord)

• Other changes:

  • Some code for old Python versions was removed (thanks to hugovk)

  • Some documentation mistakes were corrected (thanks to belm0 and hugovk)

  • Tests now run properly on 32-bit versions of Python (thanks to Millak)

  • Newer versions of CPython and PyPy are now tested against

The major version update is due to the change in the default return value of first_true. It’s now None.

>>> from more_itertools import first_true
>>> iterable = [0, '', False, [], ()]  # All these are False
>>> answer = first_true(iterable)
>>> print(answer)
None

4.3.0

• New itertools:

  • last (thanks to tmshn)

  • replace (thanks to pylang)

  • rlocate (thanks to jferard and pylang)

• Improvements to existing itertools:

  • locate can now search for multiple items

• Other changes:

  • The docs now include a nice table of tools (thanks MSeifert04)

4.2.0

• New itertools:

  • map_reduce (thanks to pylang)

  • prepend (from the Python 3.7 docs)

• Improvements to existing itertools:

  • bucket now complies with PEP 479 (thanks to irmen)

• Other changes:

  • Python 3.7 is now supported (thanks to irmen)

  • Python 3.3 is no longer supported

  • The test suite no longer requires third-party modules to run

  • The API docs now include links to source code

4.1.0

• New itertools:

  • split_at (thanks to michael-celani)

  • circular_shifts (thanks to hiqua)

  • make_decorator - see the blog post Yo, I heard you like decorators for a tour (thanks to pylang)

  • always_reversible (thanks to michael-celani)

  • nth_combination (from the Python 3.7 docs)

• Improvements to existing itertools:

  • seekable now has an elements method to return cached items.

  • The performance tradeoffs between roundrobin and interleave_longest are now documented (thanks michael-celani, pylang, and MSeifert04)

4.0.1

• No code changes - this release fixes how the docs display on PyPI.

4.0.0

• New itertools:

  • consecutive_groups (Based on the example in the Python 2.4 docs)

  • seekable (If you’re looking for how to “reset” an iterator, you’re in luck!)

  • exactly_n (thanks to michael-celani)

  • run_length.encode and run_length.decode

  • difference

• Improvements to existing itertools:

  • The number of items between filler elements in intersperse can now be specified (thanks to pylang)

  • distinct_permutations and peekable got some minor adjustments (thanks to MSeifert04)

  • always_iterable now returns an iterator object. It also now allows different types to be considered iterable (thanks to jaraco)

  • bucket can now limit the keys it stores in memory

  • one now allows for custom exceptions (thanks to kalekundert)

• Other changes:

  • A few typos were fixed (thanks to EdwardBetts)

  • All tests can now be run with python setup.py test

The major version update is due to the change in the return value of always_iterable. It now always returns iterator objects:

>>> from more_itertools import always_iterable
# Non-iterable objects are wrapped with iter(tuple(obj))
>>> always_iterable(12345)
<tuple_iterator object at 0x7fb24c9488d0>
>>> list(always_iterable(12345))
[12345]
# Iterable objects are wrapped with iter()
>>> always_iterable([1, 2, 3, 4, 5])
<list_iterator object at 0x7fb24c948c50>

3.2.0

• New itertools:

  • lstrip, rstrip, and strip (thanks to MSeifert04 and pylang)

  • islice_extended

• Improvements to existing itertools:

  • Some bugs with slicing peekable-wrapped iterables were fixed

3.1.0

• New itertools:

  • numeric_range (Thanks to BebeSparkelSparkel and MSeifert04)

  • count_cycle (Thanks to BebeSparkelSparkel)

  • locate (Thanks to pylang and MSeifert04)

• Improvements to existing itertools:

  • A few itertools are now slightly faster due to some function optimizations. (Thanks to MSeifert04)

• The docs have been substantially revised with installation notes, categories for library functions, links, and more. (Thanks to pylang)

3.0.0

• Removed itertools:

  • context has been removed due to a design flaw - see below for replacement options. (thanks to NeilGirdhar)

• Improvements to existing itertools:

  • side_effect now supports before and after keyword arguments. (Thanks to yardsale8)

• PyPy and PyPy3 are now supported.

The major version change is due to the removal of the context function. Replace it with standard with statement context management:

# Don't use context() anymore
file_obj = StringIO()
consume(print(x, file=f) for f in context(file_obj) for x in u'123')

# Use a with statement instead
file_obj = StringIO()
with file_obj as f:
    consume(print(x, file=f) for x in u'123')

2.6.0

• New itertools:

  • adjacent and groupby_transform (Thanks to diazona)

  • always_iterable (Thanks to jaraco)

  • (Removed in 3.0.0) context (Thanks to yardsale8)

  • divide (Thanks to mozbhearsum)

• Improvements to existing itertools:

  • ilen is now slightly faster. (Thanks to wbolster)

  • peekable can now prepend items to an iterable. (Thanks to diazona)

2.5.0

• New itertools:

  • distribute (Thanks to mozbhearsum and coady)

  • sort_together (Thanks to clintval)

  • stagger and zip_offset (Thanks to joshbode)

  • padded

• Improvements to existing itertools:

  • peekable now handles negative indexes and slices with negative components properly.

  • intersperse is now slightly faster. (Thanks to pylang)

  • windowed now accepts a step keyword argument. (Thanks to pylang)

• Python 3.6 is now supported.

2.4.1

• Move docs 100% to readthedocs.io.

2.4

• New itertools:

  • accumulate, all_equal, first_true, partition, and tail from the itertools documentation.

  • bucket (Thanks to Rosuav and cvrebert)

  • collapse (Thanks to abarnet)

  • interleave and interleave_longest (Thanks to abarnet)

  • side_effect (Thanks to nvie)

  • sliced (Thanks to j4mie and coady)

  • split_before and split_after (Thanks to astronouth7303)

  • spy (Thanks to themiurgo and mathieulongtin)

• Improvements to existing itertools:

  • chunked is now simpler and more friendly to garbage collection. (Contributed by coady, with thanks to piskvorky)

  • collate now delegates to heapq.merge when possible. (Thanks to kmike and julianpistorius)

  • peekable-wrapped iterables are now indexable and sliceable. Iterating through peekable-wrapped iterables is also faster.

  • one and unique_to_each have been simplified. (Thanks to coady)

2.3

• Added one from jaraco.util.itertools. (Thanks, jaraco!)

• Added distinct_permutations and unique_to_each. (Contributed by bbayles)

• Added windowed. (Contributed by bbayles, with thanks to buchanae, jaraco, and abarnert)

• Simplified the implementation of chunked. (Thanks, nvie!)

• Python 3.5 is now supported. Python 2.6 is no longer supported.

• Python 3 is now supported directly; there is no 2to3 step.

2.2

• Added iterate and with_iter. (Thanks, abarnert!)

2.1

• Added (tested!) implementations of the recipes from the itertools documentation. (Thanks, Chris Lonnen!)

• Added ilen. (Thanks for the inspiration, Matt Basta!)

2.0

• chunked now returns lists rather than tuples. After all, they’re homogeneous. This slightly backward-incompatible change is the reason for the major version bump.

• Added @consumer.

• Improved test machinery.

1.1

• Added first function.

• Added Python 3 support.

• Added a default arg to peekable.peek().

• Noted how to easily test whether a peekable iterator is exhausted.

• Rewrote documentation.

1.0

• Initial release, with collate, peekable, and chunked. Could really use better docs.