more-itertools 8.1.0

More routines for operating on iterables, beyond itertools

Python’s itertools library is a gem - you can compose elegant solutions for a variety of problems with the functions it provides. In more-itertools we collect additional building blocks, recipes, and routines for working with Python iterables.

---

Grouping	chunked, ichunked, sliced, distribute, divide, split_at, split_before, split_after, split_into, split_when, bucket, unzip, grouper, partition
Lookahead and lookback	spy, peekable, seekable
Windowing	windowed, substrings, substrings_indexes, stagger, pairwise
Augmenting	count_cycle, intersperse, padded, repeat_last, adjacent, groupby_transform, padnone, ncycles
Combining	collapse, sort_together, interleave, interleave_longest, zip_offset, dotproduct, flatten, roundrobin, prepend
Summarizing	ilen, unique_to_each, sample, consecutive_groups, run_length, map_reduce, exactly_n, all_equal, first_true, quantify
Selecting	islice_extended, first, last, one, only, strip, lstrip, rstrip, filter_except map_except nth_or_last, nth, take, tail, unique_everseen, unique_justseen
Combinatorics	distinct_permutations, distinct_combinations, circular_shifts, partitions, set_partitions, powerset, random_product, random_permutation, random_combination, random_combination_with_replacement, nth_combination
Wrapping	always_iterable, always_reversible, consumer, with_iter, iter_except
Others	locate, rlocate, replace, numeric_range, side_effect, iterate, difference, make_decorator, SequenceView, time_limited, consume, tabulate, repeatfunc

Getting started

To get started, install the library with pip:

pip install more-itertools

The recipes from the itertools docs are included in the top-level package:

>>> from more_itertools import flatten
>>> iterable = [(0, 1), (2, 3)]
>>> list(flatten(iterable))
[0, 1, 2, 3]

Several new recipes are available as well:

>>> from more_itertools import chunked
>>> iterable = [0, 1, 2, 3, 4, 5, 6, 7, 8]
>>> list(chunked(iterable, 3))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

>>> from more_itertools import spy
>>> iterable = (x * x for x in range(1, 6))
>>> head, iterable = spy(iterable, n=3)
>>> list(head)
[1, 4, 9]
>>> list(iterable)
[1, 4, 9, 16, 25]

For the full listing of functions, see the API documentation.

Development

more-itertools is maintained by @erikrose and @bbayles, with help from many others. If you have a problem or suggestion, please file a bug or pull request in this repository. Thanks for contributing!

Version History

noindex:

8.1.0

• Bug fixes

  • partition works with pred=None again. (thanks to MSeifert04)

• New itertools

  • sample (thanks to tommyod)

  • nth_or_last (thanks to d-ryzhikov)

• Changes to existing itertools:

  • The implementation for divide was improved. (thanks to jferard)

8.0.2

• Bug fixes

  • The type stub files are now part of the wheel distribution (thanks to keisheiled)

8.0.1

• Bug fixes

  • The type stub files now work for functions imported from the root package (thanks to keisheiled)

8.0.0

• New itertools and other additions

  • This library now ships type hints for use with mypy. (thanks to ilai-deutel for the implementation, and to gabbard and fmagin for assistance)

  • split_when (thanks to jferard)

  • repeat_last (thanks to d-ryzhikov)

• Changes to existing itertools:

  • The implementation for set_partitions was improved. (thanks to jferard)

  • partition was optimized for expensive predicates. (thanks to stevecj)

  • unique_everseen and groupby_transform were re-factored. (thanks to SergBobrovsky)

  • The implementation for difference was improved. (thanks to Jabbey92)

• Other changes

  • Python 3.4 has reached its end of life and is no longer supported.

  • Python 3.8 is officially supported. (thanks to jdufresne)

  • The collate function has been deprecated. It raises a DeprecationWarning if used, and will be removed in a future release.

  • one and only now provide more informative error messages. (thanks to gabbard)

  • Unit tests were moved outside of the main package (thanks to jdufresne)

  • Various documentation fixes (thanks to kriomant, gabbard, jdufresne)

7.2.0

• New itertools

  • distinct_combinations

  • set_partitions (thanks to kbarrett)

  • filter_except

  • map_except

7.1.0

• New itertools

  • ichunked (thanks davebelais and youtux)

  • only (thanks jaraco)

• Changes to existing itertools:

  • numeric_range now supports ranges specified by datetime.datetime and datetime.timedelta objects (thanks to MSeifert04 for tests).

  • difference now supports an initial keyword argument.

• Other changes

  • Various documentation fixes (thanks raimon49, pylang)

7.0.0

• New itertools:

  • time_limited

  • partitions (thanks to rominf and Saluev)

  • substrings_indexes (thanks to rominf)

• Changes to existing itertools:

  • collapse now treats bytes objects the same as str objects. (thanks to Sweenpet)

The major version update is due to the change in the default behavior of collapse. It now treats bytes objects the same as str objects. This aligns its behavior with always_iterable.

>>> from more_itertools import collapse
>>> iterable = [[1, 2], b'345', [6]]
>>> print(list(collapse(iterable)))
[1, 2, b'345', 6]

6.0.0

• Major changes:

  • Python 2.7 is no longer supported. The 5.0.0 release will be the last version targeting Python 2.7.

  • All future releases will target the active versions of Python 3. As of 2019, those are Python 3.4 and above.

  • The six library is no longer a dependency.

  • The accumulate function is no longer part of this library. You may import a better version from the standard itertools module.

• Changes to existing itertools:

  • The order of the parameters in grouper have changed to match the latest recipe in the itertools documentation. Use of the old order will be supported in this release, but emit a DeprecationWarning. The legacy behavior will be dropped in a future release. (thanks to jaraco)

  • distinct_permutations was improved (thanks to jferard - see also permutations with unique values at StackOverflow.)

  • An unused parameter was removed from substrings. (thanks to pylang)

• Other changes:

  • The docs for unique_everseen were improved. (thanks to jferard and MSeifert04)

  • Several Python 2-isms were removed. (thanks to jaraco, MSeifert04, and hugovk)

5.0.0

• New itertools:

  • split_into (thanks to rovyko)

  • unzip (thanks to bmintz)

  • substrings (thanks to pylang)

• Changes to existing itertools:

  • ilen was optimized a bit (thanks to MSeifert04, achampion, and bmintz)

  • first_true now returns None by default. This is the reason for the major version bump - see below. (thanks to sk and OJFord)

• Other changes:

  • Some code for old Python versions was removed (thanks to hugovk)

  • Some documentation mistakes were corrected (thanks to belm0 and hugovk)

  • Tests now run properly on 32-bit versions of Python (thanks to Millak)

  • Newer versions of CPython and PyPy are now tested against

The major version update is due to the change in the default return value of first_true. It’s now None.

>>> from more_itertools import first_true
>>> iterable = [0, '', False, [], ()]  # All these are False
>>> answer = first_true(iterable)
>>> print(answer)
None

4.3.0

• New itertools:

  • last (thanks to tmshn)

  • replace (thanks to pylang)

  • rlocate (thanks to jferard and pylang)

• Improvements to existing itertools:

  • locate can now search for multiple items

• Other changes:

  • The docs now include a nice table of tools (thanks MSeifert04)

4.2.0

• New itertools:

  • map_reduce (thanks to pylang)

  • prepend (from the Python 3.7 docs)

• Improvements to existing itertools:

  • bucket now complies with PEP 479 (thanks to irmen)

• Other changes:

  • Python 3.7 is now supported (thanks to irmen)

  • Python 3.3 is no longer supported

  • The test suite no longer requires third-party modules to run

  • The API docs now include links to source code

4.1.0

• New itertools:

  • split_at (thanks to michael-celani)

  • circular_shifts (thanks to hiqua)

  • make_decorator - see the blog post Yo, I heard you like decorators for a tour (thanks to pylang)

  • always_reversible (thanks to michael-celani)

  • nth_combination (from the Python 3.7 docs)

• Improvements to existing itertools:

  • seekable now has an elements method to return cached items.

  • The performance tradeoffs between roundrobin and interleave_longest are now documented (thanks michael-celani, pylang, and MSeifert04)

4.0.1

• No code changes - this release fixes how the docs display on PyPI.

4.0.0

• New itertools:

  • consecutive_groups (Based on the example in the Python 2.4 docs)

  • seekable (If you’re looking for how to “reset” an iterator, you’re in luck!)

  • exactly_n (thanks to michael-celani)

  • run_length.encode and run_length.decode

  • difference

• Improvements to existing itertools:

  • The number of items between filler elements in intersperse can now be specified (thanks to pylang)

  • distinct_permutations and peekable got some minor adjustments (thanks to MSeifert04)

  • always_iterable now returns an iterator object. It also now allows different types to be considered iterable (thanks to jaraco)

  • bucket can now limit the keys it stores in memory

  • one now allows for custom exceptions (thanks to kalekundert)

• Other changes:

  • A few typos were fixed (thanks to EdwardBetts)

  • All tests can now be run with python setup.py test

The major version update is due to the change in the return value of always_iterable. It now always returns iterator objects:

>>> from more_itertools import always_iterable
# Non-iterable objects are wrapped with iter(tuple(obj))
>>> always_iterable(12345)
<tuple_iterator object at 0x7fb24c9488d0>
>>> list(always_iterable(12345))
[12345]
# Iterable objects are wrapped with iter()
>>> always_iterable([1, 2, 3, 4, 5])
<list_iterator object at 0x7fb24c948c50>

3.2.0

• New itertools:

  • lstrip, rstrip, and strip (thanks to MSeifert04 and pylang)

  • islice_extended

• Improvements to existing itertools:

  • Some bugs with slicing peekable-wrapped iterables were fixed

3.1.0

• New itertools:

  • numeric_range (Thanks to BebeSparkelSparkel and MSeifert04)

  • count_cycle (Thanks to BebeSparkelSparkel)

  • locate (Thanks to pylang and MSeifert04)

• Improvements to existing itertools:

  • A few itertools are now slightly faster due to some function optimizations. (Thanks to MSeifert04)

• The docs have been substantially revised with installation notes, categories for library functions, links, and more. (Thanks to pylang)

3.0.0

• Removed itertools:

  • context has been removed due to a design flaw - see below for replacement options. (thanks to NeilGirdhar)

• Improvements to existing itertools:

  • side_effect now supports before and after keyword arguments. (Thanks to yardsale8)

• PyPy and PyPy3 are now supported.

The major version change is due to the removal of the context function. Replace it with standard with statement context management:

# Don't use context() anymore
file_obj = StringIO()
consume(print(x, file=f) for f in context(file_obj) for x in u'123')

# Use a with statement instead
file_obj = StringIO()
with file_obj as f:
    consume(print(x, file=f) for x in u'123')

2.6.0

• New itertools:

  • adjacent and groupby_transform (Thanks to diazona)

  • always_iterable (Thanks to jaraco)

  • (Removed in 3.0.0) context (Thanks to yardsale8)

  • divide (Thanks to mozbhearsum)

• Improvements to existing itertools:

  • ilen is now slightly faster. (Thanks to wbolster)

  • peekable can now prepend items to an iterable. (Thanks to diazona)

2.5.0

• New itertools:

  • distribute (Thanks to mozbhearsum and coady)

  • sort_together (Thanks to clintval)

  • stagger and zip_offset (Thanks to joshbode)

  • padded

• Improvements to existing itertools:

  • peekable now handles negative indexes and slices with negative components properly.

  • intersperse is now slightly faster. (Thanks to pylang)

  • windowed now accepts a step keyword argument. (Thanks to pylang)

• Python 3.6 is now supported.

2.4.1

• Move docs 100% to readthedocs.io.

2.4

• New itertools:

  • accumulate, all_equal, first_true, partition, and tail from the itertools documentation.

  • bucket (Thanks to Rosuav and cvrebert)

  • collapse (Thanks to abarnet)

  • interleave and interleave_longest (Thanks to abarnet)

  • side_effect (Thanks to nvie)

  • sliced (Thanks to j4mie and coady)

  • split_before and split_after (Thanks to astronouth7303)

  • spy (Thanks to themiurgo and mathieulongtin)

• Improvements to existing itertools:

  • chunked is now simpler and more friendly to garbage collection. (Contributed by coady, with thanks to piskvorky)

  • collate now delegates to heapq.merge when possible. (Thanks to kmike and julianpistorius)

  • peekable-wrapped iterables are now indexable and sliceable. Iterating through peekable-wrapped iterables is also faster.

  • one and unique_to_each have been simplified. (Thanks to coady)

2.3

• Added one from jaraco.util.itertools. (Thanks, jaraco!)

• Added distinct_permutations and unique_to_each. (Contributed by bbayles)

• Added windowed. (Contributed by bbayles, with thanks to buchanae, jaraco, and abarnert)

• Simplified the implementation of chunked. (Thanks, nvie!)

• Python 3.5 is now supported. Python 2.6 is no longer supported.

• Python 3 is now supported directly; there is no 2to3 step.

2.2

• Added iterate and with_iter. (Thanks, abarnert!)

2.1

• Added (tested!) implementations of the recipes from the itertools documentation. (Thanks, Chris Lonnen!)

• Added ilen. (Thanks for the inspiration, Matt Basta!)

2.0

• chunked now returns lists rather than tuples. After all, they’re homogeneous. This slightly backward-incompatible change is the reason for the major version bump.

• Added @consumer.

• Improved test machinery.

1.1

• Added first function.

• Added Python 3 support.

• Added a default arg to peekable.peek().

• Noted how to easily test whether a peekable iterator is exhausted.

• Rewrote documentation.

1.0

• Initial release, with collate, peekable, and chunked. Could really use better docs.