Skip to main content

More routines for operating on iterables, beyond itertools

Project description

https://coveralls.io/repos/github/erikrose/more-itertools/badge.svg?branch=master

Python’s itertools library is a gem - you can compose elegant solutions for a variety of problems with the functions it provides. In more-itertools we collect additional building blocks, recipes, and routines for working with Python iterables.


Grouping chunked, sliced, distribute, divide, split_at, split_before, split_after, bucket, grouper, partition
Lookahead and lookback spy, peekable, seekable
Windowing windowed, stagger, pairwise
Augmenting count_cycle, intersperse, padded, adjacent, groupby_transform, padnone, ncycles
Combining collapse, sort_together, interleave, interleave_longest, collate, zip_offset, dotproduct, flatten, roundrobin, prepend
Summarizing ilen, first, last, one, unique_to_each, locate, rlocate, consecutive_groups, exactly_n, run_length, map_reduce, all_equal, first_true, nth, quantify
Selecting islice_extended, strip, lstrip, rstrip, take, tail, unique_everseen, unique_justseen
Combinatorics distinct_permutations, circular_shifts, powerset, random_product, random_permutation, random_combination, random_combination_with_replacement, nth_combination
Wrapping always_iterable, consumer, with_iter, iter_except
Others replace, numeric_range, always_reversible, side_effect, iterate, difference, make_decorator, SequenceView, consume, accumulate, tabulate, repeatfunc

Getting started

To get started, install the library with pip:

pip install more-itertools

The recipes from the itertools docs are included in the top-level package:

>>> from more_itertools import flatten
>>> iterable = [(0, 1), (2, 3)]
>>> list(flatten(iterable))
[0, 1, 2, 3]

Several new recipes are available as well:

>>> from more_itertools import chunked
>>> iterable = [0, 1, 2, 3, 4, 5, 6, 7, 8]
>>> list(chunked(iterable, 3))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]

>>> from more_itertools import spy
>>> iterable = (x * x for x in range(1, 6))
>>> head, iterable = spy(iterable, n=3)
>>> list(head)
[1, 4, 9]
>>> list(iterable)
[1, 4, 9, 16, 25]

For the full listing of functions, see the API documentation.

Development

more-itertools is maintained by @erikrose and @bbayles, with help from many others. If you have a problem or suggestion, please file a bug or pull request in this repository. Thanks for contributing!

Version History

4.3.0

  • New itertools:
    • last (thanks to tmshn)
    • replace (thanks to pylang)
    • rlocate (thanks to jferard and pylang)
  • Improvements to existing itertools:
    • locate can now search for multiple items
  • Other changes:
    • The docs now include a nice table of tools (thanks MSeifert04)

4.2.0

  • New itertools:
  • Improvements to existing itertools:
    • bucket now complies with PEP 479 (thanks to irmen)
  • Other changes:
    • Python 3.7 is now supported (thanks to irmen)
    • Python 3.3 is no longer supported
    • The test suite no longer requires third-party modules to run
    • The API docs now include links to source code

4.1.0

  • New itertools:
    • split_at (thanks to michael-celani)
    • circular_shifts (thanks to hiqua)
    • make_decorator - see the blog post Yo, I heard you like decorators for a tour (thanks to pylang)
    • always_reversible (thanks to michael-celani)
    • nth_combination (from the Python 3.7 docs)
  • Improvements to existing itertools:
    • seekable now has an elements method to return cached items.
    • The performance tradeoffs between roundrobin and interleave_longest are now documented (thanks michael-celani, pylang, and MSeifert04)

4.0.1

  • No code changes - this release fixes how the docs display on PyPI.

4.0.0

  • New itertools:
    • consecutive_groups (Based on the example in the Python 2.4 docs)
    • seekable (If you’re looking for how to “reset” an iterator, you’re in luck!)
    • exactly_n (thanks to michael-celani)
    • run_length.encode and run_length.decode
    • difference
  • Improvements to existing itertools:
    • The number of items between filler elements in intersperse can now be specified (thanks to pylang)
    • distinct_permutations and peekable got some minor adjustments (thanks to MSeifert04)
    • always_iterable now returns an iterator object. It also now allows different types to be considered iterable (thanks to jaraco)
    • bucket can now limit the keys it stores in memory
    • one now allows for custom exceptions (thanks to kalekundert)
  • Other changes:
    • A few typos were fixed (thanks to EdwardBetts)
    • All tests can now be run with python setup.py test

The major version update is due to the change in the return value of always_iterable. It now always returns iterator objects:

>>> from more_itertools import always_iterable
# Non-iterable objects are wrapped with iter(tuple(obj))
>>> always_iterable(12345)
<tuple_iterator object at 0x7fb24c9488d0>
>>> list(always_iterable(12345))
[12345]
# Iterable objects are wrapped with iter()
>>> always_iterable([1, 2, 3, 4, 5])
<list_iterator object at 0x7fb24c948c50>

3.2.0

  • New itertools:
    • lstrip, rstrip, and strip (thanks to MSeifert04 and pylang)
    • islice_extended
  • Improvements to existing itertools:
    • Some bugs with slicing peekable-wrapped iterables were fixed

3.1.0

  • New itertools:
    • numeric_range (Thanks to BebeSparkelSparkel and MSeifert04)
    • count_cycle (Thanks to BebeSparkelSparkel)
    • locate (Thanks to pylang and MSeifert04)
  • Improvements to existing itertools:
    • A few itertools are now slightly faster due to some function optimizations. (Thanks to MSeifert04)
  • The docs have been substantially revised with installation notes, categories for library functions, links, and more. (Thanks to pylang)

3.0.0

  • Removed itertools:
    • context has been removed due to a design flaw - see below for replacement options. (thanks to NeilGirdhar)
  • Improvements to existing itertools:
    • side_effect now supports before and after keyword arguments. (Thanks to yardsale8)
  • PyPy and PyPy3 are now supported.

The major version change is due to the removal of the context function. Replace it with standard with statement context management:

# Don't use context() anymore
file_obj = StringIO()
consume(print(x, file=f) for f in context(file_obj) for x in u'123')

# Use a with statement instead
file_obj = StringIO()
with file_obj as f:
    consume(print(x, file=f) for x in u'123')

2.6.0

  • New itertools:
    • adjacent and groupby_transform (Thanks to diazona)
    • always_iterable (Thanks to jaraco)
    • (Removed in 3.0.0) context (Thanks to yardsale8)
    • divide (Thanks to mozbhearsum)
  • Improvements to existing itertools:
    • ilen is now slightly faster. (Thanks to wbolster)
    • peekable can now prepend items to an iterable. (Thanks to diazona)

2.5.0

  • New itertools:
    • distribute (Thanks to mozbhearsum and coady)
    • sort_together (Thanks to clintval)
    • stagger and zip_offset (Thanks to joshbode)
    • padded
  • Improvements to existing itertools:
    • peekable now handles negative indexes and slices with negative components properly.
    • intersperse is now slightly faster. (Thanks to pylang)
    • windowed now accepts a step keyword argument. (Thanks to pylang)
  • Python 3.6 is now supported.

2.4.1

  • Move docs 100% to readthedocs.io.

2.4

  • New itertools:
    • accumulate, all_equal, first_true, partition, and tail from the itertools documentation.
    • bucket (Thanks to Rosuav and cvrebert)
    • collapse (Thanks to abarnet)
    • interleave and interleave_longest (Thanks to abarnet)
    • side_effect (Thanks to nvie)
    • sliced (Thanks to j4mie and coady)
    • split_before and split_after (Thanks to astronouth7303)
    • spy (Thanks to themiurgo and mathieulongtin)
  • Improvements to existing itertools:
    • chunked is now simpler and more friendly to garbage collection. (Contributed by coady, with thanks to piskvorky)
    • collate now delegates to heapq.merge when possible. (Thanks to kmike and julianpistorius)
    • peekable-wrapped iterables are now indexable and sliceable. Iterating through peekable-wrapped iterables is also faster.
    • one and unique_to_each have been simplified. (Thanks to coady)

2.3

  • Added one from jaraco.util.itertools. (Thanks, jaraco!)
  • Added distinct_permutations and unique_to_each. (Contributed by bbayles)
  • Added windowed. (Contributed by bbayles, with thanks to buchanae, jaraco, and abarnert)
  • Simplified the implementation of chunked. (Thanks, nvie!)
  • Python 3.5 is now supported. Python 2.6 is no longer supported.
  • Python 3 is now supported directly; there is no 2to3 step.

2.2

  • Added iterate and with_iter. (Thanks, abarnert!)

2.1

  • Added (tested!) implementations of the recipes from the itertools documentation. (Thanks, Chris Lonnen!)
  • Added ilen. (Thanks for the inspiration, Matt Basta!)

2.0

  • chunked now returns lists rather than tuples. After all, they’re homogeneous. This slightly backward-incompatible change is the reason for the major version bump.
  • Added @consumer.
  • Improved test machinery.

1.1

  • Added first function.
  • Added Python 3 support.
  • Added a default arg to peekable.peek().
  • Noted how to easily test whether a peekable iterator is exhausted.
  • Rewrote documentation.

1.0

  • Initial release, with collate, peekable, and chunked. Could really use better docs.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, size & hash SHA256 hash help File type Python version Upload date
more_itertools-4.3.0-py2-none-any.whl (48.5 kB) Copy SHA256 hash SHA256 Wheel py2
more_itertools-4.3.0-py3-none-any.whl (48.5 kB) Copy SHA256 hash SHA256 Wheel py3
more-itertools-4.3.0.tar.gz (63.3 kB) Copy SHA256 hash SHA256 Source None

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page