Skip to main content

Like sorted but using external sorting so that large data sets can be sorted.

Project description


Like sorted but using external sorting so that large data sets can be sorted, for example:

>>> from random import random
>>> from six.moves import xrange
>>> from xsorted import xsorted
>>> nums = (random() for _ in xrange(pow(10, 7)))
>>> for x in xsorted(nums): pass

The only restriction is that the items must be pickleable (or you can provide your own serializer for externalizing partitions of items).


It is sometimes necessary to sort a dataset without having to load the entire set into memory. For example, if you want to group a very large csv file by one of it’s columns. There are several ways in which this can be achieved, a common solution is to use the unix command sort. However unix sort does not offer the flexibility of the python csv module. xsorted attempts to generalize external sorting of any python iterable in a similar way in which sorted generalises the sorting of any iterable.


$ pip install xsorted


Just like sorted

>>> from xsorted import xsorted
>>> ''.join(xsorted('qwertyuiopasdfghjklzxcvbnm'))

With reverse

>>> ''.join(xsorted('qwertyuiopasdfghjklzxcvbnm', reverse=True))

And a custom key

>>> list(xsorted(('qwerty', 'uiop', 'asdfg', 'hjkl', 'zxcv', 'bnm'), key=lambda x: x[1]))
['uiop', 'hjkl', 'bnm', 'asdfg', 'qwerty', 'zxcv']

The implementation details of xsorted can be customized using the factory xsorter (in order to provide the same interface as sorted the partition_size is treated as an implementation detail):

>>> from xsorted import xsorter
>>> xsorted_custom = xsorter(partition_size=4)
>>> ''.join(xsorted_custom('qwertyuiopasdfghjklzxcvbnm'))

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

xsorted-0.0.4.tar.gz (119.9 kB view hashes)

Uploaded source

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page