This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

This piece of code is a wrapper for the C library CRoaring. It provides a very efficient way to store and manipulate sets of (unsigned 32 bits) integers.

The wrapping used to be done with Ctypes. We recently switched to Cython for the following reasons:

  • Much better performances for short function calls (e.g. addition/deletion of a single element).
  • Easier installation, no need to install manually the C library, it is now distributed with PyRoaring.
  • Extensibility, it will be easier to write efficient codes with Cython (e.g. some data structures based on roaring bitmaps).

If for some reason you wish to keep using the old version, based on Ctypes, use PyRoaring 0.0.7.

Requirements

  • Environment like Linux and MacOS
  • Python 2.7, or Python 3.3 or better
  • A recent C compiler like GCC
  • The Python package hypothesis (optional, for testing)
  • The Python package Cython (optional, for compiling pyroaring from the sources)

Installation

To install pyroaring and the CRoaring library on your local account, use the following two lines:

pip install pyroaring --user # installs PyRoaringBitMap

To install them system-wide, use the following lines :

pip install pyroaring

Naturally, the latter may require superuser rights (consider prefixing the commands by sudo).

(If you want to use Python 3 and your system defaults on Python 2.7, you may need to adjust the above commands, e.g., replace pip by pip3 and python by python3.)

Manual compilation / installation

If you want to compile (and install) pyroaring by yourself, for instance to modify the Cython sources or because you do not have pip, follow these steps. Note that the Python package Cython is required.

Clone this repository.

git clone https://github.com/Ezibenroc/PyRoaringBitMap.git
cd PyRoaringBitMap

Build pyroaring locally, e.g. to test a new feature you made.

python setup.py build_ext -i
python test.py # run the tests, optionnal but recommended

Install pyroaring (use this if you do not have pip).

python setup.py install # may require superuser rights, add option --user if you wish to install it on your local account

Package pyroaring.

python setup.py sdist
pip install dist/pyroaring-0.0.7.tar.gz # optionnal, to install the package

Utilization

First, you can run the tests to make sure everything is ok:

pip install hypothesis --user
python test.py

You can use a bitmap nearly as the classical Python set in your code:

from pyroaring import BitMap
bm1 = BitMap()
bm1.add(3)
bm1.add(18)
bm2 = BitMap([3, 27, 42])
print("bm1       = %s" % bm1)
print("bm2       = %s" % bm2)
print("bm1 & bm2 = %s" % (bm1&bm2))
print("bm1 | bm2 = %s" % (bm1|bm2))

Output:

bm1       = BitMap([3, 18])
bm2       = BitMap([3, 27, 42])
bm1 & bm2 = BitMap([3])
bm1 | bm2 = BitMap([3, 18, 27, 42])

Benchmark

The built-in set is compared with this Python wrapper of CRoaring (designated as pyroaring in the following) and a Cython implementation of Roaring bitmaps (designated as cyroaring in the following).

Quick benchmarks for common operations

The script quick_benchmark.sh measures the time of different set operations. It uses sets initialized to range(b, 100000000, 8) with b equal to 0 or 1. It is far from being exhaustive, but rather a quick overview of how the three classes compare to each other.

Operation Pyroaring Cyroaring set
Empty constructor 0.000134 0.000154 7.75e-05
Range constructor 4.51 4.15 756
List constructor 153 124 584
Element addition 7.09e-05 7.06e-05 6.5e-05
Test for membership 3.03e-05 3.28e-05 2.6e-05
Conversion to list 653 486 146
Equality test 1.47 1.35 289
Subset test 1.54 1.46 283
Union 3.18 3.4 811
Intersection 2.58 2.52 132
Symetric difference 3.13 3.19 927
Selection 0.00754 0.00135 NA
Slice 747 2.88e+03 NA

Complete benchmark for the union

The performances of the union operation have been measured more carefully. Full results can be found here.

To sum up, both Roaring bitmap implementations are several orders of magnitude faster than the built-in set, regardless of the density of the data.

For sparse data, pyroaring is faster than cyroaring, for very dense data cyroaring is faster. Otherwise, they are similar.

Release History

Release History

0.1.0

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.7

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.6

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.5

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.4

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.3

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.2

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.0.1

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
pyroaring-0.1.0.tar.gz (142.3 kB) Copy SHA256 Checksum SHA256 Source Apr 27, 2017

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting