Skip to main content
Join the official 2020 Python Developers SurveyStart the survey!

A wrapper for the C++ Datasketches library

Project description

# Python Wrapper for Datasketches

## Installation

The release files do not include the needed python binding library ([pybind11](https://github.com/pybind/pybind11)). If building from a relase package, you must ensure that the pybind11 directory points to a local copy of pybind11.

An official pypi build is eventually planned but not yet available.

If you instead want to take a (possibly ill-advised) gamble on the current state of the master branch being useable, you can run: `pip install git+https://github.com/apache/incubator-datasketches-cpp.git`

## Developer Instructions

### Building

When cloning the source repository, you should include the pybind11 submodule with the –recursive option to the clone command: ` git clone --recursive https://github.com/apache/incubator-datasketches-cpp.git cd incubator-datasketches-cpp python -m pip install --upgrade pip setuptools wheel numpy python setup.py build `

If you cloned without –recursive, you can add the submodule post-checkout using git submodule update –init –recursive.

### Installing

Assuming you have already checked out the library and any dependent submodules, install by simply replacing the lsat line of the build command with python setup.py install.

### Unit tests

The python tests are run with tox. To ensure you have all the needed packages, from the package base directory run: ` python -m pip install --upgrade pip setuptools wheel numpy tox tox `

## Usage

Having installed the library, loading the Datasketches library in Python is simple: import datasketches.

## Available Sketch Classes

  • KLL
    • kll_ints_sketch
    • kll_floats_sketch
  • Frequent Items
    • frequent_strings_sketch
    • Error types are frequent_items_error_type.{NO_FALSE_NEGATIVES | NO_FALSE_POSITIVES}
  • Theta
    • update_theta_sketch
    • compact_theta_sketch (cannot be instantiated directly)
    • theta_union
    • theta_intersection
    • theta_a_not_b
  • HLL
    • hll_sketch
    • hll_union
    • Target HLL types are tgt_hll_type.{HLL_4 | HLL_6 | HLL_8}
  • CPC
    • cpc_sketch
    • cpc_union
  • VarOpt Sampling
    • var_opt_sketch
    • var_opt_union

## Known Differences from C++

The Python API largely mirrors the C++ API, with a few minor exceptions: The primary known differences are that Python on modern platforms does not support unsigned integer values or numeric values with fewer than 64 bits. As a result, you may not be able to produce identical sketches from within Python as you can with Java and C++. Loading those sketches after they have been serialized from another language will work as expected.

We have also removed reliance on a builder class for theta sketches as Python allows named arguments to the constructor, not strictly positional arguments.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for whylabs-datasketches, version 2.0.0b6
Filename, size File type Python version Upload date Hashes
Filename, size whylabs_datasketches-2.0.0b6-cp35-cp35m-macosx_10_6_x86_64.whl (347.4 kB) File type Wheel Python version cp35 Upload date Hashes View
Filename, size whylabs_datasketches-2.0.0b6-cp35-cp35m-manylinux1_x86_64.whl (343.1 kB) File type Wheel Python version cp35 Upload date Hashes View
Filename, size whylabs_datasketches-2.0.0b6-cp36-cp36m-macosx_10_15_x86_64.whl (353.3 kB) File type Wheel Python version cp36 Upload date Hashes View
Filename, size whylabs_datasketches-2.0.0b6-cp36-cp36m-manylinux1_x86_64.whl (343.1 kB) File type Wheel Python version cp36 Upload date Hashes View
Filename, size whylabs_datasketches-2.0.0b6-cp37-cp37m-macosx_10_15_x86_64.whl (353.4 kB) File type Wheel Python version cp37 Upload date Hashes View
Filename, size whylabs_datasketches-2.0.0b6-cp37-cp37m-manylinux1_x86_64.whl (343.3 kB) File type Wheel Python version cp37 Upload date Hashes View
Filename, size whylabs_datasketches-2.0.0b6-cp38-cp38-macosx_10_15_x86_64.whl (362.5 kB) File type Wheel Python version cp38 Upload date Hashes View
Filename, size whylabs_datasketches-2.0.0b6-cp38-cp38-manylinux1_x86_64.whl (340.8 kB) File type Wheel Python version cp38 Upload date Hashes View
Filename, size whylabs-datasketches-2.0.0b6.tar.gz (1.2 MB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page