This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

Python implementation of the DGIM algorithm: a compact datastructure to estimate the number of True statements in the last N elements of a boolean stream.

Features

  • Estimation of the number of “True” statements in the last N element of a boolean stream
  • Low memory footprint.
  • Tunable error rate (the lower the error rate, the higher the memory footprint)

Applications

When processing large streams of data such as clicks streams, server logs, financial streams. It is often necessary to maintain statistics about the N latest elements. If N is big or if you have many streams to process, it is not possible to store the N latest elements.

In such situations, if the processed stream is made of boolean, the DGIM algorithm can help you estimate the number of True statements in the last elements.

For instance, if the stream is made of server logs, DGIM algorithm can estimate the proportion of visits that come from search engines. (as opposed to direct access, or access through paid search)

Installation

At the command line:

$ pip install dgim

Usage

Sample code:

from dgim import Dgim
dgim = Dgim(N=32, error_rate=0.1)
for i in range(100):
    dgim.update(True)
dgim_result = dgim.get_count() # 30 (exact result is 32)

License

The project is licensed under the BSD license.

Authors

How to contribute

  1. Check for open issues or open a fresh issue to start a discussion around a feature idea or a bug.
  2. Fork the repository on GitHub to start making your changes to the master branch (or branch off of it).
  3. Write a test which shows that the bug was fixed or that the feature works as expected.
  4. Send a pull request and bug the maintainer until it gets merged and published. :) Make sure to add yourself to AUTHORS.

References

History

0.2.0 (2015-01-05)

  • Improved documentation
  • Make most methods and attribute private.

0.1.0 (2015-01-04)

  • First release on PyPI.
Release History

Release History

0.2.0

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

0.1.0

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
dgim-0.2.0.tar.gz (17.6 kB) Copy SHA256 Checksum SHA256 Source Jan 5, 2015

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting