Statistics for Streaming Data
statstream is a lightweight Python package providing data analysis and statistics utilities for streaming data.
Its main goal is to provide single-pass variants of conventional numpy data analysis and statistics functionality for streaming data that is either generated on the fly or to large to be handled at once. Data can be streamed as in chunks called mini-batches, which makes statstream extremely useful in combination with machine learning and deep learning packages like keras, tensorflow, or pytorch.
statstream functions consume iterators providing batches of data. They compute statistics of these batches and combine them to obtain statistics for the full data set.
import statstream mean = statstream.streaming_mean(some_iterable)
If you’d like to contribute to statstream you’re most welcome. We have written a short guide to help you get you started!
Additional information on the algorithmic aspects of statstream can be found in the following works:
- Tony F. Chan & Gene H. Golub & Randall J. LeVeque, “Updating formulae and a pairwise algorithm for computing sample variances”, 1979
- Radim, Rehurek, “Scalability of Semantic Analysis in Natural Language Processing”, 2011
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
|Filename, size||File type||Python version||Upload date||Hashes|
|Filename, size statstream-19.1.0-py2.py3-none-any.whl (12.0 kB)||File type Wheel||Python version py2.py3||Upload date||Hashes View|
|Filename, size statstream-19.1.0.tar.gz (36.0 kB)||File type Source||Python version None||Upload date||Hashes View|
Hashes for statstream-19.1.0-py2.py3-none-any.whl