hyperstream 0.3.8

Docker images

If you do not want to install all the packages separately you can use our Docker bundle available here.

Local machine

Install via pip

pip install hyperstream
python -c 'from hyperstream import HyperStream'

To get the latest version

pip install -U git+git://github.com/IRC-SPHERE/HyperStream.git#egg=hyperstream
pip install -r requirements.txt

Or clone the repository

git clone git@github.com:IRC-SPHERE/HyperStream.git
cd HyperStream
virtualenv venv
. venv/bin/activate
pip install -r requirements.txt
python -c 'from hyperstream import HyperStream'

Additionally, one of the requirements to run Hyperstream is a MongoDB server. By default, Hyperstream tries to connect to the port 27017 on the localhost.

To access via a cloud provider, you might try mLab, MongoDB Atlas, or search for a different one. If you are using mLab, here are a few notes:

• Set up an account via the mLab web site instructions. When asked to pick a server type (e.g. Amazon, Google, etc) you can just choose free option of 500MB. This is more than enough to get started.

• mLab will ask you to create a database; any name is fine, but make sure you write down what it is.

• After creating a database, note that you’ll need to create at least one database user in order to access the database.

• You can test your database connection using MongoDB’s built-in command line tools

To install MongoDB locally go to the official documentation. E.g. in a Debian OS it is possible to install with the following command

sudo apt-get install mongodb

Once the MongoDB server is installed, it can be started with the following command

service mongod start

If installing MongoDB on OSX

brew install mongodb

then

ln -sfv /usr/local/opt/mongodb/*.plist ~/Library/LaunchAgents

run MongoDB service with

launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mongodb.plist

Running the tutorials in a docker container

It is possible to run all the tutorials in your own machine ussing Docker containers defined in IRC-SPHERE/Hyperstream-Dockerfiles. You can do that by running the following commands:

git clone https://github.com/IRC-SPHERE/Hyperstream-Dockerfiles.git
cd Hyperstream-Dockerfiles
docker-compose -f docker-compose-tutorials.yml -p hyperstream-tutorials up

And then open the url http://0.0.0.0:8888/tree in a web-browser

Running the tutorials in the cloned folder

To run the tutorials in the cloned repository you will need to install additional dependencies. First you should activate the virtual environment and installed the general requirements to run HyperStream following the instructions above. After that, install the dependencies for the tutorial with

pip install -r requirements_tutorial.txt

and go to the experiments folder

cd experiments

And run a Jupyter notebook

jupyter notebook

Now you can follow the instructions from the first tutorial.

Simple use-case

from hyperstream import HyperStream, StreamId, TimeInterval
from hyperstream.utils import utcnow, UTC
from datetime import timedelta

hs = HyperStream(loglevel=20)
M = hs.channel_manager.memory
T = hs.channel_manager.tools
clock = StreamId(name="clock")
clock_tool = T[clock].window().last().value()
ticker = M.get_or_create_stream(stream_id=StreamId(name="ticker"))
now = utcnow()
before = (now - timedelta(seconds=30)).replace(tzinfo=UTC)
ti = TimeInterval(before, now)
clock_tool.execute(sources=[], sink=ticker, interval=ti, alignment_stream=None)
list(ticker.window().tail(5))

The last list contains

[StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 45, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 45, tzinfo=<UTC>)),
 StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 46, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 46, tzinfo=<UTC>)),
 StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 47, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 47, tzinfo=<UTC>)),
 StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 48, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 48, tzinfo=<UTC>)),
 StreamInstance(timestamp=datetime.datetime(2017, 7, 27, 10, 33, 49, tzinfo=<UTC>), value=datetime.datetime(2017, 7, 27, 10, 33, 49, tzinfo=<UTC>))]