This is a pre-production deployment of Warehouse, however changes made here WILL affect the production instance of PyPI.
Latest Version Dependencies status unknown Test status unknown Test coverage unknown
Project Description

AwesomeStream

AwesomeStream is a set of tools for creating a “stream server”. That is, a server which can store information about events that happen, and can query back those events in reverse-chronological order, sliced in interesting ways.

Example and Use Case

Say that you run a website like GitHub, where people interact in various different ways. People can create repositories, fork them, watch or unwatch repositories, add friends, etc. There are all kinds of things that a user can do on the site. Let’s look at how AwesomeStream can help.

First, we’ll set up a simple redis-based server:

>>> from awesomestream.backends import RedisBackend
>>> from awesomestream.jsonrpc import create_app, run_server
>>> backend = RedisBackend(
...     keys=['user', 'kind', 'repo'],
...     host='127.0.0.1',
...     port=6379
... )
>>>
>>> app = create_app(backend)
>>> run_server(app, 8080)

This simple script sets up a Redis-based AwesomeStream server–one that pays special attention to the ‘user’, ‘kind’, and ‘repo’ keys. This will make a bit more sense in a bit.

In another console, we’re going to instantiate a client.

>>> from awesomestream.jsonrpc import Client
>>> c = Client('http://127.0.0.1:8080/')

OK, now that we’ve set up our client, lets start logging user actions. Look, a user has just created a new repo!

>>> c.insert({
...     'kind': 'create-repo',
...     'repo': 17,
...     'user': 291,
...     'name': 'frist',
...     'description': 'This is my first repo ever!',
... })
>>>

But the user made a mistake, and named it ‘frist’ instead of ‘first’. So they go ahead and delete it:

>>> c.insert({
...     'kind': 'delete-repo',
...     'repo': 17,
...     'user': 291,
...     'reason': 'Made a typo :(',
... })
>>>

Then they give up and decide to watch another user’s repo instead:

>>> c.insert({'kind': 'watch', 'repo': 2842, 'user': 291, 'owner': 23})

And finally they add that user as a friend:

>>> c.insert({'kind': 'friend', 'user': 291, 'friend': 23})

That second user notices that someone is following them, and follows back:

>>> c.insert({'kind': 'friend', 'user': 23, 'friend': 291})

Now that we have data inserted into the stream server, we can query it to get back the full stream. Here’s how something like that might look:

>>> c.items()
[{'kind': 'friend', 'user': 23, 'friend': 291},
 {'kind': 'friend', 'user': 291, 'friend': 23},
 {'repo': 2842, 'owner': 23, 'kind': 'watch', 'user': 291},
 {'repo': 17, 'kind': 'delete-repo', 'reason': 'Made a typo :(', 'user': 291},
 {'repo': 17, 'kind': 'create-repo', 'user': 291, 'name': 'frist', 'description': 'This is my first repo ever!'}
]

As you can see, we got the entire stream back, in reverse chronological order. But let’s say we want to filter this out, to only see ‘friend’ requests. We can do that easily:

>>> c.items(kind='friend')
[{'kind': 'friend', 'user': 23, 'friend': 291},
 {'kind': 'friend', 'user': 291, 'friend': 23}
]

Notice that they are still in reverse chronological order. We can also combine our predicates, to get only friend requests made by a specific user.

>>> c.items(kind='friend', user=23)
[{'kind': 'friend', 'user': 23, 'friend': 291}]

But an extremely common case is that you want to see only your activity that is generated by your friends. With AwesomeStream, that’s simple:

>>> c.items(user=[23, 291])
[{'kind': 'friend', 'user': 23, 'friend': 291},
 {'kind': 'friend', 'user': 291, 'friend': 23},
 {'repo': 2842, 'owner': 23, 'kind': 'watch', 'user': 291},
 {'repo': 17, 'kind': 'delete-repo', 'reason': 'Made a typo :(', 'user': 291},
 {'repo': 17, 'kind': 'create-repo', 'user': 291, 'name': 'frist', 'description': 'This is my first repo ever!'}
]

As you can see, every user ID passed into that list is retrieved. By default, the items() function retrieves 20 items, but often times we’ll need to customize that. Here’s how that would look:

>>> c.items(user=[23, 291], start=1, end=3)
[{'kind': 'friend', 'user': 291, 'friend': 23},
 {'repo': 2842, 'owner': 23, 'kind': 'watch', 'user': 291}
]

Supported Backends

  • In-Memory (mostly for testing)
  • SQL
  • Redis

Planned Support

  • CouchDB
  • Cassandra

Maturity

I’m writing this for eventual deployment on http://radiosox.com/, but have not yet deployed it in production. Do so at your own risk.

Requirements

Short Summary:

Use pip, and do pip install -U -r requirements.txt

Longer Summary:

Strictly speaking, the only requirement is simplejson. That being said, if you want redis support, you need redis installed. If you want SQL support, you need SQLAlchemy installed. If you want support for creating a WSGI app to expose this over HTTP, you’ll need werkzeug installed. Finally, if you want a simple, pure-python way of running that WSGI app, you’ll want to install cherrypy.
Release History

Release History

0.1

This version

History Node

TODO: Figure out how to actually get changelog content.

Changelog content for this version goes here.

Donec et mollis dolor. Praesent et diam eget libero egestas mattis sit amet vitae augue. Nam tincidunt congue enim, ut porta lorem lacinia consectetur. Donec ut libero sed arcu vehicula ultricies a non tortor. Lorem ipsum dolor sit amet, consectetur adipiscing elit.

Show More

Download Files

Download Files

TODO: Brief introduction on what you do with files - including link to relevant help section.

File Name & Checksum SHA256 Checksum Help Version File Type Upload Date
awesomestream-0.1.tar.gz (8.4 kB) Copy SHA256 Checksum SHA256 Source Dec 14, 2009

Supported By

WebFaction WebFaction Technical Writing Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS HPE HPE Development Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Rackspace Rackspace Cloud Servers DreamHost DreamHost Log Hosting