A newslynx-opinionated wrapper around twython
Project description
birdfeeder ====== Feed on tweets
Install
pip install birdfeeder
Test
Requires nose
nosetests
Usage
birdfeeder simplifies twython by taking care of alot the common problems in working with the Twitter API, including:
- pagination
- rate limiting
- url unshortening
- error handling
Thus far, we’ve implemented 5 methods: connect, search, list_timeline, user_timeline, and user_stats (a custom method):
Connect
If you have TWT_API_KEY, TWT_API_SECRET, TWT_ACCESS_TOKEN, and TWT_ACCESS_SECRET set as environmental variables, you can use all the methods without explicitly connecting. However, if you want to explicitly connect beforehand, you can also pass in a connection as conn to any method:
import birdfeeder conn = birdfeeder.connect() tweets = birdfeeder.search(q="hello world", count=10, conn=conn) for t in tweets: print t
In addition, you can also pass in an authenticated user’s token with access_token:
import birdfeeder for t in birdfeeder.search(q="hello world", access_token="authenticed_users_token"): print t
Search
import birdfeeder tweets = birdfeeder.search(q="hello world", count=10) for t in tweets: print t
List Timeline
import birdfeeder tweets = birdfeeder.list_timeline(owner_screen_name = 'cspan', slug = 'members-of-congress', count=100) for t in tweets: print t
User Timeline
import birdfeeder tweets = birdfeeder.user_timeline(screen_name = 'newslynx') for t in tweets: print t
User Stats
This returns a dictionary of stats about a user, with the time it ran. It’s intended for creating a time series of a user’s metadata:
import birdfeeder stats = birdfeeder.user_stats(screen_name = "newslynx") print stats
Pagination
With birdfeeder, pagination is simple, just add pagination=True to any method, ie:
import birdfeeder tweets = birdfeeder.user_timeline(screen_name = 'newslynx', count = 5, pagination=True) for t in tweets: print t
This will keep track of the max_id of each request and iterate through results until everything has been retrieved (or until otherwise specified - more below). For each request, it will wait for a defualt of 15 seconds so as to avoid rate limiting.
Concurrency
add concurrency to any method via gevent:
tweets = birdfeeder.user_timeline(screen_name="brianabelson", concurrent=True) for t in tweets: print t
Custom Options
We’ve added some custom options for each method, they are as follows:
- throttle - the time in seconds to wait between each request (only relevant when paginate = True)
- max_requests - the maximum number of requests to make (only relevant when paginate = True)
- wait - the default number of seconds to wait after an error
- backoff - the factor to multiply wait by after each error
- timeout - the time in seconds at which point we should abandon an error prone request. Here, birdfeeder will log a warning, but will otherwise fail silently.
Here are the default arguments for all methods:
default_kws = { 'paginate' : False, 'concurrent': False, 'num_workers': 20, 'max_id': None, 'throttle' : 15, 'count' : 200, 'max_requests' : None, 'wait': 1, 'backoff': 2, 'timeout': 30 }
Streaming
Finally, we’ve included a simple streaming API client (from here). With this, you can pass in three functions on initialization: a parsing function, a storage function, and an error function, ie:
from birdfeeder import Stream def parse(data): return data['text'] def store(data): print data def error(status_code, data): pass s = Stream(parse=parse, store=store, error=error) s.statuses.filter(track='twitter')
Acknowledgements
To write this library, I heavily referenced Jeremy Singer-Vine’s excellent `twick <https://github.com/jsvine/twick>`__.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Filename, size | File type | Python version | Upload date | Hashes |
---|---|---|---|---|
Filename, size birdfeeder-0.1.9.macosx-10.9-intel.exe (75.5 kB) | File type Windows Installer | Python version any | Upload date | Hashes View |
Filename, size birdfeeder-0.1.9.tar.gz (6.8 kB) | File type Source | Python version None | Upload date | Hashes View |
Hashes for birdfeeder-0.1.9.macosx-10.9-intel.exe
Algorithm | Hash digest | |
---|---|---|
SHA256 | 621a28d9fb76d7a06b2616804a196cbd3a49208b853e14dfc07235d2417f8d21 |
|
MD5 | 373ee55af4515b1fe6381427100a8af8 |
|
BLAKE2-256 | ceacb44b5838020a1ed42c177558b128b6908335c2ccd7777717a7c6639c286f |