tweetvac 1.0.1

Import tweetvac

import tweetvac

Create a TweetVac instance

You can pass the OAuth parameters as a tuple:

vac = tweetvac.TweetVac((consumer_key, consumer_secret, oauth_token, oauth_token_secret))

or use the configuration object:

config = tweetvac.AuthConfig()
vac = tweetvac.TweetVac(config)

Suck down tweets

tweetvac expects a Twitter endpoint and a dictionary of parameters for that endpoint. Read the Twitter documentation for a list of endpoints and their parameters. It is recommended to set the count option in the params dict to the largest value supported by that endpoint.

params = {'screen_name': 'struckDC', 'count': 200}
data = vac.suck('statuses/user_timeline', params)

Work with the data

The data returned is a list of dicts. The fields in the dict are listed in the Twitter API documentation on the Tweet object.

The data can be converted back to json and stored to a file like this:

with open('data.json', 'w') as outfile:
    json.dump(data, outfile)

Filtering the tweets

Twitter provides several parameters on each endpoint for selecting what tweets you want to retrieve. Additional culling is available by passing a list of filter functions.

def remove_mention_tweets(tweet):
    return not '@' in tweet['text']

data = vac.suck('statuses/user_timeline', params, filters=[remove_mention_tweets])

Return false from your function to remove the tweet from the list.

Turning off the vacuum

tweetvac will suck down tweets until you reach your rate limit or you consume all the available tweets. To stop sooner, you can pass a cutoff function that returns true when tweetvac should stop.

def stop(tweet):
    cutoff_date = time.strptime("Wed Jan 01 00:00:00 +0000 2014", '%a %b %d %H:%M:%S +0000 %Y')
    tweet_date = time.strptime(tweet['created_at'], '%a %b %d %H:%M:%S +0000 %Y')
    return tweet_date < cutoff_date

data = vac.suck('statuses/user_timeline', params, cutoff=stop)

You can also pass a hard limit to the number of requests to stop tweetvac early:

data = vac.suck('statuses/user_timeline', params, max_requests=10)