Skip to main content

Scrape the Twitter Frontend API without authentication.

Project description

Twitter Scraper

Twitter’s API is annoying to work with, and has lots of limitations — luckily their frontend (JavaScript) has it’s own API, which I reverse–engineered. No API rate limits. No restrictions. Extremely fast.

You can use this library to get the text of any user’s Tweets trivially.

Very useful for making markov chains.

Usage

>>> from twitter_scraper import get_tweets

>>> for tweet in get_tweets('kennethreitz', pages=1):
>>>     print(tweet)
P.S. your API is a user interface
s3monkey just hit 100 github stars! Thanks, y’all!
I’m not sure what this /dev/fd/5 business is, but it’s driving me up the wall.
…

It appears you can ask for up to 25 pages of tweets reliably (~486 tweets).

Markov Example

First, install markovify:

$ pipenv install markovify
>>> import markovify

>>> tweets = '\n'.join([t for t in get_tweets('kennethreitz', pages=25)])
>>> text_model = markovify.Text(tweets)

>>> print(text_model.make_short_sentence(140))
Wtf you can’t use APFS on a prototype for “django-heroku”, which does a lot out of me.

Installation

$ pipenv install twitter-scraper

Only Python 3.6+ is supported.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

twitter-scraper-0.1.2.tar.gz (3.5 kB view hashes)

Uploaded Source

Built Distribution

twitter_scraper-0.1.2-py2.py3-none-any.whl (4.4 kB view hashes)

Uploaded Python 2 Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page