Scrape the Twitter frontend API without any authentication and restriction.
Project description
Twitter’s API is annoying to work with, and has lots of limitations — luckily their frontend (JavaScript) has it’s own API, which I reverse–engineered. No API rate limits. No restrictions. Extremely fast.
You can use this library to get the text of any user’s Tweets trivially. Follow the creator’s blog at shirishkadam.com for updates on progress.
This project is inspired from Kenneth Reitz’s similar project kennethreitz/twitter-scraper which is limited to python 3.6 anc above.
Getting Started
$ git clone https://github.com/5hirish/tweet_scrapper.git
$ cd tweet_scrapper
$ pip install -r requirements.txt
$ python -m tweetscrape.twitter_scrape -u "@5hirish" -p 3
$ python -m tweetscrape.twitter_scrape -s "#Python" -p 4
$ python -m tweetscrape.twitter_scrape -s "Avengers Infinity War" -p 2
Usage
from tweetscrape.profile_tweets import TweetScrapperProfile
tweet_scrapper = TweetScrapperProfile("@5hirish", 1)
tweets = tweet_scrapper.get_profile_tweets()
for tweet in tweets:
print(str(tweet))
Read more on tweetscrape usage here.
Id: 1056176020368191488 Type: tweet Time: 1540646960000 Author: 5hirish AuthorId: 428808036 ReTweeter: None Associated Tweet: 1056176020368191488 Text: I've completed 7 Pull Requests for #Hacktoberfest! https://hacktoberfest.digitalocean.com/stats/5hirish Always wanted to contribute to #OpenSource, thanks to @digitalocean initiative #Hacktoberfest finally got around doing it. Will keep it up. Links: ['https://t.co/J42KiNKGMG'] Hastags: ['#Hacktoberfest', '#OpenSource', '#Hacktoberfest'] Mentions: ['@digitalocean'] Replies: 0 Favorites: 3 Retweets: 1 Id: 1055883061513084928 Type: tweet Time: 1540577113000 Author: wesmckinn AuthorId: 115494880 ReTweeter: 5hirish Associated Tweet: 1055883061513084928 Text: TFW someone asks "Any update?" or "When is this feature going to be implemented?" on an open source issue tracker. Links: [] Hastags: [] Mentions: [] Replies: 5 Favorites: 84 Retweets: 11 Id: 1055151881377406976 Type: tweet Time: 1540402786000 Author: justinkan AuthorId: 28917111 ReTweeter: 5hirish Associated Tweet: 1055151881377406976 Text: 1/ Actually I’ve learned a lot from @ROWGHANI that’s worth sharing. First, your job as CEO is to: make sure there’s $ in the bank, define the company’s mission, hire the senior team, and do maybe one thing you enjoy (sales, product, etc) Links: [] Hastags: [] Mentions: ['@ROWGHANI'] Replies: 43 Favorites: 2793 Retweets: 700 ....
Requirements
Python Package dependencies listed in requirements.txt
Features
Extract user tweets with all meta-data
Extracts external links, hashtags and mentions from a tweet
Extracts reply, favorite and retweet counts of a tweet
TODO
[x] Extract tweets from a twitter user’s profile
[x] Extract tweets from twitter search
[ ] Extract tweets from a twitter thread, given the thread link
[ ] Extract the quoted tweet along with a tweet
Contributions
Please see the contributing documentation for some tips on getting started.
Maintainers
@5hirish - Shirish Kadam
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for tweetscrape-0.1.6-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1b944214c2e98f798d0be26d1c8fd93dc82b6ffbb00186f7acd1caf687033970 |
|
MD5 | 1dfeb45bb7bd2fb71c0e5f207f4c0d89 |
|
BLAKE2b-256 | 5be2ef30340d6fefbf4ef6e6e7497fc2a8b3c3235a89b7f58e9051573ed747ea |