twscrape
Twitter GraphQL and Search API implementation with SNScrape data models.
Install
pip install https://github.com/vladkens/tw-api
Features
- Support both Search & GraphQL Twitter API
- Async / Await functions (can run multiple scrappers in parallel same time)
- Login flow (with receiving verification code from email)
- Saving / restore accounts sessions
- Raw Twitter API responses & SNScrape models
- Automatic account switching to smooth Twitter API rate limits
Usage
import asyncio
from twscrape import AccountsPool, API, gather
from twscrape.logger import set_log_level
async def main():
pool = AccountsPool() # or pool = AccountsPool("path-to.db") - default is `accounts.db`
await pool.add_account("user1", "pass1", "user1@example.com", "email_pass1")
await pool.add_account("user2", "pass2", "user2@example.com", "email_pass2")
# log in to all fresh accounts
await pool.login_all()
api = API(pool)
# search api
await gather(api.search("elon musk", limit=20)) # list[Tweet]
# graphql api
tweet_id = 20
user_id, user_login = 2244994945, "twitterdev"
await api.tweet_details(tweet_id) # Tweet
await gather(api.retweeters(tweet_id, limit=20)) # list[User]
await gather(api.favoriters(tweet_id, limit=20)) # list[User]
await api.user_by_id(user_id) # User
await api.user_by_login(user_login) # User
await gather(api.followers(user_id, limit=20)) # list[User]
await gather(api.following(user_id, limit=20)) # list[User]
await gather(api.user_tweets(user_id, limit=20)) # list[Tweet]
await gather(api.user_tweets_and_replies(user_id, limit=20)) # list[Tweet]
# note 1: limit is optional, default is -1 (no limit)
# note 2: all methods have `raw` version e.g.:
async for tweet in api.search("elon musk"):
print(tweet.id, tweet.user.username, tweet.rawContent) # tweet is `Tweet` object
async for rep in api.search_raw("elon musk"):
print(rep.status_code, rep.json()) # rep is `httpx.Response` object
# change log level, default info
set_log_level("DEBUG")
# Tweet & User model can be converted to regular dict or json, e.g.:
doc = await api.user_by_id(user_id) # User
doc.dict() # -> python dict
doc.json() # -> json string
if __name__ == "__main__":
asyncio.run(main())
You can use login_all once in your program to pass the login flow and add the accounts to the database. Re-runs will use the previously activated accounts.
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
twscrape-0.1.0.tar.gz
(128.9 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
twscrape-0.1.0-py3-none-any.whl
(17.7 kB
view details)
File details
Details for the file twscrape-0.1.0.tar.gz.
File metadata
- Download URL: twscrape-0.1.0.tar.gz
- Upload date:
- Size: 128.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/4.0.1 CPython/3.11.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
3e27449bbf25a6f22dc726a6eeb98fb78f0ccd6dfbe161bcef99d8f701667ab0
|
|
| MD5 |
7dfd6de903454f1c4cf94b4d5cb398e9
|
|
| BLAKE2b-256 |
ce301dd94a3fece55246a83dc454d2d7a2de3a2d6acd1150eaef5204583eaba4
|
File details
Details for the file twscrape-0.1.0-py3-none-any.whl.
File metadata
- Download URL: twscrape-0.1.0-py3-none-any.whl
- Upload date:
- Size: 17.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/4.0.1 CPython/3.11.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
57ebb0e3d43a0b4a576bca76614a86973c5480f63eed1babfbfd6cddfcf505e7
|
|
| MD5 |
fe638f815d50a87f1cf18526b7f44540
|
|
| BLAKE2b-256 |
2a59fcb28ffadb4dde35c593771a7168e883bb9a47281d223fb1b49d79873a08
|
