Pre-release
This release is a pre-release and may not be stable for production use.
curl_cffi
Python binding for curl-impersonate via cffi.
Unlike other pure python http clients like httpx or requests, curl_cffi can
impersonate browsers' TLS signatures or JA3 fingerprints. If you are blocked by some
website for no obvious reason, you can give this package a try.
Features
- Supports JA3/TLS and http2 fingerprints impersonation.
- Much faster than requests/httpx/tls_client, on par with aiohttp/pycurl, see benchmarks.
- Mimics requests API, no need to learn another one.
- Pre-compiled, so you don't have to compile on your machine.
- Supports
asynciowith proxy rotation on each request. - Supports http 2.0, which requests does not.
Install
pip install curl_cffi --upgrade
This should work on Linux(x86_64/aarch64), macOS(Intel/Apple Silicon) and Windows(amd64).
If it does not work on you platform, you may need to compile and install curl-impersonate
first and set some environment variables like LD_LIBRARY_PATH.
Usage
requests-like
from curl_cffi import requests
# Notice the impersonate parameter
r = requests.get("https://tls.browserleaks.com/json", impersonate="chrome110")
print(r.json())
# output: {..., "ja3n_hash": "aa56c057ad164ec4fdcb7a5a283be9fc", ...}
# the js3n fingerprint should be the same as target browser
# http/socks proxies are supported
proxies = {"https": "http://localhost:3128"}
r = requests.get("https://tls.browserleaks.com/json", impersonate="chrome110", proxies=proxies)
proxies = {"https": "socks://localhost:3128"}
r = requests.get("https://tls.browserleaks.com/json", impersonate="chrome110", proxies=proxies)
Sessions
# sessions are supported
s = requests.Session()
# httpbin is a http test website
s.get("https://httpbin.org/cookies/set/foo/bar")
print(s.cookies)
# <Cookies[<Cookie foo=bar for httpbin.org />]>
r = s.get("https://httpbin.org/cookies")
print(r.json())
# {'cookies': {'foo': 'bar'}}
Supported impersonate versions, as supported by curl-impersonate:
- chrome99
- chrome100
- chrome101
- chrome104
- chrome107
- chrome110
- chrome99_android
- edge99
- edge101
- safari15_3
- safari15_5
asyncio
from curl_cffi.requests import AsyncSession
async with AsyncSession() as s:
r = await s.get("https://example.com")
More concurrency:
import asyncio
from curl_cffi.requests import AsyncSession
urls = [
"https://googel.com/",
"https://facebook.com/",
"https://twitter.com/",
]
async with AsyncSession() as s:
tasks = []
for url in urls:
task = s.get("https://example.com")
tasks.append(task)
results = await asyncio.gather(*tasks)
curl-like
Alternatively, you can use the low-level curl-like API:
from curl_cffi import Curl, CurlOpt
from io import BytesIO
buffer = BytesIO()
c = Curl()
c.setopt(CurlOpt.URL, b'https://tls.browserleaks.com/json')
c.setopt(CurlOpt.WRITEDATA, buffer)
c.impersonate("chrome110")
c.perform()
c.close()
body = buffer.getvalue()
print(body.decode())
See the docs for more details.
Acknowledgement
- Originally forked from multippt/python_curl_cffi, which is under the MIT license.
- Headers/Cookies files are copied from httpx, which is under the BSD license.
- Asyncio support is inspired by Tornado's curl http client.
1 maintainer
Metadata
1 maintainer
Release files for curl-cffi 0.5.9b1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| curl_cffi-0.5.9b1.tar.gz | 31.1 kB | Details |
Built distributions (wheels)
| File | Reset | |||
|---|---|---|---|---|
| curl_cffi-0.5.9b1-cp37-abi3-win_amd64.whl | CPython 3.7 | abi3 | Windows x86-64 | Details |
| curl_cffi-0.5.9b1-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl | CPython 3.7 | abi3 | Linux glibc 2.17+ x86-64 | Details |
| curl_cffi-0.5.9b1-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl | CPython 3.7 | abi3 | Linux glibc 2.17+ ARM64 | Details |
| curl_cffi-0.5.9b1-cp37-abi3-macosx_11_0_arm64.whl | CPython 3.7 | abi3 | macOS 11.0+ ARM64 | Details |
| curl_cffi-0.5.9b1-cp37-abi3-macosx_10_9_x86_64.whl | CPython 3.7 | abi3 | macOS 10.9+ x86-64 | Details |
Total release size: 23.8 MB
Release files / curl_cffi-0.5.9b1.tar.gz
| Download URL | curl_cffi-0.5.9b1.tar.gz |
|---|---|
| Size | 31.1 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8fdc9a1ea5ca331f45c430abb71f4d22e3997b1720606fab734c634fe9786bd5
|
|
BLAKE2b-256 checksum How to use checksums |
cbf9de2f529403479ef2dc0c2ec2ac89c8eb8f94e4ec26b5072d558f59643f81
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.17
|
Release files / curl_cffi-0.5.9b1-cp37-abi3-win_amd64.whl
| Download URL | curl_cffi-0.5.9b1-cp37-abi3-win_amd64.whl |
|---|---|
| Size | 2.6 MB |
| Tags | CPython 3.7 Windows x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
89db264e6a9dc031f6d0ed6e65f52f9f0697164f9a75325680d602134bf25e18
|
|
BLAKE2b-256 checksum How to use checksums |
7148a61590e3fe8fb9a5c802417297514712005717e630114abaf43a842ccc34
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.17
|
Release files / curl_cffi-0.5.9b1-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
| Download URL | curl_cffi-0.5.9b1-cp37-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl |
|---|---|
| Size | 7.2 MB |
| Tags | CPython 3.7 Linux glibc 2.17+ x86-64 abi3 |
|
SHA-256 checksum How to use checksums |
7a67f6cc8c2c18f5b414466f3f41cd4de95f847122a83507050282db8b9c4e91
|
|
BLAKE2b-256 checksum How to use checksums |
3eace43c18385538d4b02a32a3d106542e6f48a15fa310b3e467aedf06a91161
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.17
|
Release files / curl_cffi-0.5.9b1-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
| Download URL | curl_cffi-0.5.9b1-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
|---|---|
| Size | 6.9 MB |
| Tags | CPython 3.7 Linux glibc 2.17+ ARM64 abi3 |
|
SHA-256 checksum How to use checksums |
f76c5ef35227c0a9da74b1bd239c2992b3f6ff3e4f45756d50faf772990d6d13
|
|
BLAKE2b-256 checksum How to use checksums |
5cac413a5779a41a889f6d3ea5b92bfdbe5e7f3d9ecc7f0bb94977a28d3f7e69
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.17
|
Release files / curl_cffi-0.5.9b1-cp37-abi3-macosx_11_0_arm64.whl
| Download URL | curl_cffi-0.5.9b1-cp37-abi3-macosx_11_0_arm64.whl |
|---|---|
| Size | 2.2 MB |
| Tags | CPython 3.7 abi3 macOS 11.0+ ARM64 |
|
SHA-256 checksum How to use checksums |
b1e9c8d3d910f08b8871d0410df0ee24eb94bcc8edaa620230c78ad7fc5d511e
|
|
BLAKE2b-256 checksum How to use checksums |
7b7504eb126cb7092fc99f9bf417f0a051af5e0137ade119b2203a14f963c91f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.8.16
|
Release files / curl_cffi-0.5.9b1-cp37-abi3-macosx_10_9_x86_64.whl
| Download URL | curl_cffi-0.5.9b1-cp37-abi3-macosx_10_9_x86_64.whl |
|---|---|
| Size | 4.9 MB |
| Tags | CPython 3.7 abi3 macOS 10.9+ x86-64 |
|
SHA-256 checksum How to use checksums |
3c798eafc6603127e485fce3e5a20eb967ca863c4a5ab3866ec0c2e799cea368
|
|
BLAKE2b-256 checksum How to use checksums |
89b4b86e23eb73740cfad811d75e4f4bd2b501fe6f71d387571f506eb0a31718
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/4.0.2 CPython/3.9.17
|
