Add your description here
Project description
Python requests.HTTPAdapter for acw_sc__v2
acw_sc__v2 is a cookie used by some websites to prevent crawlers.
When the website detects that the request is sent by a crawler, it returns a javascript challenge. The crawler needs to solve the challenge and resend the request with the cookie set to the challenge value.
This project provides a Python requests.HTTPAdapter to resolve the challenge automatically.
Usage
pip install acw-sc-v2-py
# start acw_sc_v2_solver
git clone https://github.com/WangYihang/acw-sc-v2.js.git
cd acw-sc-v2.js
npm install
node app.js
import requests
session = requests.Session()
# add the following code to your original requests code
from acw_sc_v2_py import acw_sc__v2
adapter = acw_sc__v2.AcwScV2Adapter(acw_sc_v2_solver_url="http://localhost:3000/")
session.mount("http://", adapter)
session.mount("https://", adapter)
response = session.get("https://www.example.com/")
print(response.text)
Use Case
Before using acw-sc-v2-py
import requests
session = requests.Session()
for i in range(8):
response = session.get("https://www.example.com/")
print(response.text)
Usually, you will get blocked after sending 2 consecutive requests to the same website. The response will be like the following HTML code which requires you to solve a javascript challenge. If the web page is opened in a browser, the browser will automatically solve the challenge.
<html><script>
var arg1='70D9569CD5E5895C84F284A09503B1598C5762A1';
var _0x4818=['\x63\x73\x4b\x48\x77\x71\x4d\x49,...
function setCookie(name,value){var expiredate=new Date();...
function reload(x) {setCookie("acw_sc__v2", x);...
</script></html>
After using acw-sc-v2-py
import requests
# step 1: import
from acw_sc_v2_py import acw_sc__v2
session = requests.Session()
# step 2: create adapter
adapter = acw_sc__v2.AcwScV2Adapter()
# step 3: mount adapter
session.mount("http://", adapter)
session.mount("https://", adapter)
for i in range(8):
response = session.get("https://www.example.com/")
print(response.text)
By using acw-sc-v2-py, you will get the normal response. The
acw_sc__v2 will handle the javascript challenge and automatically
update the cookie.
Enable logging
Prepend the following code to enable detailed log for acw-sc-v2-py.
import logging
logger = logging.getLogger("acw_sc_v2_py.acw_sc__v2")
logger.setLevel(logging.DEBUG)
logger.propagate = True
import requests
from acw_sc_v2_py import acw_sc__v2
session = requests.Session()
adapter = acw_sc__v2.AcwScV2Adapter()
session.mount("http://", adapter)
session.mount("https://", adapter)
for i in range(8):
response = session.get("https://www.example.com/")
print(response.text)
The log will be like the following.
[2024-01-26 22:01:26] INFO:root:detected anti spam is triggered
[2024-01-26 22:01:28] INFO:root:cookie generated acw_sc__v2=65b3bb3601fe9ab002c5c1ff58fc71a1115e8322
[2024-01-26 22:01:28] INFO:root:resending the origin request
<!DOCTYPE html></html>
[2024-01-26 22:01:29] INFO:root:cookie set acw_sc__v2=65b3bb3601fe9ab002c5c1ff58fc71a1115e8322
[2024-01-26 22:01:30] INFO:root:anti spam is not triggered
<!DOCTYPE html></html>
...
References
- NodeJS version for Server API (acw-sc-v2.js)
- GoLang version for client code (acw-sc-v2-go)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file acw_sc_v2_py-0.0.6.tar.gz.
File metadata
- Download URL: acw_sc_v2_py-0.0.6.tar.gz
- Upload date:
- Size: 3.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.6.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
2389e6970b1a4ebd5edc74e3ea4467256a7541c765a8cb6f4a98fe941d5444d4
|
|
| MD5 |
34e9358aa4b0855a26e54d3037a498e1
|
|
| BLAKE2b-256 |
2e2677b495b6704db8b80e28cc0cf3ecf172fd437cfcb6f4b70f52509bd0ca9e
|
File details
Details for the file acw_sc_v2_py-0.0.6-py3-none-any.whl.
File metadata
- Download URL: acw_sc_v2_py-0.0.6-py3-none-any.whl
- Upload date:
- Size: 4.0 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: uv/0.6.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
6f59916faa9850da765849a7dca3d45a9cef7a5a5c5411cb2d8875577c0f7dab
|
|
| MD5 |
885beda3af7e7f4d7eac9f6d558f9673
|
|
| BLAKE2b-256 |
25e538bc5c6a53662fc9afb68b883a7859bdab951b553f85c4f0181f189b4b03
|
