Skip to main content

Universal caching wrapper for any sync HTTP session (curl_cffi, requests, cloudscraper, httpx)

Project description

smart_requests_dvxb

A lightweight wrapper around any sync HTTP session (requests, curl_cffi, cloudscraper, httpx, tls_client) that adds transparent disk caching with simple r / w / rr mode control and response condition validation.


Install

pip install smart_requests_dvxb

Standard vs Smart

Standard requests — fetches every time:

import requests

session = requests.Session()
resp = session.get("https://example.com/data.json")
print(resp.text)  # hits network every single run

SmartSession — fetches once, reads from disk after:

import requests
from smart_session import smart_session

sess = smart_session(requests.Session)
resp = sess.get(
    "https://example.com/data.json",
    mode="r",
    filepath="cache/data.json",
)
print(resp.text)  # network on first run, disk on every run after

Everything else (headers, cookies, params, data, json) works exactly as before.


Works with curl_cffi too

from curl_cffi import Session
from smart_session import smart_session

sess = smart_session(Session, impersonate="chrome124")
resp = sess.get(url, mode="r", filepath="cache/page.html")

Modes

Mode Binary Behaviour
r rb Cache-first — read from disk if cached, else fetch and cache
w wb Always live — fetch fresh every run, overwrite disk cache
rr rrb Read-only — offline replay; raises FileNotFoundError if no cache

Append b to any mode for binary responses (images, PDFs, etc.).


Condition Validation

Conditions are callables evaluated against the live response before writing to disk. If any condition fails, the file is never written and a ValueError is raised — so you never cache a bad response.

Without SmartSession — bad response silently saved:

resp = session.post(url, data=payload)
with open("result.html", "w") as f:
    f.write(resp.text)  # saves error page, captcha page, anything

With SmartSession — only valid responses are cached:

resp = sess.post(
    url,
    data=payload,
    mode="w",
    filepath="cache/result.html",
    conditions=[
        lambda r: r.status_code == 200,
        lambda r: len(r.content) > 500,
        lambda r: "error" not in r.text,
        lambda r: "captcha" not in r.text.lower(),
    ],
)

Full Parameter Reference

Parameter Type Description
mode str One of r, w, rr, rb, wb, rrb
filepath str | Path Path to cache file — required when mode is set
conditions list[callable] Validators run against the live response before writing

Debug Logging

sess = smart_session(Session, debug_mode=True)

Prints per-request cache hits/misses and write confirmations.


Built-in Docs

import smart_session
smart_session.info()   # module overview

sess = smart_session(Session)
sess.docs              # instance documentation
sess.supported         # available verbs + methods on wrapped session

Contact Developer

Dharmik Vadherdharmik.vadher@xbyte.io

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

smart_requests_dvxb-0.22-py3-none-any.whl (7.1 kB view details)

Uploaded Python 3

File details

Details for the file smart_requests_dvxb-0.22-py3-none-any.whl.

File metadata

File hashes

Hashes for smart_requests_dvxb-0.22-py3-none-any.whl
Algorithm Hash digest
SHA256 5ca8d940720717130f4fe81858a4aeb86e8b7d0d36395e321b72e837b7837982
MD5 5bafeaf707b94bd4060b91ac071f7773
BLAKE2b-256 68ee275986e51713ef7678aa52c71446959e7e05bc31b201ede444d1266c1408

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page