TLS Requests

TLS Requests is a powerful Python library for secure HTTP requests, offering browser-like TLS client, fingerprinting, anti-bot page bypass, and high performance.

Installation

To install the library, you can choose between two methods:

1. Install via PyPI:

pip install wrapper-tls-requests

2. Install via GitHub Repository:

pip install git+https://github.com/thewebscraping/tls-requests.git

Quick Start

Start using TLS Requests with just a few lines of code. It automatically synchronizes headers based on your chosen browser identifier:

import tls_requests
# The library automatically injects matching User-Agent and Sec-CH-UA headers
r = tls_requests.get("https://httpbin.org/headers", client_identifier="chrome_133")
r.json()["headers"]["User-Agent"]
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36'

Basic automatically rotates for proxies and TLS identifiers:

import tls_requests
proxy_rotator = tls_requests.ProxyRotator([
    "http://user1:pass1@proxy.example.com:8080",
    "http://user2:pass2@proxy.example.com:8081",
    "socks5://proxy.example.com:8082",
    "proxy.example.com:8083",  # defaults to http
    "http://user:pass@proxy.example.com:8084|1.0|US",  # weight and region support
])
r = tls_requests.get(
    "https://httpbin.org/get",
    proxy=proxy_rotator,
    client_identifier=tls_requests.TLSIdentifierRotator()
)
r
<Response [200 OK]>
r.status_code
200

Introduction

TLS Requests is a cutting-edge HTTP client for Python, offering a feature-rich, highly configurable alternative to the popular requests library.

Built on top of tls-client, it combines ease of use with advanced functionality for secure networking.

Acknowledgment: A big thank you to all contributors for their support!

Key Benefits

• Bypass TLS Fingerprinting: Mimic browser-like behaviors to navigate sophisticated anti-bot systems.
• Customizable TLS Client: Select specific TLS fingerprints to meet your needs.
• Ideal for Developers: Build scrapers, API clients, or other custom networking tools effortlessly.

Why Use TLS Requests?

Modern websites increasingly use TLS Fingerprinting and anti-bot tools like Cloudflare Bot Fight Mode to block web crawlers.

TLS Requests bypass these obstacles by mimicking browser-like TLS behaviors, making it easy to scrape data or interact with websites that use sophisticated anti-bot measures.

Unlocking Cloudflare Bot Fight Mode

Example Code:

import tls_requests
r = tls_requests.get('https://www.coingecko.com/')
r
<Response [200]>

Key Features

Enhanced Capabilities

• Browser-like TLS Fingerprinting: Enables secure and reliable browser-mimicking connections.
• Dynamic Header Synchronization: Automatically extracts browser versions from client_identifier and injects them into User-Agent and sec-ch-ua headers.
• High-Performance Backend: Built on a Go-based HTTP backend with Protocol Racing (Happy Eyeballs) enabled by default for faster connections.
• Synchronous & Asynchronous Support: Seamlessly switch between synchronous and asynchronous requests.
• Protocol Support: Fully compatible with HTTP/1.1, HTTP/2, and HTTP/3 (Alpha).
• Strict Timeouts: Reliable timeout management for precise control over request durations.

Additional Features

• Internationalized Domain & URL Support: Handles non-ASCII URLs effortlessly.
• Cookie Management: Ensures session-based cookie persistence.
• Authentication: Native support for Basic and Function authentication.
• Content Decoding: Automatic handling of gzip and brotli-encoded responses.
• Hooks: Perfect for logging, monitoring, tracing, or pre/post-processing requests and responses.
• Unicode Support: Effortlessly process Unicode response bodies.
• Advanced TLS Options: Support for protocol_racing, allow_http and stream_id.
• File Uploads: Simplified multipart file upload support.
• Proxy Configuration: Supports Socks5, HTTP, and HTTPS proxies for enhanced privacy.

Documentation

Explore the full capabilities of TLS Requests in the documentation:

• Quickstart Guide: A beginner-friendly guide.
• Advanced Topics: Learn to leverage specialized features.
• Async Support: Handle high-concurrency scenarios.
• Custom TLS Configurations:
  • Wrapper TLS Client
  • TLS Client Profiles
  • Custom TLS Configurations

Read the documentation: thewebscraping.github.io/tls-requests/

Report Issues

Found a bug? Please open an issue.

By reporting an issue you help improve the project.

Credits

Special thanks to bogdanfinn for creating the awesome tls-client.