Skip to main content
LitProxy Logo

LitProxy

Modern, Easy-to-Use Python Proxy Management Library

Intelligent proxy rotation, seamless HTTP client patching, context management, and comprehensive diagnostics for Python applications. Make proxy usage simple and universal across any Python project.

PyPI Version Python Version License Status


📋 Table of Contents


🌟 Key Features

🔄 Intelligent Proxy Management

  • Smart Rotation: Automatic proxy rotation with random selection (no priority)
  • Health Monitoring: Built-in proxy testing and health checks with configurable timeouts
  • Caching System: Efficient proxy caching with background refresh to minimize latency
  • Fallback Support: Automatic fallback to alternative proxies when primary sources fail

🔌 Universal HTTP Client Support

  • Requests Integration: Seamless integration with Python's requests library
  • HTTPX Support: Full compatibility with modern httpx async/sync clients
  • Curl_cffi Support: Advanced browser impersonation with curl_cffi sessions
  • Auto-Patching: Automatic proxy injection into existing HTTP sessions

🎯 Developer-Friendly API

  • Context Managers: Clean proxy usage with Python context managers
  • Decorators: Simple function decoration for automatic proxy injection
  • Metaclass Magic: Automatic proxy injection for class-based applications
  • Static Methods: Direct access to proxy functionality without instantiation

🛡️ Reliability & Diagnostics

  • Auto-Retry Logic: Intelligent retry mechanisms with proxy rotation on failures
  • Comprehensive Testing: Built-in proxy validation and performance testing
  • Statistics & Monitoring: Detailed proxy usage statistics and cache metrics
  • Error Handling: Graceful degradation when proxies are unavailable


⚙️ Installation

📦 Standard Installation

# Install from PyPI
pip install litproxy

# Install from local directory
pip install .

# Install in development mode
pip install -e .

🔧 Dependencies

Required:

  • requests>=2.25.0

Optional (for extended functionality):

  • httpx - For modern async/sync HTTP client support
  • curl_cffi - For advanced browser impersonation capabilities
# Install with optional dependencies
pip install httpx curl_cffi

🚀 Quick Start

Basic Usage

from litproxy import proxy, use_proxy, patch, proxyify, get_proxy_dict
import requests

# Method 1: Context Manager (Recommended)
with use_proxy():
    response = requests.get("https://httpbin.org/ip")
    print(response.json())

# Method 2: Direct Proxy Dictionary
proxies = get_proxy_dict()
response = requests.get("https://httpbin.org/ip", proxies=proxies)

# Method 3: One-liner for a working proxy
proxies = proxy()
response = requests.get("https://httpbin.org/ip", proxies=proxies)

# Method 4: Decorator
@proxyify
def fetch_data(url):
    return requests.get(url)

result = fetch_data("https://httpbin.org/ip")

# Method 5: Patch a session or function
session = requests.Session()
patch(session)
response = session.get("https://httpbin.org/ip")

Auto-Retry with Proxy Rotation

from litproxy import LitProxy

# Create session with automatic retry and proxy rotation
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)

# This will automatically retry with different proxies on failure
response = session.get("https://httpbin.org/ip")

📖 Usage Examples

Context Manager Usage

from litproxy import use_proxy
import requests

# Global proxy patching for all requests
with use_proxy():
    response1 = requests.get("https://httpbin.org/ip")
    response2 = requests.get("https://httpbin.org/user-agent")
    print("IP 1:", response1.json()['origin'])
    print("IP 2:", response2.json()['origin'])

Decorator Pattern

from litproxy import proxyify

@proxyify
def scrape_website(url):
    import requests
    return requests.get(url).text

content = scrape_website("https://httpbin.org/html")

Patch Usage

from litproxy import patch
import requests

session = requests.Session()
patch(session)
response = session.get("https://httpbin.org/ip")

Class-Based Applications with Metaclass

from litproxy import LitMeta
import requests

class WebScraper(metaclass=LitMeta):
    def __init__(self):
        self.session = requests.Session()
    def scrape(self, url):
        return self.session.get(url)

scraper = WebScraper()
response = scraper.scrape("https://httpbin.org/ip")

🔧 Advanced Features

Manual Proxy Selection

from litproxy import get_working_proxy, get_auto_proxy, test_proxy, list_proxies, refresh_proxy_cache

# Get a random working proxy from all available sources
random_proxy = get_working_proxy()
any_proxy = get_auto_proxy()          # Random selection

# Test specific proxy
proxy_url = "http://proxy.example.com:8080"
is_working = test_proxy(proxy_url, timeout=10)

# Get all available proxies
all_proxies = list_proxies()

# Force refresh proxy cache
proxy_count = refresh_proxy_cache()
print(f"Loaded {proxy_count} proxies")

Proxy Testing and Diagnostics

from litproxy import test_all_proxies, get_proxy_stats

# Test all available proxies
results = test_all_proxies(timeout=5)
for proxy, status in results.items():
    print(f"{proxy}: {'✓' if status else '✗'}")

# Get proxy statistics
stats = get_proxy_stats()
print(f"Available proxies: {stats['proxy_count']}")
print(f"Cache age: {stats['cache_age_seconds']} seconds")

Custom Configuration

from litproxy import LitProxy

# Configure cache duration (default: 300 seconds)
LitProxy.set_proxy_cache_duration(600)  # 10 minutes

# Manual request with retry logic
response = LitProxy.make_request_with_auto_retry(
    method="GET",
    url="https://httpbin.org/ip",
    max_proxy_attempts=3,
    timeout=10
)

# Enable/disable auto-retry for existing provider instances
LitProxy.enable_auto_retry_for_provider(my_provider, max_proxy_attempts=5)
LitProxy.disable_auto_retry_for_provider(my_provider)

Enhanced Auto-Retry Features

from litproxy import LitProxy

# Create session with enhanced auto-retry
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)

# Make requests with automatic proxy rotation on failure
response = session.get("https://httpbin.org/ip")

# Direct auto-retry request without session
response = LitProxy.make_request_with_auto_retry(
    method="POST",
    url="https://httpbin.org/post",
    json={"data": "test"},
    max_proxy_attempts=5,
    timeout=15
)

# Enhanced decorator with intelligent proxy fallback
@LitProxy.auto_retry_with_fallback(max_proxy_attempts=3)
def robust_api_call(url, data):
    import requests
    return requests.post(url, json=data)

result = robust_api_call("https://api.example.com/data", {"key": "value"})

🌐 Proxy Sources

LitProxy supports multiple proxy sources with no prioritization:

1. Webshare Proxies

  • Premium rotating proxies from Webshare.io
  • High reliability and performance
  • Used equally with other sources

2. Remote Proxy Lists

  • Dynamic proxy lists fetched from remote sources
  • Automatically updated and cached
  • Background refresh to maintain availability

3. NordVPN Proxies

  • Static NordVPN proxy endpoints
  • Used equally with other sources
  • Reliable but may have usage limitations

🛠️ HTTP Client Support

Requests Library

import requests
from litproxy import LitProxy

# Method 1: Session patching with auto-retry
session = requests.Session()
LitProxy.patch(session)  # Adds proxy support
response = session.get("https://httpbin.org/ip")

# Method 2: Enhanced session with auto-retry
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)
response = session.get("https://httpbin.org/ip")  # Automatically retries with different proxies

# Method 3: Direct proxy injection
proxies = LitProxy.get_proxy_dict()
response = requests.get("https://httpbin.org/ip", proxies=proxies)

HTTPX Client

import httpx
from litproxy import LitProxy

# Sync client with proxy support
client = LitProxy.get_proxied_httpx_client()
response = client.get("https://httpbin.org/ip")

# Patch existing httpx client
client = httpx.Client()
LitProxy.patch(client)  # Adds proxy support
response = client.get("https://httpbin.org/ip")

# Async client
async with LitProxy.get_proxied_httpx_client() as client:
    response = await client.get("https://httpbin.org/ip")

Curl_cffi Sessions

from litproxy import LitProxy

# Sync session with browser impersonation
session = LitProxy.get_proxied_curl_session(impersonate="chrome120")
response = session.get("https://httpbin.org/ip")

# Async session
async_session = LitProxy.get_proxied_curl_async_session(impersonate="safari15_5")

📊 Diagnostics & Monitoring

Proxy Statistics

from litproxy import LitProxy

stats = LitProxy.get_proxy_stats()
print(f"""
Proxy Statistics:
- Available Proxies: {stats['proxy_count']}
- Last Updated: {stats['last_updated']}
- Cache Duration: {stats['cache_duration']} seconds
- Cache Age: {stats['cache_age_seconds']} seconds
- Source URL: {stats['source_url']}
""")

Health Monitoring

from litproxy import LitProxy

# Test current proxy
current = LitProxy.current_proxy()
if current:
    is_healthy = LitProxy.test_proxy(current)
    print(f"Current proxy {current} is {'healthy' if is_healthy else 'unhealthy'}")

# Comprehensive health check
health_report = LitProxy.test_all_proxies()
healthy_count = sum(1 for status in health_report.values() if status)
total_count = len(health_report)
print(f"Healthy proxies: {healthy_count}/{total_count}")

🤝 Contributing

We welcome contributions to LitProxy! Here's how you can help:

Development Setup

# Clone the repository
git clone https://github.com/OEvortex/Webscout.git
cd Webscout/Litproxy

# Install in development mode
pip install -e .

# Install development dependencies
pip install pytest httpx curl_cffi

Running Tests

# Run basic tests
python -m pytest

# Test with different HTTP clients
python -c "from litproxy import LitProxy; print('✓ Basic import works')"

Areas for Contribution

  • 🐛 Bug Reports: Found an issue? Please report it!
  • 🚀 Feature Requests: Have ideas for improvements?
  • 📖 Documentation: Help improve our docs and examples
  • 🧪 Testing: Add test cases for better coverage
  • 🔌 Integrations: Support for additional HTTP clients

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.


Made with ❤️ by the Webscout Team

🏠 Main Project • 🐛 Report Bug • 💡 Request Feature

Metadata

Release files for litproxy 0.2.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for litproxy 0.2.1
File Size Uploaded
litproxy-0.2.1.tar.gz 7.8 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for litproxy 0.2.1
File Interpreter ABI Platform
litproxy-0.2.1-py3-none-any.whl Python 3 none any Details

Total release size: 14.6 kB

Release files / litproxy-0.2.1.tar.gz

Download URL litproxy-0.2.1.tar.gz
Size 7.8 kB
Tags Source
SHA-256 checksum
How to use checksums
ea836079c75a209aebcd6c6d04423706b9ff884c54e67f49ab935b0a2dc1f171
BLAKE2b-256 checksum
How to use checksums
381ebf6b6c44439a4193b55cba808f5c5ee8f52180b9470f6c3f4f8caabc2117
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.5

Release files / litproxy-0.2.1-py3-none-any.whl

Download URL litproxy-0.2.1-py3-none-any.whl
Size 6.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
8a235953e30bd951f74e53a83cce9517b0d8f350f2868177f4f836a25d449afc
BLAKE2b-256 checksum
How to use checksums
0275e6536d266deaf6ec0de45dfeb9b64c33939652c70968d0a5b137cb92acd0
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.8.5

Release history Release notifications | RSS feed

This release

0.2.1 This release

2 release files

0.2.0

2 release files

0.1.3

2 release files

0.1.2

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page