LitProxy
Modern, Easy-to-Use Python Proxy Management Library
Intelligent proxy rotation, seamless HTTP client patching, context management, and comprehensive diagnostics for Python applications. Make proxy usage simple and universal across any Python project.
📋 Table of Contents
- 🌟 Key Features
- ⚙️ Installation
- 🚀 Quick Start
- 📖 Usage Examples
- 🔧 Advanced Features
- 🌐 Proxy Sources
- 🛠️ HTTP Client Support
- 📊 Diagnostics & Monitoring
- 🤝 Contributing
- 📄 License
🌟 Key Features
🔄 Intelligent Proxy Management
- Smart Rotation: Automatic proxy rotation with random selection (no priority)
- Health Monitoring: Built-in proxy testing and health checks with configurable timeouts
- Caching System: Efficient proxy caching with background refresh to minimize latency
- Fallback Support: Automatic fallback to alternative proxies when primary sources fail
🔌 Universal HTTP Client Support
- Requests Integration: Seamless integration with Python's
requestslibrary - HTTPX Support: Full compatibility with modern
httpxasync/sync clients - Curl_cffi Support: Advanced browser impersonation with
curl_cffisessions - Auto-Patching: Automatic proxy injection into existing HTTP sessions
🎯 Developer-Friendly API
- Context Managers: Clean proxy usage with Python context managers
- Decorators: Simple function decoration for automatic proxy injection
- Metaclass Magic: Automatic proxy injection for class-based applications
- Static Methods: Direct access to proxy functionality without instantiation
🛡️ Reliability & Diagnostics
- Auto-Retry Logic: Intelligent retry mechanisms with proxy rotation on failures
- Comprehensive Testing: Built-in proxy validation and performance testing
- Statistics & Monitoring: Detailed proxy usage statistics and cache metrics
- Error Handling: Graceful degradation when proxies are unavailable
⚙️ Installation
📦 Standard Installation
# Install from PyPI
pip install litproxy
# Install from local directory
pip install .
# Install in development mode
pip install -e .
🔧 Dependencies
Required:
requests>=2.25.0
Optional (for extended functionality):
httpx- For modern async/sync HTTP client supportcurl_cffi- For advanced browser impersonation capabilities
# Install with optional dependencies
pip install httpx curl_cffi
🚀 Quick Start
Basic Usage
from litproxy import proxy, use_proxy, patch, proxyify, get_proxy_dict
import requests
# Method 1: Context Manager (Recommended)
with use_proxy():
response = requests.get("https://httpbin.org/ip")
print(response.json())
# Method 2: Direct Proxy Dictionary
proxies = get_proxy_dict()
response = requests.get("https://httpbin.org/ip", proxies=proxies)
# Method 3: One-liner for a working proxy
proxies = proxy()
response = requests.get("https://httpbin.org/ip", proxies=proxies)
# Method 4: Decorator
@proxyify
def fetch_data(url):
return requests.get(url)
result = fetch_data("https://httpbin.org/ip")
# Method 5: Patch a session or function
session = requests.Session()
patch(session)
response = session.get("https://httpbin.org/ip")
Auto-Retry with Proxy Rotation
from litproxy import LitProxy
# Create session with automatic retry and proxy rotation
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)
# This will automatically retry with different proxies on failure
response = session.get("https://httpbin.org/ip")
📖 Usage Examples
Context Manager Usage
from litproxy import use_proxy
import requests
# Global proxy patching for all requests
with use_proxy():
response1 = requests.get("https://httpbin.org/ip")
response2 = requests.get("https://httpbin.org/user-agent")
print("IP 1:", response1.json()['origin'])
print("IP 2:", response2.json()['origin'])
Decorator Pattern
from litproxy import proxyify
@proxyify
def scrape_website(url):
import requests
return requests.get(url).text
content = scrape_website("https://httpbin.org/html")
Patch Usage
from litproxy import patch
import requests
session = requests.Session()
patch(session)
response = session.get("https://httpbin.org/ip")
Class-Based Applications with Metaclass
from litproxy import LitMeta
import requests
class WebScraper(metaclass=LitMeta):
def __init__(self):
self.session = requests.Session()
def scrape(self, url):
return self.session.get(url)
scraper = WebScraper()
response = scraper.scrape("https://httpbin.org/ip")
🔧 Advanced Features
Manual Proxy Selection
from litproxy import get_working_proxy, get_auto_proxy, test_proxy, list_proxies, refresh_proxy_cache
# Get a random working proxy from all available sources
random_proxy = get_working_proxy()
any_proxy = get_auto_proxy() # Random selection
# Test specific proxy
proxy_url = "http://proxy.example.com:8080"
is_working = test_proxy(proxy_url, timeout=10)
# Get all available proxies
all_proxies = list_proxies()
# Force refresh proxy cache
proxy_count = refresh_proxy_cache()
print(f"Loaded {proxy_count} proxies")
Proxy Testing and Diagnostics
from litproxy import test_all_proxies, get_proxy_stats
# Test all available proxies
results = test_all_proxies(timeout=5)
for proxy, status in results.items():
print(f"{proxy}: {'✓' if status else '✗'}")
# Get proxy statistics
stats = get_proxy_stats()
print(f"Available proxies: {stats['proxy_count']}")
print(f"Cache age: {stats['cache_age_seconds']} seconds")
Custom Configuration
from litproxy import LitProxy
# Configure cache duration (default: 300 seconds)
LitProxy.set_proxy_cache_duration(600) # 10 minutes
# Manual request with retry logic
response = LitProxy.make_request_with_auto_retry(
method="GET",
url="https://httpbin.org/ip",
max_proxy_attempts=3,
timeout=10
)
# Enable/disable auto-retry for existing provider instances
LitProxy.enable_auto_retry_for_provider(my_provider, max_proxy_attempts=5)
LitProxy.disable_auto_retry_for_provider(my_provider)
Enhanced Auto-Retry Features
from litproxy import LitProxy
# Create session with enhanced auto-retry
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)
# Make requests with automatic proxy rotation on failure
response = session.get("https://httpbin.org/ip")
# Direct auto-retry request without session
response = LitProxy.make_request_with_auto_retry(
method="POST",
url="https://httpbin.org/post",
json={"data": "test"},
max_proxy_attempts=5,
timeout=15
)
# Enhanced decorator with intelligent proxy fallback
@LitProxy.auto_retry_with_fallback(max_proxy_attempts=3)
def robust_api_call(url, data):
import requests
return requests.post(url, json=data)
result = robust_api_call("https://api.example.com/data", {"key": "value"})
🌐 Proxy Sources
LitProxy supports multiple proxy sources with no prioritization:
1. Webshare Proxies
- Premium rotating proxies from Webshare.io
- High reliability and performance
- Used equally with other sources
2. Remote Proxy Lists
- Dynamic proxy lists fetched from remote sources
- Automatically updated and cached
- Background refresh to maintain availability
3. NordVPN Proxies
- Static NordVPN proxy endpoints
- Used equally with other sources
- Reliable but may have usage limitations
🛠️ HTTP Client Support
Requests Library
import requests
from litproxy import LitProxy
# Method 1: Session patching with auto-retry
session = requests.Session()
LitProxy.patch(session) # Adds proxy support
response = session.get("https://httpbin.org/ip")
# Method 2: Enhanced session with auto-retry
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)
response = session.get("https://httpbin.org/ip") # Automatically retries with different proxies
# Method 3: Direct proxy injection
proxies = LitProxy.get_proxy_dict()
response = requests.get("https://httpbin.org/ip", proxies=proxies)
HTTPX Client
import httpx
from litproxy import LitProxy
# Sync client with proxy support
client = LitProxy.get_proxied_httpx_client()
response = client.get("https://httpbin.org/ip")
# Patch existing httpx client
client = httpx.Client()
LitProxy.patch(client) # Adds proxy support
response = client.get("https://httpbin.org/ip")
# Async client
async with LitProxy.get_proxied_httpx_client() as client:
response = await client.get("https://httpbin.org/ip")
Curl_cffi Sessions
from litproxy import LitProxy
# Sync session with browser impersonation
session = LitProxy.get_proxied_curl_session(impersonate="chrome120")
response = session.get("https://httpbin.org/ip")
# Async session
async_session = LitProxy.get_proxied_curl_async_session(impersonate="safari15_5")
📊 Diagnostics & Monitoring
Proxy Statistics
from litproxy import LitProxy
stats = LitProxy.get_proxy_stats()
print(f"""
Proxy Statistics:
- Available Proxies: {stats['proxy_count']}
- Last Updated: {stats['last_updated']}
- Cache Duration: {stats['cache_duration']} seconds
- Cache Age: {stats['cache_age_seconds']} seconds
- Source URL: {stats['source_url']}
""")
Health Monitoring
from litproxy import LitProxy
# Test current proxy
current = LitProxy.current_proxy()
if current:
is_healthy = LitProxy.test_proxy(current)
print(f"Current proxy {current} is {'healthy' if is_healthy else 'unhealthy'}")
# Comprehensive health check
health_report = LitProxy.test_all_proxies()
healthy_count = sum(1 for status in health_report.values() if status)
total_count = len(health_report)
print(f"Healthy proxies: {healthy_count}/{total_count}")
🤝 Contributing
We welcome contributions to LitProxy! Here's how you can help:
Development Setup
# Clone the repository
git clone https://github.com/OEvortex/Webscout.git
cd Webscout/Litproxy
# Install in development mode
pip install -e .
# Install development dependencies
pip install pytest httpx curl_cffi
Running Tests
# Run basic tests
python -m pytest
# Test with different HTTP clients
python -c "from litproxy import LitProxy; print('✓ Basic import works')"
Areas for Contribution
- 🐛 Bug Reports: Found an issue? Please report it!
- 🚀 Feature Requests: Have ideas for improvements?
- 📖 Documentation: Help improve our docs and examples
- 🧪 Testing: Add test cases for better coverage
- 🔌 Integrations: Support for additional HTTP clients
📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
Made with ❤️ by the Webscout Team
Metadata
Release files for litproxy 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| litproxy-0.2.1.tar.gz | 7.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| litproxy-0.2.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 14.6 kB
Release files / litproxy-0.2.1.tar.gz
| Download URL | litproxy-0.2.1.tar.gz |
|---|---|
| Size | 7.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ea836079c75a209aebcd6c6d04423706b9ff884c54e67f49ab935b0a2dc1f171
|
|
BLAKE2b-256 checksum How to use checksums |
381ebf6b6c44439a4193b55cba808f5c5ee8f52180b9470f6c3f4f8caabc2117
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.5
|
Release files / litproxy-0.2.1-py3-none-any.whl
| Download URL | litproxy-0.2.1-py3-none-any.whl |
|---|---|
| Size | 6.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8a235953e30bd951f74e53a83cce9517b0d8f350f2868177f4f836a25d449afc
|
|
BLAKE2b-256 checksum How to use checksums |
0275e6536d266deaf6ec0de45dfeb9b64c33939652c70968d0a5b137cb92acd0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.5
|