LitProxy
Modern, Easy-to-Use Python Proxy Management Library
Intelligent proxy rotation, seamless HTTP client patching, context management, and comprehensive diagnostics for Python applications. Make proxy usage simple and universal across any Python project.
📋 Table of Contents
- 🌟 Key Features
- ⚙️ Installation
- 🚀 Quick Start
- 📖 Usage Examples
- 🔧 Advanced Features
- 🌐 Proxy Sources
- 🛠️ HTTP Client Support
- 📊 Diagnostics & Monitoring
- 🤝 Contributing
- 📄 License
🌟 Key Features
🔄 Intelligent Proxy Management
- Smart Rotation: Automatic proxy rotation with random selection (no priority)
- Health Monitoring: Built-in proxy testing and health checks with configurable timeouts
- Caching System: Efficient proxy caching with background refresh to minimize latency
- Fallback Support: Automatic fallback to alternative proxies when primary sources fail
🔌 Universal HTTP Client Support
- Requests Integration: Seamless integration with Python's
requestslibrary - HTTPX Support: Full compatibility with modern
httpxasync/sync clients - Curl_cffi Support: Advanced browser impersonation with
curl_cffisessions - Auto-Patching: Automatic proxy injection into existing HTTP sessions
🎯 Developer-Friendly API
- Context Managers: Clean proxy usage with Python context managers
- Decorators: Simple function decoration for automatic proxy injection
- Metaclass Magic: Automatic proxy injection for class-based applications
- Static Methods: Direct access to proxy functionality without instantiation
🛡️ Reliability & Diagnostics
- Auto-Retry Logic: Intelligent retry mechanisms with proxy rotation on failures
- Comprehensive Testing: Built-in proxy validation and performance testing
- Statistics & Monitoring: Detailed proxy usage statistics and cache metrics
- Error Handling: Graceful degradation when proxies are unavailable
⚙️ Installation
📦 Standard Installation
# Install from PyPI
pip install litproxy
# Install from local directory
pip install .
# Install in development mode
pip install -e .
🔧 Dependencies
Required:
requests>=2.25.0
Optional (for extended functionality):
httpx- For modern async/sync HTTP client supportcurl_cffi- For advanced browser impersonation capabilities
# Install with optional dependencies
pip install httpx curl_cffi
🚀 Quick Start
Basic Usage
from litproxy import proxy, use_proxy, patch, proxyify, get_proxy_dict
import requests
# Method 1: Context Manager (Recommended)
with use_proxy():
response = requests.get("https://httpbin.org/ip")
print(response.json())
# Method 2: Direct Proxy Dictionary
proxies = get_proxy_dict()
response = requests.get("https://httpbin.org/ip", proxies=proxies)
# Method 3: One-liner for a working proxy
proxies = proxy()
response = requests.get("https://httpbin.org/ip", proxies=proxies)
# Method 4: Decorator
@proxyify
def fetch_data(url):
return requests.get(url)
result = fetch_data("https://httpbin.org/ip")
# Method 5: Patch a session or function
session = requests.Session()
patch(session)
response = session.get("https://httpbin.org/ip")
Auto-Retry with Proxy Rotation
from litproxy import LitProxy
# Create session with automatic retry and proxy rotation
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)
# This will automatically retry with different proxies on failure
response = session.get("https://httpbin.org/ip")
📖 Usage Examples
Context Manager Usage
from litproxy import use_proxy
import requests
# Global proxy patching for all requests
with use_proxy():
response1 = requests.get("https://httpbin.org/ip")
response2 = requests.get("https://httpbin.org/user-agent")
print("IP 1:", response1.json()['origin'])
print("IP 2:", response2.json()['origin'])
Decorator Pattern
from litproxy import proxyify
@proxyify
def scrape_website(url):
import requests
return requests.get(url).text
content = scrape_website("https://httpbin.org/html")
Patch Usage
from litproxy import patch
import requests
session = requests.Session()
patch(session)
response = session.get("https://httpbin.org/ip")
Class-Based Applications with Metaclass
from litproxy import LitMeta
import requests
class WebScraper(metaclass=LitMeta):
def __init__(self):
self.session = requests.Session()
def scrape(self, url):
return self.session.get(url)
scraper = WebScraper()
response = scraper.scrape("https://httpbin.org/ip")
🔧 Advanced Features
Manual Proxy Selection
from litproxy import get_working_proxy, get_auto_proxy, test_proxy, list_proxies, refresh_proxy_cache
# Get a random working proxy from all available sources
random_proxy = get_working_proxy()
any_proxy = get_auto_proxy() # Random selection
# Test specific proxy
proxy_url = "http://proxy.example.com:8080"
is_working = test_proxy(proxy_url, timeout=10)
# Get all available proxies
all_proxies = list_proxies()
# Force refresh proxy cache
proxy_count = refresh_proxy_cache()
print(f"Loaded {proxy_count} proxies")
Proxy Testing and Diagnostics
from litproxy import test_all_proxies, get_proxy_stats
# Test all available proxies
results = test_all_proxies(timeout=5)
for proxy, status in results.items():
print(f"{proxy}: {'✓' if status else '✗'}")
# Get proxy statistics
stats = get_proxy_stats()
print(f"Available proxies: {stats['proxy_count']}")
print(f"Cache age: {stats['cache_age_seconds']} seconds")
Custom Configuration
from litproxy import LitProxy
# Configure cache duration (default: 300 seconds)
LitProxy.set_proxy_cache_duration(600) # 10 minutes
# Manual request with retry logic
response = LitProxy.make_request_with_auto_retry(
method="GET",
url="https://httpbin.org/ip",
max_proxy_attempts=3,
timeout=10
)
# Enable/disable auto-retry for existing provider instances
LitProxy.enable_auto_retry_for_provider(my_provider, max_proxy_attempts=5)
LitProxy.disable_auto_retry_for_provider(my_provider)
Enhanced Auto-Retry Features
from litproxy import LitProxy
# Create session with enhanced auto-retry
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)
# Make requests with automatic proxy rotation on failure
response = session.get("https://httpbin.org/ip")
# Direct auto-retry request without session
response = LitProxy.make_request_with_auto_retry(
method="POST",
url="https://httpbin.org/post",
json={"data": "test"},
max_proxy_attempts=5,
timeout=15
)
# Enhanced decorator with intelligent proxy fallback
@LitProxy.auto_retry_with_fallback(max_proxy_attempts=3)
def robust_api_call(url, data):
import requests
return requests.post(url, json=data)
result = robust_api_call("https://api.example.com/data", {"key": "value"})
🌐 Proxy Sources
LitProxy supports multiple proxy sources with no prioritization:
1. Webshare Proxies
- Premium rotating proxies from Webshare.io
- High reliability and performance
- Used equally with other sources
2. Remote Proxy Lists
- Dynamic proxy lists fetched from remote sources
- Automatically updated and cached
- Background refresh to maintain availability
3. NordVPN Proxies
- Static NordVPN proxy endpoints
- Used equally with other sources
- Reliable but may have usage limitations
🛠️ HTTP Client Support
Requests Library
import requests
from litproxy import LitProxy
# Method 1: Session patching with auto-retry
session = requests.Session()
LitProxy.patch(session) # Adds proxy support
response = session.get("https://httpbin.org/ip")
# Method 2: Enhanced session with auto-retry
session = LitProxy.create_auto_retry_session(max_proxy_attempts=3)
response = session.get("https://httpbin.org/ip") # Automatically retries with different proxies
# Method 3: Direct proxy injection
proxies = LitProxy.get_proxy_dict()
response = requests.get("https://httpbin.org/ip", proxies=proxies)
HTTPX Client
import httpx
from litproxy import LitProxy
# Sync client with proxy support
client = LitProxy.get_proxied_httpx_client()
response = client.get("https://httpbin.org/ip")
# Patch existing httpx client
client = httpx.Client()
LitProxy.patch(client) # Adds proxy support
response = client.get("https://httpbin.org/ip")
# Async client
async with LitProxy.get_proxied_httpx_client() as client:
response = await client.get("https://httpbin.org/ip")
Curl_cffi Sessions
from litproxy import LitProxy
# Sync session with browser impersonation
session = LitProxy.get_proxied_curl_session(impersonate="chrome120")
response = session.get("https://httpbin.org/ip")
# Async session
async_session = LitProxy.get_proxied_curl_async_session(impersonate="safari15_5")
📊 Diagnostics & Monitoring
Proxy Statistics
from litproxy import LitProxy
stats = LitProxy.get_proxy_stats()
print(f"""
Proxy Statistics:
- Available Proxies: {stats['proxy_count']}
- Last Updated: {stats['last_updated']}
- Cache Duration: {stats['cache_duration']} seconds
- Cache Age: {stats['cache_age_seconds']} seconds
- Source URL: {stats['source_url']}
""")
Health Monitoring
from litproxy import LitProxy
# Test current proxy
current = LitProxy.current_proxy()
if current:
is_healthy = LitProxy.test_proxy(current)
print(f"Current proxy {current} is {'healthy' if is_healthy else 'unhealthy'}")
# Comprehensive health check
health_report = LitProxy.test_all_proxies()
healthy_count = sum(1 for status in health_report.values() if status)
total_count = len(health_report)
print(f"Healthy proxies: {healthy_count}/{total_count}")
🤝 Contributing
We welcome contributions to LitProxy! Here's how you can help:
Development Setup
# Clone the repository
git clone https://github.com/OEvortex/Webscout.git
cd Webscout/Litproxy
# Install in development mode
pip install -e .
# Install development dependencies
pip install pytest httpx curl_cffi
Running Tests
# Run basic tests
python -m pytest
# Test with different HTTP clients
python -c "from litproxy import LitProxy; print('✓ Basic import works')"
Areas for Contribution
- 🐛 Bug Reports: Found an issue? Please report it!
- 🚀 Feature Requests: Have ideas for improvements?
- 📖 Documentation: Help improve our docs and examples
- 🧪 Testing: Add test cases for better coverage
- 🔌 Integrations: Support for additional HTTP clients
📄 License
This project is licensed under the MIT License - see the LICENSE file for details.
Made with ❤️ by the Webscout Team
Metadata
Release files for litproxy 0.2.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| litproxy-0.2.1.tar.gz | 7.8 kB | Without provenance | Details for litproxy-0.2.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| litproxy-0.2.1-py3-none-any.whl | Python 3 | none | any | Details for litproxy-0.2.1-py3-none-any.whl |
Total release size: 14.6 kB
Release files / litproxy-0.2.1.tar.gz
| Download URL | litproxy-0.2.1.tar.gz |
|---|---|
| Size | 7.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
ea836079c75a209aebcd6c6d04423706b9ff884c54e67f49ab935b0a2dc1f171
|
|
BLAKE2b-256 checksum How to use checksums |
381ebf6b6c44439a4193b55cba808f5c5ee8f52180b9470f6c3f4f8caabc2117
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.5
|
Release files / litproxy-0.2.1-py3-none-any.whl
| Download URL | litproxy-0.2.1-py3-none-any.whl |
|---|---|
| Size | 6.9 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
8a235953e30bd951f74e53a83cce9517b0d8f350f2868177f4f836a25d449afc
|
|
BLAKE2b-256 checksum How to use checksums |
0275e6536d266deaf6ec0de45dfeb9b64c33939652c70968d0a5b137cb92acd0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.8.5
|
Security for litproxy 0.2.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.