proxy-fleet 2.1.0

A production-ready proxy server with intelligent load balancing and health monitoring

proxy-fleet 🚢

A production-ready Python proxy server and pool manager with intelligent load balancing, health monitoring, and enterprise-grade features. Built for high-performance proxy rotation similar to HAProxy but specifically designed for proxy management.

✨ Features

Core Proxy Management

• 🔄 Intelligent proxy rotation - Multiple load balancing strategies for optimal performance
• ⚡ High-performance architecture - Multi-process workers for handling thousands of concurrent requests
• 🏥 Advanced health monitoring - Circuit breakers, health checks, and automatic failover
• 📊 Real-time statistics - Live metrics, performance tracking, and monitoring endpoints
• 💾 Persistent state management - JSON-based proxy storage with automatic persistence
• 🛠️ Flexible configuration - JSON configuration with hot-reload support
• 📝 Comprehensive logging - Detailed request/response tracking with proxy attribution

Enhanced Proxy Server Features

• 🏭 Enterprise-grade proxy server - Production-ready HTTP proxy server with advanced features
• ⚖️ Multiple load balancing strategies - Round-robin, random, least-connections, weighted, response-time, and fail-over
• 🔄 Multi-process architecture - Scale across multiple CPU cores for maximum concurrency
• 🏥 Circuit breaker pattern - Automatic proxy isolation and recovery
• 📈 Performance monitoring - Built-in /stats and /health endpoints
• 🛑 Graceful shutdown - Intelligent handling of active connections during termination
• 💪 HAProxy-like capabilities - Enterprise load balancer features for proxy pools

Proxy Validation & Discovery

• 🧪 Fast SOCKS validation - Raw socket validation inspired by TheSpeedX/socker
• 📥 Automatic proxy discovery - Download and validate proxies from TheSpeedX/PROXY-List
• 🔒 Authentication support - Handle username/password proxy authentication
• 🚫 Automatic blacklisting - Remove unreliable proxies after consecutive failures
• ⚡ Concurrent validation - Validate multiple proxies simultaneously for speed

🌐 Proxy Protocols Overview

proxy-fleet supports multiple proxy protocols, each with distinct characteristics and use cases. Understanding these differences helps you choose the right proxy type for your specific requirements.

Protocol Comparison

Feature	HTTP Proxy	SOCKS4	SOCKS5
Protocol Layer	Application Layer (Layer 7)	Session Layer (Layer 5)	Session Layer (Layer 5)
Supported Protocols	HTTP/HTTPS only	TCP connections	TCP/UDP connections
Authentication	Basic authentication	No authentication	Multiple auth methods
IPv6 Support	Yes	No	Yes
DNS Resolution	Client or proxy side	Client side only	Can be done on proxy side
Connection Speed	Slower (HTTP parsing overhead)	Faster	Fast
Security	Can inspect/modify content	Basic forwarding	More secure, encrypted auth
Complexity	Simple	Simple	Medium

Protocol Details

HTTP Proxy

• Best for: Web browsing, API requests, HTTP-based applications
• Advantages: Content filtering, caching, easy debugging
• Disadvantages: Limited to HTTP/HTTPS protocols
• Security: Can inspect and modify HTTP traffic

SOCKS4

• Best for: Simple TCP applications, legacy systems
• Advantages: Lightweight, fast, universal TCP support
• Disadvantages: No authentication, IPv4 only, no UDP support
• Security: Basic TCP tunneling without inspection

SOCKS5

• Best for: Modern applications, gaming, VoIP, comprehensive proxy needs
• Advantages: Full protocol support, authentication, IPv6, remote DNS
• Disadvantages: Slightly more complex setup
• Security: Support for various authentication methods

proxy-fleet Support

proxy-fleet's enhanced proxy server provides:

• ✅ HTTP Proxy Server: Full HTTP/HTTPS proxy functionality with intelligent load balancing
• ✅ SOCKS4/5 Client Support: Can connect through SOCKS4 and SOCKS5 upstream proxies
• ✅ Protocol Detection: Automatic detection and validation of different proxy types
• ✅ Mixed Pool Management: Handle HTTP, SOCKS4, and SOCKS5 proxies in the same pool

Current Implementation:

• Self-hosted proxy server: Operates as an HTTP proxy server
• Upstream proxy support: Can route through HTTP, SOCKS4, and SOCKS5 upstream proxies
• Protocol validation: Validates all three proxy types during proxy discovery

Usage Examples with curl

Testing proxy-fleet's HTTP Proxy Server

# Start proxy-fleet server
proxy-fleet --enhanced-proxy-server --proxy-server-port 8888

# Use proxy-fleet as HTTP proxy
curl --proxy http://127.0.0.1:8888 http://httpbin.org/ip
curl -x http://127.0.0.1:8888 https://ipinfo.io/json

# With verbose output
curl -v --proxy http://127.0.0.1:8888 http://httpbin.org/get

Testing Different Upstream Proxy Types

HTTP Proxy:

# Basic usage
curl --proxy http://proxy-server:port http://example.com

# With authentication
curl --proxy http://username:password@proxy-server:port http://example.com

# Alternative syntax
curl -x http://proxy-server:port http://example.com

SOCKS4 Proxy:

# Basic usage
curl --socks4 proxy-server:port http://example.com

# With user specification (rarely needed)
curl --socks4 username@proxy-server:port http://example.com

SOCKS5 Proxy:

# Basic usage
curl --socks5 proxy-server:port http://example.com

# With authentication
curl --socks5 username:password@proxy-server:port http://example.com

# Force hostname resolution through proxy
curl --socks5-hostname proxy-server:port http://example.com

Advanced curl Options:

# Exclude specific domains from proxy
curl --proxy http://proxy:port --noproxy localhost,127.0.0.1 http://example.com

# Show detailed connection information
curl -v --proxy http://proxy:port http://example.com

# Set proxy timeout
curl --proxy http://proxy:port --connect-timeout 30 http://example.com

Choosing the Right Protocol

Use HTTP Proxy when:

• You need content filtering or caching
• Working primarily with web applications
• Debugging HTTP traffic is important
• You need application-layer features

Use SOCKS4 when:

• You need simple TCP tunneling
• Working with legacy applications
• IPv4 is sufficient for your needs
• Minimal overhead is important

Use SOCKS5 when:

• You need comprehensive protocol support
• Working with modern applications
• IPv6 support is required
• Authentication is necessary
• You need UDP support (gaming, VoIP)

proxy-fleet intelligently handles all these proxy types, providing a unified interface for managing diverse proxy infrastructure while maintaining optimal performance for each protocol type.

� Proxy Protocols Deep Dive

Understanding the differences between proxy protocols helps you make informed decisions for your specific use cases. Here's a comprehensive comparison:

Protocol Characteristics Comparison

Aspect	HTTP Proxy	SOCKS4	SOCKS5
Protocol Layer	Application Layer (Layer 7)	Session Layer (Layer 5)	Session Layer (Layer 5)
Supported Traffic	HTTP/HTTPS only	TCP connections	TCP/UDP connections
Authentication	Basic/Digest auth	No authentication	Multiple auth methods
IPv6 Support	Yes	No (IPv4 only)	Yes
DNS Resolution	Client or proxy side	Client side only	Proxy side (remote DNS)
Performance	Slower (HTTP overhead)	Fast	Fast
Security Level	Can inspect content	Basic tunneling	Encrypted auth options
Setup Complexity	Simple	Simple	Medium
Firewall Traversal	Excellent	Good	Good

When to Use Each Protocol

HTTP Proxy 🌐

Best for:

• Web scraping and crawling
• API requests and HTTP-based applications
• Content filtering and caching scenarios
• Debugging HTTP traffic

Advantages:

• Easy to implement and debug
• Content filtering capabilities
• Caching support
• Works well with web browsers

Disadvantages:

• Limited to HTTP/HTTPS protocols
• Higher overhead due to HTTP parsing
• Can inspect and modify your traffic

Real-world examples:

• Corporate web filtering
• Web scraping services
• CDN and caching proxies

SOCKS4 ⚡

Best for:

• Legacy applications
• Simple TCP applications
• Scenarios where speed is critical
• IPv4-only environments

Advantages:

• Lightweight and fast
• Universal TCP support
• Low overhead
• Simple protocol

Disadvantages:

• No authentication support
• IPv4 only
• No UDP support
• Client-side DNS resolution only

Real-world examples:

• Gaming applications (TCP-based)
• Legacy enterprise software
• Simple TCP tunneling

SOCKS5 🚀

Best for:

• Modern applications requiring full protocol support
• Gaming and real-time applications (UDP support)
• VoIP and video streaming
• Security-conscious applications

Advantages:

• Full protocol support (TCP/UDP)
• Multiple authentication methods
• IPv6 support
• Remote DNS resolution (privacy benefit)
• Most versatile option

Disadvantages:

• Slightly more complex setup
• Higher overhead than SOCKS4
• May require authentication configuration

Real-world examples:

• VPN-like applications
• Gaming and P2P software
• Modern secure applications
• Torrent clients

Technical Implementation Notes

proxy-fleet's Protocol Handling

As a Server (What proxy-fleet provides):

• ✅ HTTP Proxy Server: proxy-fleet runs as an HTTP proxy server
• ✅ Intelligent Load Balancing: Distributes requests across upstream proxies
• ✅ Multi-protocol Upstream Support: Can route through HTTP, SOCKS4, or SOCKS5 upstream proxies

As a Client (What proxy-fleet connects to):

• ✅ HTTP Upstream: Connect through HTTP proxies in your pool
• ✅ SOCKS4 Upstream: Connect through SOCKS4 proxies in your pool
• ✅ SOCKS5 Upstream: Connect through SOCKS5 proxies in your pool
• ✅ Mixed Pools: Handle all three types in the same proxy pool

Example Architecture:

Your App → proxy-fleet (HTTP Proxy) → Upstream Proxy Pool
                                      ├── HTTP Proxy 1
                                      ├── SOCKS4 Proxy 2  
                                      ├── SOCKS5 Proxy 3
                                      └── HTTP Proxy 4

Performance Considerations

Latency Ranking (Fastest to Slowest)

1. SOCKS4 - Minimal overhead, no authentication
2. SOCKS5 - Low overhead, optional authentication
3. HTTP Proxy - Higher overhead due to HTTP parsing

Feature Ranking (Most to Least Features)

1. SOCKS5 - Full protocol support, authentication, IPv6
2. HTTP Proxy - Content inspection, caching, filtering
3. SOCKS4 - Basic TCP tunneling only

Security Ranking (Most to Least Secure)

1. SOCKS5 - Encrypted authentication, remote DNS
2. HTTP Proxy - Can inspect traffic (pro/con depending on use case)
3. SOCKS4 - No authentication, basic tunneling

Choosing the Right Protocol for Your Use Case

For Web Scraping:

• HTTP Proxy - Best choice for HTTP-only traffic
• SOCKS5 - If you need additional protocols or privacy

For Gaming:

• SOCKS5 - UDP support for real-time games
• SOCKS4 - For TCP-only games where speed matters

For Privacy/Security:

• SOCKS5 - Remote DNS resolution prevents DNS leaks
• HTTP Proxy - Avoid if you don't want traffic inspection

For Enterprise:

• HTTP Proxy - Content filtering and monitoring
• SOCKS5 - Secure authenticated access

For High Performance:

• SOCKS4 - Minimal overhead for TCP traffic
• SOCKS5 - Good balance of features and performance

�🚀 Quick Start

Installation

pip install proxy-fleet

Enhanced Proxy Server (Recommended)

The enhanced proxy server is the recommended way to use proxy-fleet in production:

# 1. Validate and store some proxies
curl -sL 'https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks5.txt' | proxy-fleet --test-proxy-server - --concurrent 50

# 2. Generate default configuration
proxy-fleet --generate-config

# 3. Start the enhanced proxy server
proxy-fleet --enhanced-proxy-server --proxy-server-port 8989

# 4. Test the server
curl --proxy http://127.0.0.1:8989 http://httpbin.org/ip

Key Benefits:

• High Performance: Multi-process architecture for maximum throughput
• Intelligent Load Balancing: Multiple strategies for optimal proxy utilization
• Automatic Failover: Circuit breakers and health checks ensure reliability
• Production Ready: Comprehensive logging, monitoring, and graceful shutdown
• Easy Configuration: JSON-based config with sensible defaults

📖 Usage Guide

Command Line Interface

proxy-fleet provides comprehensive proxy management through its CLI:

Proxy Validation

# Validate proxies from file
proxy-fleet --test-proxy-server proxies.txt

# Validate from stdin with high concurrency
curl -sL 'https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks5.txt' | \
  proxy-fleet --test-proxy-server - --concurrent 100 --test-proxy-timeout 10 --test-proxy-type socks5

curl -sL 'https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/socks4.txt' | \
  proxy-fleet --test-proxy-server - --concurrent 100 --test-proxy-timeout 10 --test-proxy-type socks4

curl -sL 'https://raw.githubusercontent.com/TheSpeedX/SOCKS-List/master/http.txt' | \
  proxy-fleet --test-proxy-server - --concurrent 100 --test-proxy-timeout 10 --test-proxy-type http

# Test with HTTP request validation
proxy-fleet --test-proxy-server proxies.txt --test-proxy-with-request 'https://httpbin.org/ip'

# Test with custom API that returns location info
# The tool will automatically extract 'country' field, or fallback to 'region' field
# Only proxies returning 2XX or 3XX status codes are considered valid
proxy-fleet --test-proxy-server proxies.txt --test-proxy-with-request 'https://myserver.com/api/location'

# Test existing proxies in storage
proxy-fleet --test-proxy-storage

Proxy Management

# List all proxy status
proxy-fleet --list-proxy

# List only verified/valid proxies
proxy-fleet --list-proxy-verified

# List only failed/invalid proxies
proxy-fleet --list-proxy-failed

# Remove failed proxies from storage
proxy-fleet --remove-proxy-failed

Basic HTTP Proxy Server

# Start basic proxy server with round-robin rotation
proxy-fleet --start-proxy-server --proxy-server-port 8888

# Start with random rotation
proxy-fleet --start-proxy-server --proxy-server-rotation random

Enhanced HTTP Proxy Server (Production)

# Generate configuration file
proxy-fleet --generate-config

# Start enhanced server with default settings
proxy-fleet --enhanced-proxy-server

# Start with specific strategy and multiple workers
proxy-fleet --enhanced-proxy-server \
  --proxy-server-strategy least_connections \
  --proxy-server-workers 8 \
  --proxy-server-host 0.0.0.0

# Start with custom configuration file
proxy-fleet --enhanced-proxy-server --proxy-server-config my_config.json

# Development mode (single process)
proxy-fleet --enhanced-proxy-server --single-process

Load Balancing Strategies

1. Least Connections (Recommended for production)

proxy-fleet --enhanced-proxy-server --proxy-server-strategy least_connections

Routes requests to the proxy with the fewest active connections.

2. Response Time Based

proxy-fleet --enhanced-proxy-server --proxy-server-strategy response_time

Routes requests to the proxy with the best average response time.

3. Round Robin

proxy-fleet --enhanced-proxy-server --proxy-server-strategy round_robin

Distributes requests evenly across all available proxies.

4. Random

proxy-fleet --enhanced-proxy-server --proxy-server-strategy random

Randomly selects an available proxy for each request.

5. Weighted Round Robin

Configure proxy weights in the configuration file:

{
  "load_balancing": {
    "strategy": "weighted",
    "strategies": {
      "weighted": {
        "proxy_weights": {
          "fast-proxy.com:1080": 3.0,
          "medium-proxy.com:1080": 2.0,
          "slow-proxy.com:1080": 1.0
        }
      }
    }
  }
}

6. Fail Over

Configure primary and backup proxies:

{
  "load_balancing": {
    "strategy": "fail_over",
    "strategies": {
      "fail_over": {
        "primary_proxies": ["primary1.com:1080", "primary2.com:1080"],
        "backup_proxies": ["backup1.com:1080", "backup2.com:1080"]
      }
    }
  }
}

High Concurrency Setup

For high-traffic production environments:

proxy-fleet --enhanced-proxy-server \
  --proxy-server-workers 8 \
  --proxy-server-strategy least_connections \
  --proxy-server-host 0.0.0.0 \
  --proxy-server-port 8888

Dynamic Proxy Pool Management

proxy-fleet supports hot-reloading of proxy pools without server restart using API endpoints:

API-Based Refresh (Recommended) ⭐

The most efficient and controlled way to update proxy pools:

# Force refresh proxy pool from storage (no external requests)
curl http://127.0.0.1:8888/refresh

# Force refresh with immediate health check (minimal external requests)
curl "http://127.0.0.1:8888/refresh?health_check=true"

# Check current status
curl http://127.0.0.1:8888/stats | jq .rotator_stats

Key Benefits:

• ✅ No external service pressure - Only reloads from local storage
• ✅ Instant updates - Changes take effect immediately
• ✅ Zero downtime - Server continues serving requests
• ✅ Full control - Trigger refreshes only when needed

Manual Refresh Workflow (Recommended)

For controlled proxy validation and hot-reload without overwhelming third-party services:

# 1. Start the enhanced proxy server
proxy-fleet --enhanced-proxy-server --proxy-server-port 8888

# 2. When needed, validate proxy health in a separate terminal
# This updates the proxy storage with fresh health data
proxy-fleet --test-proxy-storage --test-proxy-with-request 'https://ipinfo.io/json'

# 3. Force the running server to reload from updated storage
curl "http://127.0.0.1:8888/refresh?health_check=true"

# 4. Verify the refresh worked
curl http://127.0.0.1:8888/stats | jq .rotator_stats

Conservative Automatic Health Checks

For minimal automated monitoring without overloading external services:

{
  "health_checks": {
    "enabled": true,
    "interval": 86400,
    "timeout": 15,
    "max_failures": 5,
    "test_url": "http://httpbin.org/ip"
  }
}

Note:

• Default health checks run every 24 hours to minimize external service load
• Protocol-Specific Validation: Health checks automatically use the appropriate validation method:
  • SOCKS4/SOCKS5 proxies: Use raw socket handshake validation (fast and reliable)
  • HTTP proxies: Use HTTP request validation with the configured test_url
• Use the /refresh API for immediate updates when needed For scheduled proxy validation and hot-reload:

# 1. Start the enhanced proxy server
proxy-fleet --enhanced-proxy-server --proxy-server-port 8888

# 2. In a separate terminal, validate proxy health (every 12 hours)
# This updates the proxy storage with fresh health data
proxy-fleet --test-proxy-storage --test-proxy-with-request 'https://ipinfo.io/json'

# 3. Force the running server to reload from updated storage
curl "http://127.0.0.1:8888/refresh?health_check=true"

# 4. Verify the refresh worked
curl http://127.0.0.1:8888/stats | jq .rotator_stats

Automated Refresh Script

Create a controlled refresh script for periodic validation:

#!/bin/bash
# refresh-proxies.sh - Run this manually or via cron when needed

echo "Starting proxy refresh at $(date)"

# Re-validate all proxies in storage with rate limiting
proxy-fleet --test-proxy-storage \
  --test-proxy-with-request 'https://httpbin.org/ip' \
  --concurrent 10 \
  --test-proxy-timeout 15

# Tell running server to reload the proxy pool
result=$(curl -s "http://127.0.0.1:8888/refresh?health_check=false")
echo "Refresh result: $result"

echo "Proxy pool refreshed at $(date)"

Best Practices for External Services:

• Use httpbin.org/ip instead of ipinfo.io for basic connectivity tests
• Limit concurrent validation (--concurrent 10 instead of 50+)
• Increase timeout values to reduce retry pressure
• Consider running validation only when actually needed, not on a rigid schedule
• Use the /refresh API endpoint to reload without external requests

Recommended Production Workflow

1. Startup: Load proxies and start server with minimal health checks
2. Operation: Use API endpoints for real-time monitoring and control
3. Maintenance: Manually validate proxies when proxy pool needs refreshing
4. Update: Use /refresh API to hot-reload updated proxy data

Monitoring & Statistics

# Get real-time statistics
curl http://127.0.0.1:8888/stats | jq .

# Health check endpoint
curl http://127.0.0.1:8888/health

# Force refresh proxy pool from storage (without restarting server)
curl http://127.0.0.1:8888/refresh

# Force refresh with immediate health check
curl "http://127.0.0.1:8888/refresh?health_check=true"

# Monitor proxy performance
watch -n 1 'curl -s http://127.0.0.1:8888/stats | jq .rotator_stats.proxy_details'

Example Statistics Output

{
  "requests_total": 1000,
  "requests_success": 950,
  "requests_failed": 50,
  "uptime_seconds": 3600,
  "rotator_stats": {
    "strategy": "least_connections",
    "total_proxies": 10,
    "healthy_proxies": 8,
    "proxy_details": {
      "proxy1.com:1080": {
        "active_connections": 5,
        "total_requests": 120,
        "success_rate": 0.95,
        "avg_response_time": 0.8,
        "is_healthy": true
      }
    }
  },
  "worker_stats": {
    "total_workers": 4,
    "active_workers": 4
  }
}

🔧 Configuration

Generate Default Configuration

proxy-fleet --generate-config

This creates a proxy_server_config.json file with comprehensive default settings.

For production environments with conservative health checking, see proxy_server_config_production.json which includes:

• Very long health check intervals (24 hours vs frequent checks)
• More reliable test URLs (httpbin.org vs third-party services)
• Reduced concurrent checks to minimize external service load

Configuration Structure

{
  "proxy_server": {
    "host": "127.0.0.1",
    "port": 8888,
    "workers": 4,
    "graceful_shutdown_timeout": 30,
    "access_log": true
  },
  "load_balancing": {
    "strategy": "least_connections",
    "strategies": {
      "weighted": {
        "proxy_weights": {}
      },
      "fail_over": {
        "primary_proxies": [],
        "backup_proxies": []
      }
    }
  },
  "health_checks": {
    "enabled": true,
    "interval": 86400,
    "timeout": 15,
    "max_failures": 5,
    "parallel_checks": 5,
    "test_url": "http://httpbin.org/ip"
  },
  "circuit_breaker": {
    "enabled": true,
    "failure_threshold": 5,
    "recovery_timeout": 300,
    "half_open_max_calls": 3
  },
  "logging": {
    "level": "INFO",
    "format": "detailed",
    "file": null
  }
}

Configuration Options

Proxy Server Settings

• host: Server bind address (default: 127.0.0.1)
• port: Server port (default: 8888)
• workers: Number of worker processes (default: CPU count)
• graceful_shutdown_timeout: Graceful shutdown timeout in seconds
• access_log: Enable access logging

Load Balancing

• strategy: Load balancing strategy (least_connections, round_robin, random, weighted, response_time, fail_over)
• strategies: Strategy-specific configurations

Health Checks

• enabled: Enable automatic health checking
• interval: Health check interval in seconds (default: 86400 = 24 hours)
• timeout: Health check timeout
• max_failures: Maximum failures before marking proxy unhealthy
• parallel_checks: Number of parallel health checks
• test_url: URL for health checks (use httpbin.org/ip for basic tests)

Circuit Breaker

• enabled: Enable circuit breaker pattern
• failure_threshold: Failures before opening circuit
• recovery_timeout: Time before attempting recovery
• half_open_max_calls: Max calls in half-open state

📚 Python API

Basic Usage

from proxy_fleet.cli.main import ProxyStorage
from proxy_fleet.server.enhanced_proxy_server import EnhancedHTTPProxyServer

# Initialize proxy storage
storage = ProxyStorage("./proxy_data")

# Add some proxies
storage.update_proxy_status("proxy1.com", 1080, True)
storage.update_proxy_status("proxy2.com", 1080, True)

# Start enhanced proxy server
config_file = "proxy_server_config.json"
server = EnhancedHTTPProxyServer(config_file)
await server.start()

SOCKS Validation

from proxy_fleet.utils.socks_validator import SocksValidator

# Create validator with HTTP request validation
validator = SocksValidator(
    timeout=10, 
    check_server_via_request=True,
    request_url="https://httpbin.org/ip"
)

# Create validator without HTTP request validation (basic validation only)
validator = SocksValidator(timeout=10)

# Validate a proxy
result = validator.validate_socks5("proxy.example.com", 1080)
print(f"Valid: {result.is_valid}, IP: {result.ip_info}")

# Async validation
result = await validator.async_validate_socks5("proxy.example.com", 1080)

Proxy Rotation

from proxy_fleet.server.enhanced_proxy_server import EnhancedProxyRotator

# Create rotator with configuration
config = {
    "load_balancing": {"strategy": "least_connections"},
    "health_checks": {"enabled": True, "interval": 60}
}

rotator = EnhancedProxyRotator("./proxy_data", config)

# Get next proxy
proxy_info, stats = await rotator.get_next_proxy()
print(f"Using proxy: {proxy_info['host']}:{proxy_info['port']}")

# Record request result
await rotator.record_request_result(
    proxy_info['host'], 
    proxy_info['port'], 
    success=True, 
    response_time=0.5
)

🗂️ Project Structure

proxy-fleet/
├── proxy_fleet/
│   ├── cli/                 # Command-line interface
│   │   └── main.py         # CLI implementation and ProxyStorage
│   ├── server/             # Proxy server implementations
│   │   ├── enhanced_proxy_server.py  # Enhanced server with load balancing
│   │   └── proxy_server.py           # Basic proxy server
│   ├── utils/              # Utility modules
│   │   ├── socks_validator.py        # SOCKS proxy validation
│   │   ├── proxy_utils.py           # Proxy utility functions
│   │   └── output.py               # Output formatting
│   └── models/             # Data models
│       ├── proxy.py        # Proxy data models
│       ├── config.py       # Configuration models
│       └── task.py         # Task models
├── tests/                  # Test suite
│   ├── test_main.py                 # CLI and storage tests
│   ├── test_integration.py          # Integration tests
│   ├── test_proxy_functionality.py  # Core functionality tests
│   └── test_socks_validation.py     # SOCKS validation tests
├── examples/               # Usage examples
├── proxy/                  # Default proxy storage directory
└── proxy_server_config.json        # Default configuration file

🏗️ Architecture

Enhanced Proxy Server Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Load Balancer                            │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐           │
│  │   Worker 1  │ │   Worker 2  │ │   Worker N  │           │
│  └─────────────┘ └─────────────┘ └─────────────┘           │
└─────────────────────────────────────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────────┐
│                 Proxy Rotator                               │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐           │
│  │   Strategy  │ │ Health Check│ │Circuit Break│           │
│  │   Manager   │ │   Manager   │ │   Manager   │           │
│  └─────────────┘ └─────────────┘ └─────────────┘           │
└─────────────────────────────────────────────────────────────┘
                           │
                           ▼
┌─────────────────────────────────────────────────────────────┐
│                  Proxy Pool                                 │
│  ┌─────────────┐ ┌─────────────┐ ┌─────────────┐           │
│  │   Proxy 1   │ │   Proxy 2   │ │   Proxy N   │           │
│  │   + Stats   │ │   + Stats   │ │   + Stats   │           │
│  └─────────────┘ └─────────────┘ └─────────────┘           │
└─────────────────────────────────────────────────────────────┘

Key Components

1. Load Balancer: Distributes incoming requests across worker processes
2. Worker Processes: Handle HTTP proxy requests independently
3. Proxy Rotator: Manages proxy selection and load balancing strategies
4. Health Check Manager: Monitors proxy health and availability
5. Circuit Breaker: Provides automatic failover and recovery
6. Statistics Manager: Tracks performance metrics and proxy statistics

🚀 Performance

Benchmarks

The enhanced proxy server is designed for high performance:

• Throughput: 10,000+ requests/second on modern hardware
• Concurrency: Handles 1,000+ simultaneous connections
• Latency: <1ms overhead for proxy selection
• Memory: Efficient memory usage with connection pooling
• CPU: Scales linearly with worker processes

Optimization Tips

1. Worker Count: Set workers to 2x CPU cores for I/O bound workloads
2. Strategy Selection: Use least_connections for balanced load distribution
3. Health Checks: Tune health check intervals based on proxy stability
4. Circuit Breaker: Configure thresholds based on acceptable failure rates
5. Connection Pooling: Enable keep-alive for better performance

🔒 Security

Security Features

• Input Validation: Comprehensive validation of proxy configurations
• Connection Limits: Configurable limits on concurrent connections
• Access Control: Host-based access controls (configurable)
• Secure Defaults: Conservative default configurations
• Error Handling: Robust error handling prevents information leakage

Best Practices

1. Bind Address: Use 127.0.0.1 for local-only access
2. Firewall: Configure firewall rules for production deployment
3. Monitoring: Monitor access logs for suspicious activity
4. Updates: Keep proxy-fleet updated to latest version
5. Proxy Validation: Regularly validate proxy server credentials

📊 Monitoring & Observability

Built-in Endpoints

• GET /stats - Real-time statistics and metrics
• GET /health - Health check endpoint for load balancers
• GET /refresh - Force refresh proxy pool from storage (hot-reload)
• GET /refresh?health_check=true - Refresh proxy pool and perform immediate health check

Logging

proxy-fleet provides comprehensive logging:

# Configure logging level
{
  "logging": {
    "level": "INFO",
    "format": "detailed",
    "file": "/var/log/proxy-fleet.log"
  }
}

Metrics Integration

Easily integrate with monitoring systems:

# Prometheus-style metrics
curl http://127.0.0.1:8888/stats | jq .

# Custom monitoring
curl -s http://127.0.0.1:8888/stats | python my_monitor.py

🤝 Contributing

We welcome contributions! Please see our Contributing Guide for details.

Development Setup

# Clone the repository
git clone https://github.com/your-org/proxy-fleet.git
cd proxy-fleet

# Install development dependencies
python dev.py install

# Run tests
python dev.py test

# Run with coverage
python dev.py test-cov

# Format code
python dev.py format

# Lint code
python dev.py lint

Testing

The project includes comprehensive test coverage:

• Unit Tests: Core functionality and components
• Integration Tests: End-to-end workflow testing
• Performance Tests: Load and stress testing
• SOCKS Validation Tests: Protocol-specific validation

# Run all tests
python dev.py test

# Run specific test categories
pytest tests/test_proxy_functionality.py -v
pytest tests/test_integration.py -v

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• TheSpeedX/socker - Inspiration for SOCKS validation
• TheSpeedX/PROXY-List - Proxy list resources
• aiohttp - Async HTTP framework
• aiohttp-socks - SOCKS proxy connector

📞 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: Wiki

---

Built with ❤️ for the proxy management community.