Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

Lightweight, Programmable, TLS interceptor Proxy for HTTP(S), HTTP2, WebSockets protocols in a single Python file.

Project description


License PyPi Downloads Docker Pulls Build Status No Dependencies Coverage

Tested With MacOS Tested With Ubuntu Tested With Windows Tested With Android Tested With Android Emulator Tested With iOS Tested With iOS Simulator

Maintenance Ask Me Anything Contributions Welcome Gitter

Python 3.6 Python 3.7 Checked with mypy

Become a Backer

Table of Contents


  • Lightweight
    • Distributed as a single file module ~50KB
    • Uses only ~5-20MB RAM
    • No external dependency other than standard Python library
  • Programmable
    • Optionally enable builtin Web Server
    • Customize proxy and http routing via plugins
    • Enable plugin using command line option e.g. --plugins plugin_examples.CacheResponsesPlugin
    • Plugin API is currently in development state, expect breaking changes.
  • Secure
  • Man-In-The-Middle
    • Can decrypt TLS traffic between clients and upstream servers
    • See TLS Encryption
  • Supported proxy protocols
    • http
    • https
    • http2
    • websockets
  • Optimized for large file uploads and downloads
  • IPv4 and IPv6 support
  • Basic authentication support
  • Can serve a PAC (Proxy Auto-configuration) file
    • See --pac-file and --pac-file-url-path flags


Stable version

Install from PyPi

$ pip install --upgrade

or from GitHub master branch

$ pip install git+

or download from here or simply wget it:

$ wget -q

Development version

$ pip install git+

For Docker usage see Docker Image.


Command line

Simply type on command line to start it with default configuration.

...[redacted]... - Loaded plugin <class 'proxy.HttpProxyPlugin'>
...[redacted]... - Starting 8 workers
...[redacted]... - Started server on ::1:8899

Things to notice from above logs:

  • Loaded plugin - will load HttpProxyPlugin by default. It adds http(s) proxy server capabilities to

  • Started N workers - Use --num-workers flag to customize number of Worker processes. By default, will start as many workers as there are CPU cores on the machine.

  • Started server on ::1:8899 - By default, listens on IPv6 ::1, which is equivalent of IPv4 If you want to access externally, use --hostname :: or --hostname or bind to any other interface available on your machine.

  • Port 8899 - Use --port flag to customize default TCP port.

All the logs above are INFO level logs, default --log-level for

Lets start with DEBUG level logging:

$ --log-level d
...[redacted]... - Open file descriptor soft limit set to 1024
...[redacted]... - Loaded plugin <class 'proxy.HttpProxyPlugin'>
...[redacted]... - Started 8 workers
...[redacted]... - Started server on ::1:8899

As we can see, before starting up:

  • also tried to set open file limit ulimit on the system.
  • Default value for --open-file-limit used is 1024.
  • --open-file-limit flag is a no-op on Windows operating systems.

See flags for full list of available configuration options.

Docker image

$ docker run -it -p 8899:8899 --rm abhinavsingh/

By default docker binary is started with IPv4 networking flags:

--hostname --port 8899

To override input flags, start docker image as follows. For example, to check --version:

$ docker run -it \
    -p 8899:8899 \
    --rm abhinavsingh/ \

WARNING docker image is currently broken on macOS due to incompatibility with vpnkit.

Plugin Examples

See for full code.

All the examples below also works with https traffic but require additional flags and certificate generation. See TLS Interception.


Mock responses for your server REST API. Use to test and develop client side applications without need of an actual upstream REST API server.

Start as:

$ \
    --plugins plugin_examples.ProposedRestApiPlugin

Verify mock API response using curl -x localhost:8899

{"count": 2, "next": null, "previous": null, "results": [{"email": "", "groups": [], "url": "", "username": "admin"}, {"email": "", "groups": [], "url": "", "username": "admin"}]}

Verify the same by inspecting logs:

2019-09-27 12:44:02,212 - INFO - pid:7077 - access_log:1210 - ::1:64792 - GET None:None/v1/users/ - None None - 0 byte

Access log shows None:None as server ip:port. None simply means that the server connection was never made, since response was returned by our plugin.

Now modify ProposedRestApiPlugin to returns REST API mock responses as expected by your clients.


Redirects all incoming http requests to custom web server. By default, it redirects client requests to inbuilt web server, also running on 8899 port.

Start and enable inbuilt web server:

$ \
    --enable-web-server \
    --plugins plugin_examples.RedirectToCustomServerPlugin

Verify using curl -v -x localhost:8899

... [redacted] ...
< HTTP/1.1 404 NOT FOUND
< Server: v1.0.0
< Connection: Close
* Closing connection 0

Above 404 response was returned from web server.

Verify the same by inspecting the logs for Along with the proxy request log, you must also see a http web server request log.

2019-09-24 19:09:33,602 - INFO - pid:49996 - access_log:1241 - ::1:49525 - GET /
2019-09-24 19:09:33,603 - INFO - pid:49995 - access_log:1157 - ::1:49524 - GET localhost:8899/ - 404 NOT FOUND - 70 bytes


Drops traffic by inspecting upstream host. By default, plugin drops traffic for and

Start as:

$ \
    --plugins plugin_examples.FilterByUpstreamHostPlugin

Verify using curl -v -x localhost:8899

... [redacted] ...
< HTTP/1.1 418 I'm a tea pot
< Proxy-agent: v1.0.0
* no chunk, no close, no size. Assume close to signal end
* Closing connection 0

Above 418 I'm a tea pot is sent by our plugin.

Verify the same by inspecting logs for

2019-09-24 19:21:37,893 - ERROR - pid:50074 - handle_readables:1347 - ProtocolException type raised
Traceback (most recent call last):
... [redacted] ...
2019-09-24 19:21:37,897 - INFO - pid:50074 - access_log:1157 - ::1:49911 - GET None:None/ - None None - 0 bytes


Caches Upstream Server Responses.

Start as:

$ \
    --plugins plugin_examples.CacheResponsesPlugin

Verify using curl -v -x localhost:8899

... [redacted] ...
< HTTP/1.1 200 OK
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Origin: *
< Content-Type: application/json
< Date: Wed, 25 Sep 2019 02:24:25 GMT
< Referrer-Policy: no-referrer-when-downgrade
< Server: nginx
< X-Content-Type-Options: nosniff
< X-Frame-Options: DENY
< X-XSS-Protection: 1; mode=block
< Content-Length: 202
< Connection: keep-alive
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "", 
    "User-Agent": "curl/7.54.0"
  "origin": ",", 
  "url": ""
* Connection #0 to host localhost left intact

Get path to the cache file from logs:

... [redacted] ... - GET - 200 OK - 556 bytes
... [redacted] ... - Cached response at /var/folders/k9/x93q0_xn1ls9zy76m2mf2k_00000gn/T/

Verify contents of the cache file cat /path/to/your/cache/

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Wed, 25 Sep 2019 02:24:25 GMT
Referrer-Policy: no-referrer-when-downgrade
Server: nginx
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Content-Length: 202
Connection: keep-alive

  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "", 
    "User-Agent": "curl/7.54.0"
  "origin": ",", 
  "url": ""


Modifies upstream server responses.

Start as:

$ \
    --plugins plugin_examples.ManInTheMiddlePlugin

Verify using curl -v -x localhost:8899

... [redacted] ...
< HTTP/1.1 200 OK
< Content-Length: 28
* Connection #0 to host localhost left intact
Hello from man in the middle

Response body Hello from man in the middle is sent by our plugin.

Plugin Ordering

When using multiple plugins, depending upon plugin functionality, it might be worth considering the order in which plugins are passed on the command line.

Plugins are called in the same order as they are passed. Example, say we are using both FilterByUpstreamHostPlugin and RedirectToCustomServerPlugin. Idea is to drop all incoming http requests for and and redirect other http requests to our inbuilt web server.

Hence, in this scenario it is important to use FilterByUpstreamHostPlugin before RedirectToCustomServerPlugin. If we enable RedirectToCustomServerPlugin before FilterByUpstreamHostPlugin, google requests will also get redirected to inbuilt web server, instead of being dropped.

End-to-End Encryption

By default, uses http protocol for communication with clients e.g. curl, browser. For enabling end-to-end encrypting using tls / https first generate certificates:

make https-certificates

Start as:

$ \
    --cert-file https-cert.pem \
    --key-file https-key.pem

Verify using curl -x https://localhost:8899 --proxy-cacert https-cert.pem

  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "", 
    "User-Agent": "curl/7.54.0"
  "origin": ",", 
  "url": ""

TLS Interception

By default, doesn't decrypt https traffic between client and server. To enable TLS interception first generate CA certificates:

make ca-certificates

Lets also enable CacheResponsePlugin so that we can verify decrypted response from the server. Start as:

$ \
    --plugins plugin_examples.CacheResponsesPlugin \
    --ca-key-file ca-key.pem \
    --ca-cert-file ca-cert.pem \
    --ca-signing-key-file ca-signing-key.pem

Verify using curl -v -x localhost:8899 --cacert ca-cert.pem

*  issuer: C=US; ST=CA; L=SanFrancisco;; OU=CA; CN=Proxy PY CA;
*  SSL certificate verify ok.
> GET /get HTTP/1.1
... [redacted] ...
< Connection: keep-alive
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "", 
    "User-Agent": "curl/7.54.0"
  "origin": ",", 
  "url": ""

The issuer line confirms that response was intercepted.

Also verify the contents of cached response file. Get path to the cache file from logs.

$ cat /path/to/your/tmp/directory/

HTTP/1.1 200 OK
Access-Control-Allow-Credentials: true
Access-Control-Allow-Origin: *
Content-Type: application/json
Date: Wed, 25 Sep 2019 23:07:05 GMT
Referrer-Policy: no-referrer-when-downgrade
Server: nginx
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Content-Length: 202
Connection: keep-alive

  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "", 
    "User-Agent": "curl/7.54.0"
  "origin": ",", 
  "url": ""

Viola!!! If you remove CA flags, encrypted data will be found in the cached file instead of plain text.

Now use CA flags with other plugin examples to see them work with https traffic.


You can directly import into your Python code. Example:

$ python
>>> import proxy


Attempts to create an IPv4 connection, then IPv6 and finally a dual stack connection to provided address.

>>> conn = proxy.new_socket_connection(('', 80))
>>> ...[ use connection ]...
>>> conn.close()


socket_connection is a convenient decorator + context manager around new_socket_connection which ensures conn.close is implicit.

As a context manager:

>>> with proxy.new_socket_connection(('', 80)) as conn:
>>>   ... [ use connection ] ...

As a decorator:

>>> @proxy.new_socket_connection(('', 80))
>>> def my_api_call(conn, *args, **kwargs):
>>>   ... [ use connection ] ...


Generate HTTP GET request

>>> proxy.build_http_request(b'GET', b'/')
b'GET / HTTP/1.1\r\n\r\n'

Generate HTTP GET request with headers

>>> proxy.build_http_request(b'GET', b'/', 
        headers={b'Connection': b'close'})
b'GET / HTTP/1.1\r\nConnection: close\r\n\r\n'

Generate HTTP POST request with headers and body

>>> import json
>>> proxy.build_http_request(b'POST', b'/form', 
        headers={b'Content-type': b'application/json'}, 
        body=proxy.bytes_(json.dumps({'email': ''})))
    b'POST /form HTTP/1.1\r\nContent-type: application/json\r\n\r\n{"email": ""}'

To start server from imported module, simply do:

import proxy

if __name__ == '__main__':
  proxy.main(['--hostname', '::1', '--port', 8899])

See Internal Documentation for all available classes and utility methods.

Plugin Developer and Contributor Guide

Everything is a plugin

As you might have guessed by now, in everything is a plugin.

  • We enabled proxy server plugins using --plugins flag. All the plugin examples were implementing HttpProxyBasePlugin. See documentation of HttpProxyBasePlugin for available lifecycle hooks. Use HttpProxyBasePlugin to modify behavior of http(s) proxy protocol between client and upstream server. Example, FilterByUpstreamHostPlugin.

  • We also enabled inbuilt web server using --enable-web-server. Inbuilt web server implements ProtocolHandlerPlugin plugin. See documentation of ProtocolHandlerPlugin for available lifecycle hooks. Use ProtocolHandlerPlugin to add new features for http(s) clients. Example, HttpWebServerPlugin.

  • There also is a --disable-http-proxy flag. It disables inbuilt proxy server. Use this flag with --enable-web-server flag to run as a programmable http(s) server. HttpProxyPlugin also implements ProtocolHandlerPlugin.

Internal Architecture

  • ProtocolHandler thread is started with the accepted TcpClientConnection. ProtocolHandler is responsible for parsing incoming client request and invoking ProtocolHandlerPlugin lifecycle hooks.

  • HttpProxyPlugin which implements ProtocolHandlerPlugin also has its own plugin mechanism. Its responsibility is to establish connection between client and upstream TcpServerConnection and invoke HttpProxyBasePlugin lifecycle hooks.

  • ProtocolHandler threads are started by Worker processes.

  • --num-workers Worker processes are started by AcceptorPool on start-up.

  • AcceptorPool listens on server socket and pass the handler to Worker processes. Workers are responsible for accepting new client connections and starting ProtocolHandler thread.

Sending a Pull Request

Install dependencies for local development testing:

$ pip install -r requirements-testing.txt

Every pull request goes through set of tests which must pass:

  • mypy: Run make lint locally for compliance check. Fix all warnings and errors before sending out a PR.

  • coverage: Run make coverage locally for coverage report. Its ideal to add tests for any critical change. Depending upon the change, it's ok if test coverage falls by <0.5%.

  • formatting: Run make autopep8 locally to format the code in-place. autopep8 is run with --aggresive flag. Sometimes it may result in weird formatting. But let's stick to one consistent formatting tool. I am open to flag changes for autopep8.

Internal Documentation

Browse through internal class hierarchy and documentation using pydoc3. Example:

$ pydoc3 proxy
Help on module proxy:


    Lightweight, Programmable, TLS interceptor Proxy for HTTP(S), HTTP2, WebSockets protocols in a single Python file.

    :copyright: (c) 2013-present by Abhinav Singh and contributors.
    :license: BSD, see LICENSE for more details.


Frequently Asked Questions

Unable to connect with from remote host

Make sure is listening on correct network interface. Try following flags:

  • For IPv6 --hostname ::
  • For IPv4 --hostname

Basic auth not working with a browser

Most likely it's a browser integration issue with system keychain.

  • First verify that basic auth is working using curl

    curl -v -x username:password@localhost:8899

  • See this thread for further details.

Docker image not working on macOS

It's a compatibility issue with vpnkit.

See moby/vpnkit exhausts docker resources and Connection refused: The proxy could not connect for some background.

Unable to load custom plugins

Make sure your plugin modules are discoverable by adding them to PYTHONPATH. Example:

PYTHONPATH=/path/to/my/app --plugins my_app.proxyPlugin

...[redacted]... - Loaded plugin proxy.HttpProxyPlugin
...[redacted]... - Loaded plugin my_app.proxyPlugin

GCE log viewer integration for

A starter fluentd.conf template is available.

  1. Copy this configuration file as under /etc/google-fluentd/config.d/

  2. Update path field to log file path as used with --log-file flag. By default /tmp/proxy.log path is tailed.

  3. Reload google-fluentd:

    sudo service google-fluentd restart

Now logs can be browsed using GCE log viewer.

ValueError: filedescriptor out of range in select is made to handle thousands of connections per second.

  1. Make use of --open-file-limit flag to customize ulimit -n.
    • To set a value upper than the hard limit, run as root.
  2. Make sure to adjust --backlog flag for higher concurrency.

If nothing helps, open an issue with requests per second sent and output of following debug script:

# PID of
PROXY_PY_PID=<... Put value here or use --pid-file option ...>;

# Prints number of open files by main process
lsof -p $PROXY_PY_PID | wc -l;

# Prints number of open files per worker process
pgrep -P $PROXY_PY_PID | while read pid; do lsof -p $pid | wc -l; done;


$ -h
usage: [-h] [--backlog BACKLOG] [--basic-auth BASIC_AUTH]
                [--ca-key-file CA_KEY_FILE] [--ca-cert-dir CA_CERT_DIR]
                [--ca-cert-file CA_CERT_FILE]
                [--ca-signing-key-file CA_SIGNING_KEY_FILE]
                [--cert-file CERT_FILE]
                [--client-recvbuf-size CLIENT_RECVBUF_SIZE]
                [--devtools-ws-path DEVTOOLS_WS_PATH]
                [--disable-headers DISABLE_HEADERS] [--disable-http-proxy]
                [--enable-devtools] [--enable-static-server]
                [--enable-web-server] [--hostname HOSTNAME]
                [--key-file KEY_FILE] [--log-level LOG_LEVEL]
                [--log-file LOG_FILE] [--log-format LOG_FORMAT]
                [--num-workers NUM_WORKERS]
                [--open-file-limit OPEN_FILE_LIMIT] [--pac-file PAC_FILE]
                [--pac-file-url-path PAC_FILE_URL_PATH] [--pid-file PID_FILE]
                [--plugins PLUGINS] [--port PORT]
                [--server-recvbuf-size SERVER_RECVBUF_SIZE]
                [--static-server-dir STATIC_SERVER_DIR] [--version] v1.1.0

optional arguments:
  -h, --help            show this help message and exit
  --backlog BACKLOG     Default: 100. Maximum number of pending connections to
                        proxy server
  --basic-auth BASIC_AUTH
                        Default: No authentication. Specify colon separated
                        user:password to enable basic authentication.
  --ca-key-file CA_KEY_FILE
                        Default: None. CA key to use for signing dynamically
                        generated HTTPS certificates. If used, must also pass
                        --ca-cert-file and --ca-signing-key-file
  --ca-cert-dir CA_CERT_DIR
                        Default: ~/ Directory to store dynamically
                        generated certificates. Also see --ca-key-file, --ca-
                        cert-file and --ca-signing-key-file
  --ca-cert-file CA_CERT_FILE
                        Default: None. Signing certificate to use for signing
                        dynamically generated HTTPS certificates. If used,
                        must also pass --ca-key-file and --ca-signing-key-file
  --ca-signing-key-file CA_SIGNING_KEY_FILE
                        Default: None. CA signing key to use for dynamic
                        generation of HTTPS certificates. If used, must also
                        pass --ca-key-file and --ca-cert-file
  --cert-file CERT_FILE
                        Default: None. Server certificate to enable end-to-end
                        TLS encryption with clients. If used, must also pass
  --client-recvbuf-size CLIENT_RECVBUF_SIZE
                        Default: 1 MB. Maximum amount of data received from
                        the client in a single recv() operation. Bump this
                        value for faster uploads at the expense of increased
  --devtools-ws-path DEVTOOLS_WS_PATH
                        Default: /devtools. Only applicable if --enable-
                        devtools is used.
  --disable-headers DISABLE_HEADERS
                        Default: None. Comma separated list of headers to
                        remove before dispatching client request to upstream
  --disable-http-proxy  Default: False. Whether to disable
  --enable-devtools     Default: False. Enables integration with Chrome
                        Devtool Frontend.
                        Default: False. Enable inbuilt static file server.
                        Optionally, also use --static-server-dir to serve
                        static content from custom directory. By default,
                        static file server serves from public folder.
  --enable-web-server   Default: False. Whether to enable
  --hostname HOSTNAME   Default: ::1. Server IP address.
  --key-file KEY_FILE   Default: None. Server key file to enable end-to-end
                        TLS encryption with clients. If used, must also pass
  --log-level LOG_LEVEL
                        Valid options: DEBUG, INFO (default), WARNING, ERROR,
                        CRITICAL. Both upper and lowercase values are allowed.
                        You may also simply use the leading character e.g.
                        --log-level d
  --log-file LOG_FILE   Default: sys.stdout. Log file destination.
  --log-format LOG_FORMAT
                        Log format for Python logger.
  --num-workers NUM_WORKERS
                        Defaults to number of CPU cores.
  --open-file-limit OPEN_FILE_LIMIT
                        Default: 1024. Maximum number of files (TCP
                        connections) that can open concurrently.
  --pac-file PAC_FILE   A file (Proxy Auto Configuration) or string to serve
                        when the server receives a direct file request. Using
                        this option enables proxy.HttpWebServerPlugin.
  --pac-file-url-path PAC_FILE_URL_PATH
                        Default: /. Web server path to serve the PAC file.
  --pid-file PID_FILE   Default: None. Save parent process ID to a file.
  --plugins PLUGINS     Comma separated plugins
  --port PORT           Default: 8899. Server port.
  --server-recvbuf-size SERVER_RECVBUF_SIZE
                        Default: 1 MB. Maximum amount of data received from
                        the server in a single recv() operation. Bump this
                        value for faster downloads at the expense of increased
  --static-server-dir STATIC_SERVER_DIR
                        Default: /Users/abhinav/Dev/ Static
                        server root directory. This option is only applicable
                        when static server is also enabled. See --enable-
  --version, -v         Prints version. not working? Report at:

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for, version 1.1.1
Filename, size File type Python version Upload date Hashes
Filename, size (58.1 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size (54.6 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page