Skip to main content

NETunnel

NETunnel is a tool to create network tunnels over HTTP/S written in Python 3.
It can be used both in a client-server and in a server-server model.

Getting Started

Installing

pip install netunnel

We officially support Python 3.6+.

Usage

The following example creates an HTTP tunnel from the client to the server's port 22 (SSHD service).

Running the server: (In production, use --config-path to preserve changes)

$ python -m netunnel.server
The server is running in stateless mode. Use --config-path to generate a config file
netunnel_server - INFO - Generating default secret-key
netunnel_server - INFO - Starting server on 127.0.0.1:4040

Running the client:

$ python -m netunnel.client --remote-port 22
Opening tunnel to the server...
Tunnel entrance socket: 127.0.0.1:54781
Tunnel exit socket: 127.0.0.1:22

The server's SSHD service is now accessible from the client:

$ ssh -p 54781 localhost

Please take a look at the examples directory for additional usages.

How it works

  1. The client connects to the server and creates a websocket connection we call a "Channel". The channel is used by server to send commands to the client and it performs heartbeat pings so both sides will know if there are connection issues.
  2. The client makes a POST request to create a tunnel.
  3. Either the client or the server (depends on the tunnel's direction) listens on a socket locally for incoming connections.
  4. For every new connection, it generates a websocket connection to the remote, and stream the data from the connection over to the websocket and vice versa.
  5. Whenever a websocket is created, the remote connects to a socket locally and stream data from the websocket to the socket and vice versa.
Connection1 ->                      Websocket1 ->               ->
Connection2 ->  Tunnel Entrance ->  Websocket2 ->  Tunnel Exit  -> Service
Connection3 ->                      Websocket3 ->               ->

Under the hood

There are 2 core objects which performs the tunnelling:

  • InputTunnel - The tunnel's entrance which listens on a socket.
  • OutputTunnel - The tunnel's exit which creates connections to a socket.

When a normal tunnel is created, the client creates an InputTunnel and the server creates an OutputTunnel, while reverse tunnels are essentially the server is creating InputTunnel while the client is creating an OutputTunnel.

InputTunnel is initialized with a feeder of websockets that the client/server provides, so that the implementation can be generic. In reverse tunnels, The server uses the channel to request the client for a new websocket when it needs to feed a new websocket.

Server Configurations

The server's configuration file is optional, but recommended in production environments.

When running the NETunnel server, you can provide a path to a configuration file using -c or --config-path flags, and the server will generate a default configuration file at that location. If there is an existing configuration in that path, the server will load it, and merge it with its default configurations, and for any change that was made dynamically to the server using the API, it will commit it to the configuration file.

The configuration file is in JSON format and support the following keys:

  • allowed_tunnel_destinations - A key-value mapping of IPs and ports(as strings separated by comma) allowed to be used as a tunnel's exit sockets. The special symbol * supported to allow all ports for a certain IP. Defaults to {"127.0.0.1": "*"}
  • secret_key - A passphrase used as an encryption key for sensitive settings to avoid storing them in the disk as plain text. The key is generated automatically, but we recommend using the -s/--secret-key when running the server which will avoid storing the key in the configuration file. Setting the environment variable NETUNNEL_SECRET_KEY will behave just the same as the flag, and won't be stored in the configuration. If you wish to decrypt, encrypt, or generate a key manually, see python -m netunnel.common.security.
  • peers - A list of remote NETunnel servers that can be used to set static tunnels (See Peers in the Additional Features). For an example of how to set a peer, look at examples/server-server.
  • allow_unverified_ssl_peers - When set to true, remote peers certificates won't be verified. Defaults to false.
  • revision - Currently unused. This will be used for configuration migrations purposes. You should not modify this field manually in any use case.
  • http_proxy - Settings for an optional global HTTP proxy to use for any requests the server may need to make, for example to remote peers. The setting include a key-value mapping of the following:
    • proxy_url - The URL to the remote proxy server
    • username - An encrypted (using the secret_key) username string
    • password - An encrypted (using the secret_key) password string

A useful feature of NETunnel configuration is that it can parse environment variables on load to modify the default values of any key. The configuration will search for variables starting with the prefix NETUNNEL_, following by the uppercase of any existing key. The value is expected to be in JSON format.

For example, in POSIX environments, running:

export NETUNNEL_ALLOWED_TUNNEL_DESTINATIONS='{"127.0.0.1": "22"}'
export NETUNNEL_ALLOW_UNVERIFIED_SSL_PEERS='true'
python -m netunnel.server

Will change the default allowed_tunnel_destinations to {"127.0.0.1": "22"} and the default allow_unverified_ssl_peers to true.

An example for a configuration file: examples/netunnel.example.conf

Additional Features

  • Peers - A NETunnel server can register remote NETunnel servers called peers. The peers are stored in the configuration and can be used to create static tunnels.
  • Static Tunnels - NETunnel supports permanent tunnels between servers. This is a useful feature for when we want a long term tunnels between machines. It can be used by making the netunnel server to run as a service and create a configuration file with peers and static tunnels which initialized on startup. Both peers and static tunnels can also be created dynamically via the server's API. An example for a server-server model can be found here: examples/server-server
  • HTTP proxy support - You can use --proxy-url --proxy-username --proxy-password to configure a proxy for the client. When used in a server-server model, there can be a global proxy used by the server to connect with. The credentials of the global proxy in that case are stored encrypted in the server's configuration using a secret_key.
  • Authentication plugins - By default, no authentication is made between NETunnel instances. This can be configured by inherit the auth classes on netunnel/common/auth.py and pass them to the client and server using --auth-plugin. A plugin example: examples/secret-auth-plugin

Changelog

1.0.5 (2021-03-30)

Features

  • Add a dynamic factory reset REST API call

Misc

  • Set marshmallow requirement to less than 4.0.0

1.0.4 (2021-01-25)

Misc

  • Include LICENSE, CHANGES.md and README.md in the PyPi package.

1.0.3 (2021-01-25)

Misc

  • Extend supported version of importlib-metadata to the lowest.

Metadata

Release files for netunnel 1.0.13

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for netunnel 1.0.13
File Size Uploaded Provenance Actions
netunnel-1.0.13.tar.gz 45.3 kB Without provenance Details for netunnel-1.0.13.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for netunnel 1.0.13
File Interpreter ABI Platform Actions
netunnel-1.0.13-py3-none-any.whl Python 3 none any Details for netunnel-1.0.13-py3-none-any.whl

Total release size: 94.0 kB

Release files / netunnel-1.0.13.tar.gz

Download URL netunnel-1.0.13.tar.gz
Size 45.3 kB
Tags Source
SHA-256 checksum
How to use checksums
a1e82b028c41028c73f484aca29926a7f6efbda1ec61ac5bbadb70fe1996dcf7
BLAKE2b-256 checksum
How to use checksums
91adcf44cbe82f844c2b55ba400a838e0a05443696005c0fa71d5b56002fce9c
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.0.1 CPython/3.12.8

Release files / netunnel-1.0.13-py3-none-any.whl

Download URL netunnel-1.0.13-py3-none-any.whl
Size 48.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
e3c618f37a418e61a7ab6637cc1b7bfba248b6610374f50e5fb35f8bf954ee28
BLAKE2b-256 checksum
How to use checksums
924cea9d9611224c742577533ea054a4f0bdbdca6b6bc2d54e5790d07b543272
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.0.1 CPython/3.12.8

Security for netunnel 1.0.13

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page