A shadowsocks manager for multi-user and traffic statistics
Project description
shadowsocks-manager
A web-based Shadowsocks management tool.
Features:
- Central user management
- Heartbeat on Shadowsocks ports(users)
- Shadowsocks multi-user API
- Shadowsocks node cluster
- Statistic for network traffic usage
- Scheduled jobs
- name.com API
- Auto-creating DNS records
- Production deployment ready
- How's the Shadowsocks supported:
- libev edition:
- Full functional.
- No builtin service manager, you need to install it and start the service by yourself.
- python edition:
- Lacks the collection of traffic statistics.
- Lacks the ability to test user port creation status.
Pre-installed, and have a builtin service manager.- No builtin service manager, you need to install it and start the service by yourself.
- libev edition:
Code in Python, base on Django, Django REST framework, Celery, and SQLite.
The development status can be found at: project home.
Screenshots
Shadowsocks Node List:
Add Shadowsocks Node:
Add Shadowsocks Account:
1. Requirements
- Python 2.7, Python 3.x
- Django 1.11.x, Django 3.x
- Docker
- Shadowsocks-libev 3.2.0 for Linux (multi-user API is required)
2. Install
This project is a part of an entire VPN solution, which includes the Shadowsocks server and Shadowsocks manager. The Shadowsocks server serves the traffic, the Shadowsocks manager serves the users and the traffic statistics. The solution is designed to be deployed in the AWS cloud. If you are looking for such a solution, you can refer to the repo aws-cfn-vpn. With aws-cfn-vpn
, you can deploy the entire solution with a few commands.
2.1. Dependencies
Assume you have installed the Docker on your host.
2.2. Manual installation
# create a docker network
docker network create ssm-network
# run memcached, used by django cache
docker run -d --network ssm-network --name ssm-memcached memcached
# run rabbitmq, used by celery
docker run -d --network ssm-network --name ssm-rabbitmq rabbitmq
# create a directory to store the data, it will be mounted to the shadowsocks-manager container
mkdir -p ~/ssm-volume
# run the shadowsocks-manager
docker run -d -p 80:80 -v ~/ssm-volume:/var/local/ssm \
--network ssm-network --name ssm alexzhangs/shadowsocks-manager \
-e SSM_SECRET_KEY=yourkey -e SSM_DEBUG=False \
-e SSM_MEMCACHED_HOST=ssm-memcached -e SSM_RABBITMQ_HOST=ssm-rabbitmq \
-u admin -p yourpassword -M admin@yourdomain.com
2.3. Install with script
git clone https://github.com/alexzhangs/shadowsocks-manager
bash shadowsocks-manager/install.sh -u admin -p yourpassword -M admin@yourdomain.com
2.4. Verify the installation
If all go smoothly, the shadowsocks-manager services should have been all started. Open the web admin console in a web browser, and log on with the admin user.
Use:
http://<your_server_ip>/admin
or
http://localhost/admin
If goes well, then congratulations! The installation has succeeded.
3. Using shadowsocks-manager
-
Shadowsocks server
First, you need to have a Shadowsocks server with the multi-user API enabled.
Install it with docker on the same server with shadowsocks-manager.
# run shadowsocks-libev MGR_PORT=6001 SS_PORTS=8381-8480 ENCRYPT=aes-256-cfb docker run -d -p $SS_PORTS:$SS_PORTS/UDP -p $SS_PORTS:$SS_PORTS \ --network ssm-network --name ssm-ss-libev shadowsocks/shadowsocks-libev:edge \ ss-manager --manager-address 0.0.0.0:$MGR_PORT \ --executable /usr/local/bin/ss-server -m $ENCRYPT -s 0.0.0.0 -u # Use below command to get the private IP address of the shadowsocks-libev container for later configuration. docker inspect ssm-ss-libev | grep IPAddress
-
Add Shadowsocks server to shadowsocks-manager
Add the Shadowsocks server as a Node of shadowsocks-manager from web admin console:
Home › Shadowsocks › Shadowsocks Nodes
. -
Create users(ports) and assign Shadowsocks Node
Create users from web admin console:
Home › Shadowsocks › Shadowsocks Accounts
and assign the existing nodes to them.After a few seconds, the created user ports should be available to your Shadowsocks client.
4. Sendmail (Optional)
sendmail
is used to send account notification Email, it should
be configured on the same server with shadowsocks-manager.
About how to configure sendmail
client to use AWS SES as SMTP server on AWS EC2 instance, refer to repo
aws-ec2-ses.
On macOS, refer to repo macos-aws-ses.
NOTE: This dependency needs the manual setup anyway, it is not handled by any installation script.
5. Differences from the alternation: shadowsocks/shadowsocks-manager
This repo Do's:
- Serve as a nonprofit business model.
- Have central user management for multi nodes.
- Collect traffic statistic that can be viewed by account, node, and period.
- Show the existence and accessibility of ports in the admin.
- Handle the DNS records if using Name.com as nameserver.
This repo Don'ts:
- Handle self-serviced user registration.
- Handle bill or payment.
- Need to run an additional agent on each Shadowsocks server.
6. Some differences between the Shadowsocks Python edition (2.8.2) and libev edition
Version status for the Shadowsocks Python edition:
Although the Shadowsocks Python edition supports the multi-user API, but it doesn't fit this project, here's why:
- The python edition code and doc seem to be out of maintenance due to some reason. If you really need this you probably need to fork it and make your own.
- They are having different service process names and CLI interfaces which introduces the complexity of installation.
- The Python edition lacks the
list
commands. A pull request was opened years ago but never merged. - The Python edition's
stat
command has a very different way to use, I didn't figure the usage syntax out by looking into the code. - The Python edition's
ping
command returns a simple stringpong
rather than a list of ports. - The Python edition's
ping
command has to be sent as the syntax:ping:{}
in order to work if tested withnc
. It caused by the tailing newline:ping\n
.
So either you get some change on your own or stick with the libev edition.
7. Known Issues
-
DNS records matching for Node may not be accurate on macOS. For unknown reason sometimes DNS query returns only one IP address while multiple IP addresses were configured for the domain.
-
The Shadowsocks Python edition's ssserver won't start on macOS. The error message is like:
$ ssserver -k passw0rd WARNING: /Users/***/.pyenv/versions/3.12.0/envs/ssm-3.12/bin/python3.12 is loading libcrypto in an unsafe way Abort trap: 6
Solution: Link the homebrew openssl library to the system library.
sudo ln -s /opt/homebrew/opt/openssl/lib/libcrypto.dylib /usr/local/lib/ sudo ln -s /opt/homebrew/opt/openssl/lib/libssl.dylib /usr/local/lib/
-
Install the project by source with pip under Python 2.7 get error:
python --version Python 2.7.18 pip install .
Error message:
... Collecting pyyaml Downloading PyYAML-5.4.1.tar.gz (175 kB) |████████████████████████████████| 175 kB 392 kB/s Installing build dependencies ... done Getting requirements to build wheel ... error ERROR: Command errored out with exit status 1: ... raise AttributeError, attr AttributeError: cython_sources ...
Solution:
pip install setuptools wheel pip install --no-build-isolation .
-
Install the project with pip under python 2.7 get error:
python --version Python 2.7.18 pip install .
Error message:
... Building wheels for collected packages: cryptography Building wheel for cryptography (PEP 517) ... error ERROR: Command errored out with exit status 1: build/temp.macosx-14.2-arm64-2.7/_openssl.c:18674:10: error: call to undeclared function 'ERR_GET_FUNC'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] return ERR_GET_FUNC(x0); ^ build/temp.macosx-14.2-arm64-2.7/_openssl.c:18690:14: error: call to undeclared function 'ERR_GET_FUNC'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] { result = ERR_GET_FUNC(x0); } ^ build/temp.macosx-14.2-arm64-2.7/_openssl.c:23389:10: error: call to undeclared function 'FIPS_mode'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] return FIPS_mode(); ^ build/temp.macosx-14.2-arm64-2.7/_openssl.c:23400:14: error: call to undeclared function 'FIPS_mode'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] { result = FIPS_mode(); } ^ build/temp.macosx-14.2-arm64-2.7/_openssl.c:23415:10: error: call to undeclared function 'FIPS_mode_set'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] return FIPS_mode_set(x0); ^ build/temp.macosx-14.2-arm64-2.7/_openssl.c:23431:14: error: call to undeclared function 'FIPS_mode_set'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] { result = FIPS_mode_set(x0); } ... ERROR: Failed building wheel for cryptography
Solution:
brew install openssl@1.1
8. Development
The development of this project requires Python 3.x.
However, the installation of the project is compatible with both Python 2.7 and 3.x. To keep the compatibility is difficult, but it's kept due to the historical reason. The following files are kept only for installing the source distribution of the PyPI package under Python 2.7:
- setup.py
- setup.cfg
8.1. Development Environment Setup
-
Install the dependencies
# run memcached, used by django cache docker run -d -p 11211:11211 --name ssm-dev-memcached memcached # run rabbitmq, used by celery docker run -d -p 5672:5672 --name ssm-dev-rabbitmq rabbitmq # run shadowsocks-libev, simulate localhost node MGR_PORT=6001 SS_PORTS=8381-8384 ENCRYPT=aes-256-cfb docker run -d -p 127.0.0.1:$MGR_PORT:$MGR_PORT/UDP \ -p 127.0.0.1:$SS_PORTS:$SS_PORTS/UDP -p 127.0.0.1:$SS_PORTS:$SS_PORTS \ --name ssm-dev-ss-libev-localhost shadowsocks/shadowsocks-libev:edge \ ss-manager --manager-address 0.0.0.0:$MGR_PORT \ --executable /usr/local/bin/ss-server -m $ENCRYPT -s 0.0.0.0 -u # get the private IP address of the host, double check the result, it might not be correct PRIVATE_IP=$(ipconfig getifaddr en0 2>/dev/null || hostname -i | awk '{print $1}' 2>/dev/null) echo "PRIVATE_IP=$PRIVATE_IP" # run shadowsocks-libev, simulate private IP node MGR_PORT=6002 SS_PORTS=8381-8384 ENCRYPT=aes-256-cfb docker run -d -p $PRIVATE_IP:$MGR_PORT:$MGR_PORT/UDP \ -p $PRIVATE_IP:$SS_PORTS:$SS_PORTS/UDP -p $PRIVATE_IP:$SS_PORTS:$SS_PORTS \ --name ssm-dev-ss-libev-private shadowsocks/shadowsocks-libev:edge \ ss-manager --manager-address 0.0.0.0:$MGR_PORT \ --executable /usr/local/bin/ss-server -m $ENCRYPT -s 0.0.0.0 -u # run shadowsocks-libev, simulate public IP node MGR_PORT=6003 SS_PORTS=8385 ENCRYPT=aes-256-cfb docker run -d -p $PRIVATE_IP:$MGR_PORT:$MGR_PORT/UDP \ -p $PRIVATE_IP:$SS_PORTS:$SS_PORTS/UDP -p $PRIVATE_IP:$SS_PORTS:$SS_PORTS \ --name ssm-dev-ss-libev-public shadowsocks/shadowsocks-libev:edge \ ss-manager --manager-address 0.0.0.0:$MGR_PORT \ --executable /usr/local/bin/ss-server -m $ENCRYPT -s 0.0.0.0 -u
-
Link the project code in your workspace to the Python environment
cd shadowsocks-manager pip install -e .
-
Set the SSM_DATA_HOME environment variable (optional, default is Django root directory)
export SSM_DATA_HOME=~/.ssm-dev-data
-
Configure the shadowsocks-manager
ssm-setup -c -m -l -u admin -p yourpassword
-
Run the development server
Make sure the memcached and rabbitmq are running.
ssm-dev-start
-
Stop the development server
ssm-dev-stop
-
Test the Django code
Make sure the memcached and rabbitmq are running, and also the 3 shadowsocks-libev node are running.
ssm-test -t # or use the django test command directly for more options, use the `-t .` for the Python 2.7 compatibility ssm-manage test --no-input -v 2 -t .
-
Test the Django code with coverage
pip install coverage ssm-test -c
-
Upload the coverage report to codecov
Make sure the
CODECOV_TOKEN
is exported in the environment before uploading.pip install codecov-cli ssm-test -u
-
Test the Github workflows locally
brew install act act -j test
-
Build the PyPI package
pip install build # build source and binary distribution, equivalent to `python setup.py sdist bdist_wheel` # universal wheel is enabled in the pyproject.toml to make the wheel compatible with both Python 2 and 3 python -m build
-
Upload the PyPI package
Set the ~/.pypirc file with the API token from the TestPyPI and PyPI before uploading.
pip install twine # upload the package to TestPyPI python -m twine upload --repository testpypi dist/* # upload the package to PyPI python -m twine upload dist/*
-
Test the PyPI package
# install the package from TestPyPI # --no-deps is used to skip installing dependencies for the TestPyPI environment pip install -i https://test.pypi.org/simple/ --no-deps shadowsocks-manager # install the package from PyPI # --no-binary is used to force building the package from the source # --use-pep517 is used together to make sure the PEP 517 is tested pip install --no-binary shadowsocks-manager --use-pep517 shadowsocks-manager
-
Build the docker image
docker build -t alexzhangs/shadowsocks-manager -f docker/debian/Dockerfile . # or use: bash docker/docker-build-and-run.sh
8.2. CI/CD
Github Actions is currently used for the CI/CD. Travis CI is removed due to the limitation of the free plan.
The CI/CD workflows are defined in the .github/workflows
directory.
- ci-unittest.yml: Run the unit tests.
- ci-testpypi.yml: Build and upload the package to TestPyPI.
- ci-pypi.yml: Build and upload the package to PyPI. It can be triggered by the tag:
ci-pypi
orci-pypi-(major|minor|patch|suffx)
. - ci-docker.yml: Build and push the docker image to Docker Hub. It can be triggered by the Github release.
9. Troubleshooting
-
Docker
# containers docker ps -a # network docker network ls # logs docker logs <container_id>
-
Check the logs (inside container)
# supervisor (debian) cat /var/log/supervisor/supervisord.log # supervisor (alpine) cat /var/log/supervisord.log # uWSGI cat /var/log/ssm-uwsgi.log # Celery cat /var/log/ssm-cerlery*
-
Check the services (inside container)
# nginx nginx -s {stop|quit|reopen|reload} # supervisor supervisorctl reload supervisorctl start all # uWSGI supervisorctl start ssm-uwsgi # Celery supervisorctl start ssm-celery-worker supervisorctl start ssm-celery-beat # acme.sh acme.sh --list crontab -l ls -l /root/.acme.sh # volume data ls -la /var/local/ssm # setup done files ls -la /var/local/ssm/.*done ls -la /root/.*done
-
Check the listening ports and processes (Linux)
# TCP netstat -tanp # UDP netstat -uanp
-
Check the listening ports (MacOS)
# TCP netstat -anp tcp # UDP netstat -anp udp # find the process by port lsof -i :80 -P
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for shadowsocks_manager-0.1.10.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | de6200c07253e2ad29b32a36df71fe3705aadc9154d4d21c1be57b6db585760f |
|
MD5 | 424342a80950ac379d750ad065d4895c |
|
BLAKE2b-256 | 90bf37ef5e131b18e371ea4ed415aafea162820a2c3ade43be35eef872838c33 |
Hashes for shadowsocks_manager-0.1.10-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 2d4bd204478875afb2ef725f44783e7d779031a3b4500443af8e6bee9fe77fba |
|
MD5 | 82b690c53a6fdf1b6d968477f2f4d054 |
|
BLAKE2b-256 | 33d954f00340ffc6d35e30e38226f0e14d344a98cedab0f7e91034f1018da2ab |