Skip to main content

3xsd is a native epoll server serving TCP/UDP connections, a high performance static web server, a failover dns server, a http-based distributed file server, a load-balance proxy-cache server, and a warp-drive server. Written in python, take the full power of multi-cores.

Project description

##3xsd 3xsd is a native epoll server serving TCP/UDP connections, a high performance static web server, a failover dns server, a http-based distributed file server, a load-balance proxy-cache server, and a ‘warp drive’ server. Written in python, take the full power of multi-cores.

##Features in detail:

###3wsd - web server
supporting: static files, event driven(epoll), using mmap & sendfile to send files, in-mem xcache, transparent gzip content transfer with fixed length(small file) & chunked(large file), persistent storage of gzip files, partial support of WebDAV(PUT/DELETE), pipelining support
###3nsd - dns server
supporting: only A record resolution, domainname failover(refer to conf file), ip icmp probe & hide when fail, round robbin ip resolving global DNS Left-Right Range Resolve(LRRR)(experimental)
###3zsd - proxy server
supporting: load balance backend servers, in-mem file caching & persistent cache file storage
###3fsd - distribute web file system
supporting: mass unlimitted file storage, easy to expand, O(1) location algorithm, non-centralized, can work with standard web server(WebDAV) in proxy mode, file redundancy, file persistent caching
###3wdd - ‘warp drive’ server
supporting: data tunneling over UDT and tun, better congestion control than TCP/UDP over wan link, better thoughput(above 80%) over wan link, refer to this report: tunnel ip/mtu/txqueuelen/route define, auto create/recreate/destroy encrypt packages through AES-128-ECB/CBC/CFB and Blowfish-CBC tunnel on-the-fly compress with zlib/lzo, tunnel data relaying route metric, routing data through different path, depending on tunnel rtt(choose the best one)

More to find in .conf file.



Small file under 1KB single process test(full in-mem), contrast with nginx configuring accept_mutex off, 80% performance. Multi processes test, with reuse_port enabling kernel, 95% performance of nginx(and beyond, may be 105% or more, based on process number, I tested 2-4). The tests above is not quite strict, but I just want to say that it’s fast enough.

And with pipelining enabled, 3wsd will perform better with 3-4 requests/send(5%-10% performance increase), 2 requests/send have the same speed with non-piplining.

About 80% performance of 3wsd.
Fast enough…about 2800-3000 queries/s per processes, with 1GHz bcm2709 4-cores ARMv7 cpu testing, better when multi-processes with reuse_port enabling kernel.
Same with 3zsd.
Early testing indicated that: UDT tunnel(no encrypt) performing 50%-60% speed of direct TCP connection with ZetaTCP, and package lost rate remaining below 0.6%, while direct connection has 1.4%-3%. (Test CN-US WAN link with 150ms-280ms latency, through the always-jammed CUCN submarine cable) However, UDT tunnel beats normal TCP connection without ZetaTCP, with 50% - 4 times (commonly 1-2 times) outperforming.(v)(Test link like above) And an encrypted UDT tunnel will has 50% performance decrease.

###More performance: There are at lease two ways to increase the performance of 3xsd:

1.Install Cython, and rename to _3xsd.pyx, run it. Cython will compile lib into a file, using static type declarations. This can gain about 5%-6% performance increasement. 2.Use PyPy.This can gain about 10%-15% performance increasement(or more).

#OS requirement & install:

CentOS 6/7 with python 2.6/2.7, Debian 6/7. Python 2.7 recommended.

Doing this before running the program(minimal requirement):

yum install python-gevent pysendfile python-setproctitle python-psutil python-pip

(python-pip is optional if install dpkt)

Dpkt module is also needed when running 3nsd DNS server, pip install it.

If you want to use 3wdd, python-pytun, pyudt4, pycrypto, python-lzo are also needed.

yum install python-crypto2.6 python-lzo

will quickly install pycrypto(probably do some ‘linking’ works) and lzo. The other two depended on pip install.

Probably you need this easy-install.pth file in python’s site-packages dir:

import sys; sys.__plen = len(sys.path) ./pycrypto-2.6.1-py2.6-linux-x86_64.egg ./pyudt4-0.6.0-py2.6-linux-x86_64.egg import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,’__egginsert’,0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

I provide a pre-compiled package [pyudt_tun-centos6-x86_64.tar.gz]( to simplify the installation procedure of pyudt4 & python-pytun.

Be aware of pyudt4 having some bugs, you’d better download it’s source code of epoll-fixes branch and apply the patch I offered. See changelog.txt v0.0.20 2016.03.07 fixed section for detail. (Already included in [pyudt_tun-centos6-x86_64.tar.gz](

Or, of cause you can let pip do it all for you(not including patching pyudt4):

pip install 3xsd

In a debian, you can use apt-get to install python-pip(pip) or python-setuptools(easy_install), then to install the packages following.

Python Packages(Modules) version reference:

gevent==0.13.8(1.0.1, 1.1) greenlet==0.4.2 pysendfile==2.0.1 setproctitle==1.0.1 psutil==0.6.1 dpkt==1.6(1.8.6) python-pytun==2.2.1 pyudt4==0.6.0(epoll-fixes branch) pycrypto==2.6.1 python-lzo==1.8

System libs version reference:

libevent-1.4.13-4(not actually used, just needed for gevent to function) udt-4.11-6 lzo-2.03-3.1

To install a module of specific version(like gevent 0.13.8), you can:

pip install gevent==0.13.8

This will install the latest version of gevent(pypy will need it):

pip install git+git://

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for 3xsd, version 0.0.25
Filename, size File type Python version Upload date Hashes
Filename, size 3xsd-0.0.25.tar.gz (41.8 kB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring DigiCert DigiCert EV certificate Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page