SimPy.IO-SemVer 0.2.4rc20200630232623

Asynchronous networking based on SimPy.

simpy.io is an asynchronous networking library based on SimPy. The nature of networking is thoroughly event-based (messages can for example be thought of as events). simpy.io marries the solid event system of SimPy with asynchronous input and output.

It provides several degrees of abstraction (raw sockets, packets and request-reply messages), supports various back-ends ((e)poll, select, asyncore, virtual) and lets you use different socket types, like TCP, SSL-encrypted, simulated). Furthermore protocols like HTTP, WebSockets and an extensible RPC interface are also supported.

Current status

simpy.io is currently in the early alpha phase of development. There is no guarantee for API stability (modules will almost certainly be renamed before the 1.0 release) and simpy.io may break on your system. Sadly, there isn’t any documentation available as of now (apart from the examples and tests).

The state of the individual modules is as follows:

• sockets: stable alpha

• packets: stable alpha

• message: stable alpha

• http: draft

• websockets: draft

• rpc: draft

The following table represents the support matrix. As of now, there’s no CI system in place to guarantee the test stability, yet.

System	Flavor	Python	Backends	Successful tests
Linux	ArchLinux	2.6.9, 2.7.6, 3.3, 3.4	all	all
Linux	Ubuntu (12.04)	2.6.9, 2.7.6, 3.3	all	most
Windows	Windows 7	2.6.9, 2.7.6, 3.3	select	all
OS X	Maverick (10.9)	3.3	no epoll	most

Note

Python 2 does not natively support all of SimPy’s features (e.g. no exception chains or return statements inside of generators) but workarounds are available.

Installation

simpy.io requires Python 2.7 or 3.3 and SimPy 3. You can install it from Bitbucket via pip:

$ pip install hg+https://bitbucket.org/simpy/simpy.io

Examples

The following three examples demonstrate simpy.io’s levels of abstraction:

Socket level

When working directly with simpy.io sockets, you can try to read and write a specified number of bytes from or to a socket (note that there is no guarantee from the OS that all data will be received or transmitted):

>>> from simpy.io import select as backend
>>>
>>> def server(env, addr):
...     server_sock = backend.TCPSocket.server(env, addr)
...     sock = yield server_sock.accept()
...     data = yield sock.read(4)
...     print(data.decode())
...     yield sock.write('cya'.encode())
>>>
>>> def client(env, addr):
...     sock = backend.TCPSocket.connection(env, addr)
...     yield sock.write('ohai'.encode())
...     data = yield sock.read(3)
...     print(data.decode())
>>>
>>> addr = ('127.0.0.1', 5555)
>>> env = backend.Environment()
>>> srv = env.process(server(env, addr))
>>> cli = env.process(client(env, addr))
>>> env.run(until=cli)
ohai
cya

Packet level

simpy.io packets alleviate the limitation of raw sockets and allow you to read and write complete packets. These can either be bytes (if you use Packet) or (unicode) strings (if you use PacketUTF8):

>>> from simpy.io import select as backend
>>> from simpy.io.packet import Packet
>>>
>>> def server(env, addr):
...     server_sock = backend.TCPSocket.server(env, addr)
...     sock = yield server_sock.accept()
...     packet = Packet(sock)
...     data = yield packet.read()
...     print(data.decode())
...     yield packet.write('cya'.encode())
>>>
>>> def client(env, addr):
...     packet = Packet(backend.TCPSocket.connection(env, addr))
...     yield packet.write('ohai'.encode())
...     data = yield packet.read()
...     print(data.decode())
>>>
>>> addr = ('127.0.0.1', 5556)
>>> env = backend.Environment()
>>> srv = env.process(server(env, addr))
>>> cli = env.process(client(env, addr))
>>> env.run(until=cli)
ohai
cya

Message level

The message level adds message counters that allow you to asynchronously send messages (even concurrently) and maps replies to their proper requests. Furthermore, you can specify (de)serializers (by default, JSON is used) and replies can signal success/failure:

>>> from simpy.io import select as backend
>>> from simpy.io.packet import PacketUTF8
>>> from simpy.io.message import Message
>>>
>>> def server(env, addr):
...     server_sock = backend.TCPSocket.server(env, addr)
...     sock = yield server_sock.accept()
...     message = Message(env, PacketUTF8(sock))
...     request = yield message.recv()
...     print(request.content)
...     yield request.succeed('cya')
>>>
>>> def client(env, addr):
...     message = Message(env, PacketUTF8(
...             backend.TCPSocket.connection(env, addr)))
...     reply = yield message.send('ohai')
...     print(reply)
>>>
>>> addr = ('127.0.0.1', 5557)
>>> env = backend.Environment()
>>> srv = env.process(server(env, addr))
>>> cli = env.process(client(env, addr))
>>> env.run(until=cli)
ohai
cya

Help & Contact

Please feel free to post a message to the SimPy-Users mailing list to ask for help or to discuss the ongoing development. Bugs should be posted on our issue tracker here on BitBucket.

Maintenance

Updating the ca certificate and server key files:

$ openssl req -x509 -nodes -newkey rsa:4096 -keyout server.key.pem -out certificate.pem -days 3650

Verifying the ssl files:

$ openssl x509 -days 3650 -text -noout -in certificate.pem

Freeze the requirements:

$ .tox/py37/bin/python -m pip freeze –all –exclude-editable > requirements.txt

Benchmark

$ python examples/benchmark.py async: 7693 messages of length 10 per 1 s means 7693 messages/s, which saturate 76.930 kilobyte/s or 0.615 megabit/s. poll: 8162 messages of length 10 per 1 s means 8162 messages/s, which saturate 81.620 kilobyte/s or 0.653 megabit/s. select: 8147 messages of length 10 per 1 s means 8147 messages/s, which saturate 81.470 kilobyte/s or 0.652 megabit/s. virtual: 8142 messages of length 10 per 1 s means 8142 messages/s, which saturate 81.420 kilobyte/s or 0.651 megabit/s.

Stress Test

$ python examples/stress_test.py async: 5036 messages of length 16000 bytes per 1 s means 5036 messages/s, which saturate 80576.000 kilobyte/s or 644.608 megabit/s. poll: 5338 messages of length 16000 bytes per 1 s means 5338 messages/s, which saturate 85408.000 kilobyte/s or 683.264 megabit/s. select: 5255 messages of length 16000 bytes per 1 s means 5255 messages/s, which saturate 84080.000 kilobyte/s or 672.640 megabit/s. virtual: 4066 messages of length 16000 bytes per 1 s means 4066 messages/s, which saturate 65056.000 kilobyte/s or 520.448 megabit/s.