A cython wrapping of the C++ Cap'n Proto library with support for asyncio
Project description
pycapnp-async
Requirements
C++14 supported compiler
gcc 6.1+ (5+ may work)
clang 6 (3.4+ may work)
Visual Studio 2017+
cmake (needed for bundled capnproto)
ninja (macOS + Linux)
Visual Studio 2017+
capnproto-0.7.0
Not necessary if using bundled capnproto
32-bit Linux requires that capnproto be compiled with -fPIC. This is usually set correctly unless you are compiling canproto yourself. This is also called -DCMAKE_POSITION_INDEPENDENT_CODE=1 for cmake.
pycapnp has additional development dependencies, including cython and pytest. See requirements.txt for them all.
Building and installation
Install with pip install pycapnp. You can set the CC environment variable to control which compiler is used, ie CC=gcc-8.2 pip install pycapnp.
Or you can clone the repo like so:
git clone https://github.com/haata/pycapnp-async.git
cd pycapnp-async
pip install .
If you wish to install using the latest upstream C++ Cap’n Proto:
pip install \
--install-option "--libcapnp-url" \
--install-option "https://github.com/sandstorm-io/capnproto/archive/master.tar.gz" \
--install-option "--force-bundled-libcapnp" .
To force bundled python:
pip install --install-option "--force-bundled-libcapnp" .
Python Versions
Python 3.7+ is supported. Earlier versions of Python have asyncio bugs that might be possible to work around, but may require significant work (3.5 and 3.6).
Development
Git flow has been abandoned, use master.
To test, use a pipenv (or install requirements.txt and run pytest manually).
pip install pipenv
pipenv install
pipenv run pytest
Binary Packages
Building a dumb binary distribution:
python setup.py bdist_dumb
Building a Python wheel distributiion:
python setup.py bdist_wheel
Pypi Upload Instructions
Only necessary if uploading release to pypi.org.
TODO
Documentation/Example
There is some basic documentation here.
The examples directory has one example that shows off pycapnp quite nicely. Here it is, reproduced:
from __future__ import print_function
import os
import capnp
import addressbook_capnp
def writeAddressBook(file):
addresses = addressbook_capnp.AddressBook.new_message()
people = addresses.init('people', 2)
alice = people[0]
alice.id = 123
alice.name = 'Alice'
alice.email = 'alice@example.com'
alicePhones = alice.init('phones', 1)
alicePhones[0].number = "555-1212"
alicePhones[0].type = 'mobile'
alice.employment.school = "MIT"
bob = people[1]
bob.id = 456
bob.name = 'Bob'
bob.email = 'bob@example.com'
bobPhones = bob.init('phones', 2)
bobPhones[0].number = "555-4567"
bobPhones[0].type = 'home'
bobPhones[1].number = "555-7654"
bobPhones[1].type = 'work'
bob.employment.unemployed = None
addresses.write(file)
def printAddressBook(file):
addresses = addressbook_capnp.AddressBook.read(file)
for person in addresses.people:
print(person.name, ':', person.email)
for phone in person.phones:
print(phone.type, ':', phone.number)
which = person.employment.which()
print(which)
if which == 'unemployed':
print('unemployed')
elif which == 'employer':
print('employer:', person.employment.employer)
elif which == 'school':
print('student at:', person.employment.school)
elif which == 'selfEmployed':
print('self employed')
print()
if __name__ == '__main__':
f = open('example', 'w')
writeAddressBook(f)
f = open('example', 'r')
printAddressBook(f)
Also, pycapnp has gained RPC features that include pipelining and a promise style API. Refer to the calculator example in the examples directory for a much better demonstration:
import capnp
import socket
import test_capability_capnp
class Server(test_capability_capnp.TestInterface.Server):
def __init__(self, val=1):
self.val = val
def foo(self, i, j, **kwargs):
return str(i * 5 + self.val)
def server(write_end):
server = capnp.TwoPartyServer(write_end, bootstrap=Server(100))
def client(read_end):
client = capnp.TwoPartyClient(read_end)
cap = client.bootstrap()
cap = cap.cast_as(test_capability_capnp.TestInterface)
remote = cap.foo(i=5)
response = remote.wait()
assert response.x == '125'
if __name__ == '__main__':
read_end, write_end = socket.socketpair(socket.AF_UNIX)
# This is a toy example using socketpair.
# In real situations, you can use any socket.
server(write_end)
client(read_end)
Changelog
v0.6.4 (2019-01-31)
Fix bugs in read_multiple_bytes (thanks to @tsh56)
Remove end-of-life Python versions 2.6, 3.2, and 3.3. Add CI tests for 3.6
Expose SchemaParser in Cython header
v0.6.3 (2018-01-14)
Bump bundled capnp version to v0.6.1 (thanks to @E8Yuval)
Fix a memleak in RemotePromise (thanks to @E8Yuval)
v0.6.2 (2017-11-30)
Add support for buffers/memoryviews in from_bytes (thanks to @aldanor)
v0.6.1 (2017-07-27)
Fixed upload to PyPi (forgot to cythonize)
v0.6.0 (2017-07-27)
Update bundled capnp version to v0.6.0 and fix related problems (thanks to @benmoran)
Fix memleak with KjException (thanks to @tsh56)
v0.5.12 (2017-04-18)
Bump bundled capnp version to v0.5.3.1
v0.5.11 (2017-04-10)
Make enums hashable (thanks to @madeleine-empirical)
Rework logic on when to build bundled libcapnp. Fixes cross-compilation (thanks to @benizl)
Add traversal_limit_in_words and nesting_limit to RPC classes (thanks to @asilversempirical)
Include class attributes in dir. This allows for code completion of class methods (thanks to @chaoflow )
Allow setting lists with python tuples (thanks to @chaoflow)
Fix traversal_limit_in_words and nesting_limit being ignored by from_bytes (thanks to @plesner)
v0.5.10 (2016-11-28)
Fix bug that prevented event loop from actually being lazy initialized
Fix possible recursive loop in KjException
Add clear_write_flag method to builder classes
v0.5.9 (2016-07-07)
Make the event loop be lazy initialized
Add support for segment (de)serialization (thanks to @gcv). See to_segments/from_segments methods.
Fix response objects not referencing parents correctly
Add test for large reads
v0.5.8 (2016-05-27)
Fix build problem with Cython v0.24
Include the changelog in the manifest (should fix install problems if pandoc is present)
Include the traceback in exceptions
Make sure to encode to utf-8, not the default encoding (thanks to @novas0x2a)
Add –libcapnp-url option in installer to allow installing arbitrary libcapnp versions
Support mmap objects for reading with from_bytes (thanks to @bpiwowar)
Change read_multiple and read_multiple_packed to copy by default
Fix mistakenly discarding the file parameter on reads
Add reraise_kj_exception to the prettyPrint functions. (thanks to @kdienes)
Fix KjException init (missing wrapper). (thanks to @E8-Storage)
Add result_type to InterfaceMethodSchema
v0.5.7 (2015-06-16)
Update bundled libcapnp to v0.5.2
Add warnings for using old restorer methods. You should use bootstrap instead
Fix warning from PyEventPort
Handle AnyPointers better as arguments to RPC functions
Add support for using keyword arguments with a named struct in an RPC
Add bootstrap method to TwoPartyServer
Add init method to lists
Add support for unix sockets in RPC
v0.5.6 (2015-04-13)
Fix a serious bug in TwoPartyServer that was preventing it from working when passed a string address.
Fix bugs that were exposed by defining KJDEBUG (thanks @davidcarne for finding this)
v0.5.5 (2015-03-06)
Update bundled C++ libcapnp to v0.5.1.2 security release
v0.5.4 (2015-03-02)
Update bundled C++ libcapnp to v0.5.1.1 security release
Add bootstrap RPC methods
Fix possible segfault when importing multiple schemas
v0.5.3 (2015-02-23)
Fix possible crash due to bad destructor ordering in MessageReader (by @JohnEmhoff)
Default to no longer using cython
v0.5.2 (2015-02-20)
Add read_multiple_bytes/read_multiple_bytes_packed methods
Added Python 3.4 to the travis build matrix
Bump version for bundled C++ libcapnp to v0.5.1
v0.5.1 (2014-12-27)
Remove installation dependency on cython. We now have no dependencies since libcapnp will automatically build as well.
v0.5.0 (2014-12-15)
Timer class capnp.getTimer()
pycapnp is now thread-safe and allows an event loop to be run in each thread
You must destroy and re-create the event loop to get this functionality (see test_threads.py)
Inheritance now works correctly for interfaces (previously inherited methods were inaccessible from pycapnp)
Add ability to import modules with dashes or spaces. Use underscores in place of them
from_bytes with builder=True is no longer zero copy. It never worked correctly, and is much safer now
Add num_first_segment_words argument wherever message creation can occur
Allow restoring a null objectId by passing None to restore
Support ordered dictionary in to_dict
Add ListSchema class and schemas for native types under capnp.types which completes all the Schemas needed to be wrapped. See test_schema.py for examples using it
Add automatic build of C++ libcapnp if it’s not detected on the system. Also add flags –force-bundled-libcapnp and –force-system-libcapnp respectively
v0.4.6 (2014-9-10)
Fix build for new 0.21 release of Cython. 0.21 is now the minimum supported version of Cython.
v0.4.5 (2014-6-26)
Fix to_dict not converting enums to strings
v0.4.4 (2014-04-25)
Fix compilation problem with gcc 4.8
v0.4.3 (2014-02-18)
Fix problem with uninitialized unions in _from_dict
Add accesible version numbers for C++ libcapnp
v0.4.2 (2014-02-13)
Remove onDrained since it was removed upstream
Replace usage of strings as enum type with custom _DynamicEnum class.
Also change Struct.which() method to be a property Struct.which and return an enum type (_DynamicEnumField, which behaves much like _DynamicEnum).
TwoPartyServer.run_forever() now will handle more than 1 simulataneous connection.
Change exception wrapper to detect and raise AttributeError for field lookup exceptions (Fixes problem in Python3.x __dir__)
Allow setting of fields with python dicts.
0.4.1 (2013-12-18)
Remove python 3.2 from travis tests. Python 3.2 still should work fine, but it’s more trouble than it’s worth to write unicode tests that work in both it and Python2.
Fix problems with null characters in Text/Data fields. Fixes #19
0.4.0 (2013-12-12)
Initial working version of RPC
Add get_root_as_any to _MessageReader
Add capnp.pxd for public declarations of cython classes
Fix problems compiling with gcc4.7
v0.3.18 (2013-11-05)
Change naming of ReaderOption parameters to be pep8 compliant
v0.3.17 (2013-11-05)
Add ReaderOptions to read/read_packed/from_bytes
v0.3.16 (2013-10-28)
Add defaults flag to capnp-json. Also remove ‘which’ field
Add capnp-json serializer script. Also fix bugs in from_dict
Fix build for clang/python3. Also remove -fpermissive
Add as_builder method to Struct Reader
Add warning when writing the same message more than once
First working version of capability interfaces
Wrap InterfaceSchema
Fix setting string fields to support all types of strings
Fix changed API for DynamicObject/ObjectPointer
v0.3.15 (2013-09-19)
Add not having installed the C++ libcapnp library to ‘Common Problems’
Add _short_str function for use in capnp_test_pycapnp.py
Add test script for testing with https://github.com/kaos/capnp_test
Add handling of DynamicObject
Fix lists of lists or dicts for from_dict
v0.3.14 (2013-09-04)
Fix problem with to_dict
v0.3.13 (2013-09-04)
Add _DynamicStructBuilder.to_bytes() and .from_bytes()
Change == on StructSchema to return cbool
Add Builder and Reader ABCs for each struct type
v0.3.12 (2013-09-03)
Fix handling of empty path ’’ in load_module
Add from_dict
Fix bug in exception handling for which(). Also standardize exceptions.
Change import hook to require modules to end in ’_capnp’
Add import monkey patch function.
Change naming for functions to conform to PEP 8. Also deprecate old read/write API
Update preferred method for reading/writing messages from files
v0.3.11 (2013-09-01)
Forgot to change project name in setup.py
v0.3.10 (2013-09-01)
Change all references to old project name (change from capnpc-python-cpp to pycapnp)
Change DynamicValue.Reader lists to be returned as _DynamicListReader
Unify setters for DynamicList and DynamicStruct
Add shortcuts for reading from / writing to files. In Python, it doesn’t make much sense to force people to muck around with MessageReaders and MessageBuilders since everything is landing on the heap anyway. Instead, let’s make it easy: MyType.read[Packed]From(file) reads a file and returns a MyType reader. MyType.newMessage() returns a MyType builder representing the root of a new message. You can call this builder’s write[Packed]To(file) method to write it to a file.
Store Builders by value rather than allocate them separately on the heap (matches treatment of Readers). v0.3 fixes the bug that made this not work.
Wrap MessageBuilder::setRoot().
Add tests based on TestAllTypes from the C++ test.capnp. Fix problems uncovered in capnp.pyx.
Implement str and repr for struct and list builders. str uses prettyPrint while repr shows the type name and the low-whitespace stringification. Also implement repr for StructSchema, just because why not?
v0.3.9 (2013-08-30)
Change load to use a global SchemaParser. Make structs settable as field
Add docstrings for new functions and _DynamicResizableListBuilder
v0.3.8 (2013-08-29)
Add initial tests
Add _capnp for original Cython module. Meant for testing.
Lowercase schema so it conforms to member naming conventions
Expose _StructSchema’s raw node
Add some useful _StructSchema, reader, and builder methods
Add full orphan functionality. Also, allow special orphan lists
Finish up adding docstrings to all public classes/methods
v0.3.7 (2013-08-26)
Add a ton of docstrings and add to official docs
Add DynamicOrphan
v0.3.6 (2013-08-26)
Add intersphinx for linking to python docs
Add C++ library version check
v0.3.5 (2013-08-25)
Add handling of constants in schemas
Fix new error with DynamicValue.Builder no longer being copyable
v0.3.4 (2013-08-22)
Fix Void namespace change
Updated capnp schema to conform with new union rules
v0.3.3 (2013-08-22)
Fix for the removal of DynamicUnion from the C++ API
v0.3.2 (2013-08-21)
Add MANIFEST.in to include README
v0.3.1 (2013-08-21)
Update docs with lines about upgrading setuptools
0.3.0 (2013-08-21)
Initial commit of docs
Add querying unnamed enums to structs
0.2.1 (2013-08-13)
Fix enum interface change for benchmark
Random formatting cleanup
Allow import paths in the schema loader
Add travis CI
0.2.0 (2013-08-12)
Initial working version
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distributions
Hashes for pycapnp_async-0.7.0-cp37-cp37m-win_amd64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 36a3c2feb04496e0d7c74b40ed0deaebc932a27979e70271449a013b08a3c31c |
|
MD5 | 9dfe18d8ace9682f15f12e972882bf30 |
|
BLAKE2b-256 | 3a8224ad4799bb0d1d55a4393d2765dab3d62842592a07a3206e68b4369f1c26 |
Hashes for pycapnp_async-0.7.0-cp37-cp37m-win32.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f2247045355df9aa8560e06aa807a0693242432bf25e8bccab96ed139023ddcb |
|
MD5 | 27bed60934328c38ea5642f1dafd29e1 |
|
BLAKE2b-256 | d2af0a8b67c626f6134c5ca129ad351f50382ddd2da666f6dc8f2ad1a811ffce |
Hashes for pycapnp_async-0.7.0-cp37-cp37m-macosx_10_14_x86_64.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 032c1f71158484102a3dbea29654be12dc695421181bdbaf2876728a1e60f00f |
|
MD5 | f48dce318f593d44a35b5260d0bfb827 |
|
BLAKE2b-256 | 703239ed3f00ddc2f4c53c4b910d0c66fe95bcc0e1ff2fc2cbe1c385c8f95ba9 |