PyCyBase

gitlab

GitHub Actions: ⛔ temporarily disabled — kept disabled until the GitHub Actions issue is resolved. See .github/workflows/README.md. CI runs on the GitLab mirror pipeline while disabled.

Common C / Cython dual interfaces for Python HFT Projects.

Overview

Low-level allocator infrastructure shared across HFT projects:

  • Allocator Protocol — Pluggable memory allocation with heap, shared-memory (SHM), and raw malloc backends. Ref-counting, vigilant mode, thread-safety controls.
  • Heap Allocator — In-process allocator with free-list reuse and auto-growing pages.
  • SHM Allocator — POSIX shared-memory page allocator with fixed-address virtual region mapping for cross-process pointer stability.
  • ByteMap — Fast xxHash3-backed hash maps for byte-string keys (ByteMap, ByteMapEx, bound variants).
  • Intern String — FNV-1a-hashed string interning with SHM-backed cross-process pools. 1.7–2.2× speedup over Python str.

Quick Start

from cbase.allocator_protocol import AllocatorProtocol, AP_SHARED

with AP_SHARED:
    alloc = AllocatorProtocol(1024)
    alloc.buf[0] = b'x'

from cbase.intern_string import POOL

aapl = POOL.istr("AAPL")
assert aapl is POOL.istr("AAPL")  # Identity — string deduplication

from cbase.bytemap import ByteMap

bm = ByteMap()
bm[b"ticker"] = b"AAPL"
assert bm[b"ticker"] == b"AAPL"

Build & Install

PyCyBase is a Cython extension package — it cannot be used with pip install -e . (editable mode breaks Cython .pxd resolution for downstream projects).

POSIX (Linux / macOS)

# Build extensions in-place, then install
./build.sh -i

# Or equivalently:
make build && pip install -U . --no-build-isolation

# Or step by step:
python setup.py build_ext --inplace --verbose --force
pip install -U . --no-build-isolation

Build script reference:

Command Effect
./build.sh Clean + build in-place
./build.sh -i Clean + build + pip install .
./build.sh -r Clean + build + force-reinstall
./build.sh -c Clean only (no build)
./build.sh -a Deep clean (remove .c and .so files)
./build.sh -l List all compile-time macros
./build.sh -v /path/to/venv Use a specific venv
make or make build Clean + build
make install Build + pip install .
make reinstall Build + force-reinstall
make list-args List compile-time macros

Windows (NT)

# Local build
.\build.ps1 -VenvPath "C:\Users\...\venv_313"

# Remote build (from Linux host):
python nt_build.py

Compile-Time Macros

Override any macro at build time via environment variable:

DEBUG=1 ./build.sh         # Enable debug mode
DEBUG=1 make build          # Same via Makefile

List all available macros and defaults:

./build.sh -l               # Auto-generates macros.json via probe.py
make list-args
python probe.py             # Run the probe directly

Key macros (see macros.json for full list):

Macro Default Description
AP_ALLOC_VIGILANT 1 Enable bounds/canary checks
AP_ALLOC_WITH_LOCK 1 Thread-safety via pthread mutex
AP_ALLOC_WITH_SHM 0 Default to SHM allocator
AP_ALLOC_WITH_FREELIST 1 Free-list reuse
AP_HEAP_AUTOPAGE_CAPACITY 64 KiB Default heap page size
AP_HEAP_AUTOPAGE_CAPACITY_MAX 16 MiB Max heap page size
AP_HEAP_EXACT_BIN_COUNT 8192 Exact 8-byte-granular free-list bins
AP_HEAP_LARGE_BIN_COUNT 9 Pow2-class free-list bins for large blocks
AP_HEAP_PAGE_EXTEND_MAX 128 MiB Max auto-page extension on bin miss
AP_HEAP_EXACT_BIN_PROBE_COUNT 2 Larger exact bins to probe before extending
AP_SHM_AUTOPAGE_CAPACITY 64 KiB Default SHM page size
AP_SHM_ALLOCATOR_DEFAULT_REGION_SIZE 128 GiB SHM virtual region
AP_SHM_EXACT_BIN_COUNT 8192 Exact 8-byte-granular free-list bins
AP_SHM_PAGE_EXTEND_MAX 128 MiB Max auto-page extension on bin miss

Using get_include() in Downstream Projects

from setuptools import setup, Extension
from Cython.Build import cythonize
import cbase

ext = Extension(
    "my_module",
    sources=["my_module.pyx"],
    include_dirs=cbase.get_include(),
)
setup(ext_modules=cythonize([ext]))

Modules

Module Description
cbase.env EnvConfigContext — context manager for temporary config changes
cbase.allocator_protocol AllocatorProtocol, AllocatorConfigContext, AP sentinels
cbase.allocator_protocol.c_heap_allocator In-process heap allocator with free-list
cbase.allocator_protocol.c_shm_allocator POSIX shared-memory page allocator
cbase.bytemap.c_bytemap xxHash3-backed hash maps
cbase.intern_string.c_intern_string FNV-1a-hashed string interning
cbase.backports.pylong Python int ↔ 128-bit integer (uint128_t/int128_t) backport
cbase.backports.pydict PyDict_Pop backport for pre-3.13 Python

Documentation

pip install sphinx furo
cd docs
sphinx-build -M html . _build
# Or with the project venv:
~/Projects/venv_313/bin/sphinx-build -M html . _build

Open docs/_build/html/index.html.

Online: https://bolunhan.github.io/PyCyBase/

License

Proprietary.

Project links

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.2.9)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for BolunHan from gravatar.com BolunHan

Credits

Author: Han Bolun

Requires

Python >=3.12

Provides Extra

docs

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

pycybase-0.2.7-cp315-cp315t-win_amd64.whl (4.1 MB view details)

Uploaded CPython 3.15tWindows x86-64

pycybase-0.2.7-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (8.6 MB view details)

Uploaded CPython 3.15tmanylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

pycybase-0.2.7-cp315-cp315-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.15Windows x86-64

pycybase-0.2.7-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (8.5 MB view details)

Uploaded CPython 3.15manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

pycybase-0.2.7-cp314-cp314t-win_amd64.whl (4.1 MB view details)

Uploaded CPython 3.14tWindows x86-64

pycybase-0.2.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (8.6 MB view details)

Uploaded CPython 3.14tmanylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

pycybase-0.2.7-cp314-cp314-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.14Windows x86-64

pycybase-0.2.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (8.5 MB view details)

Uploaded CPython 3.14manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

pycybase-0.2.7-cp313-cp313-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.13Windows x86-64

pycybase-0.2.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (8.5 MB view details)

Uploaded CPython 3.13manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

pycybase-0.2.7-cp312-cp312-win_amd64.whl (4.0 MB view details)

Uploaded CPython 3.12Windows x86-64

pycybase-0.2.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (8.5 MB view details)

Uploaded CPython 3.12manylinux: glibc 2.17+ x86-64manylinux: glibc 2.28+ x86-64

File details

Details for the file pycybase-0.2.7-cp315-cp315t-win_amd64.whl.

File metadata

  • Download URL: pycybase-0.2.7-cp315-cp315t-win_amd64.whl
  • Upload date:
  • Size: 4.1 MB
  • Tags: CPython 3.15t, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.2

File hashes

Hashes for pycybase-0.2.7-cp315-cp315t-win_amd64.whl
Algorithm Hash digest
SHA256 b7300dc225d3a143218c7549769663e0dda6663452ae41b33cfd290f9031b38f
MD5 a2ed989fe8756890ee9d7f621f47f70a
BLAKE2b-256 f06f1f3aa923ab9eaaed0f6ca6c591c1f78463fbd855e89eec62ab52fbca744b

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pycybase-0.2.7-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 15b039374470bfc6af767a6a1fd5562b0a92a2ab1ad634d8f58d96fc7cf64f3c
MD5 c5d4cc737c838fd58a7c80ef2e8f451d
BLAKE2b-256 e6ef7836605cf34173623e6c49d4803c2a2eb515befe9e85021a5c29f56b9276

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp315-cp315-win_amd64.whl.

File metadata

  • Download URL: pycybase-0.2.7-cp315-cp315-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.15, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.2

File hashes

Hashes for pycybase-0.2.7-cp315-cp315-win_amd64.whl
Algorithm Hash digest
SHA256 1b176f955c4ed4ba83ca8e5cbe6fa172a98fb11af5db66819907e8acc6da80c1
MD5 6097054574a24a96a26e41c827187128
BLAKE2b-256 20c3cedf1ccda8eb4747ca6f96a9296907c70335f90699411623047babe4fee2

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pycybase-0.2.7-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 17ec7e92aa8abce14363e974f8eaa94004e901804978e3017d0e4838025e1bac
MD5 7be5d51d459d59dc76b413a8b33bc516
BLAKE2b-256 e1d669b88c55bb07da573b296ef72c4b0ba5eecec5a194e3e2b8b4170307eddb

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp314-cp314t-win_amd64.whl.

File metadata

  • Download URL: pycybase-0.2.7-cp314-cp314t-win_amd64.whl
  • Upload date:
  • Size: 4.1 MB
  • Tags: CPython 3.14t, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.2

File hashes

Hashes for pycybase-0.2.7-cp314-cp314t-win_amd64.whl
Algorithm Hash digest
SHA256 e8799b6708e36fc27ed068ef172f5cc4747bcf4def18fcfd35e6010b8df2bc84
MD5 e6d28b4f6c2624b6e56b2466fed4f882
BLAKE2b-256 0c963d71ade610202a9494f7d2f8f39c7a4125abdbb332d666d631262053d1cb

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pycybase-0.2.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 09351d1cea4cc4122121bff9046031af9d70b71b6310c5451463e30a3119dbfe
MD5 82106c7158350238fc2a98028ec50b41
BLAKE2b-256 67e1532184b948ad26c83eae30a59da2cc9a76aea294fb5010cac052e200238f

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp314-cp314-win_amd64.whl.

File metadata

  • Download URL: pycybase-0.2.7-cp314-cp314-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.14, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.2

File hashes

Hashes for pycybase-0.2.7-cp314-cp314-win_amd64.whl
Algorithm Hash digest
SHA256 5d14c69a735a54e60bf4c772df94fce86af7b6efada6d2f56cd269db6602affd
MD5 10a3ef170891f53fa7a8813032327e98
BLAKE2b-256 479783a62e6f8b15e682da1bc8c5f4aa9d376683925402d78ba6ff235b8a3137

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pycybase-0.2.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 b0656e8be5e41d22aad4e7a38f10aa9d652775d47e6925d884ba283b32ab17be
MD5 826008dc78e598f7d33d0fd8a6e8bde4
BLAKE2b-256 686d3ef90015147ec2d3686f845825c8ffa2c8afbb0773d77b1b5b0aeca98d83

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp313-cp313-win_amd64.whl.

File metadata

  • Download URL: pycybase-0.2.7-cp313-cp313-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.13, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.2

File hashes

Hashes for pycybase-0.2.7-cp313-cp313-win_amd64.whl
Algorithm Hash digest
SHA256 2fdd95442c33b9ab2e6ddad1315e5e35079c8ce35191f80700fad97893007241
MD5 1f8c5661e7e12ccb874c2031f7140e54
BLAKE2b-256 c7e92f79120647bf0503bcdcfe181d1a30d1bd87e84d3b2798e1a59f779d23ed

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pycybase-0.2.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 da898b3feece8e71c56c0acb8f22c966f3b520871885334dceec9c8914fb35ad
MD5 25d36e29750cbe5f655f470c341e83fb
BLAKE2b-256 5b4abfea9ade8a30bd1fbd02d1ffc04cb7258e045e48ec38d192b069937d3031

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp312-cp312-win_amd64.whl.

File metadata

  • Download URL: pycybase-0.2.7-cp312-cp312-win_amd64.whl
  • Upload date:
  • Size: 4.0 MB
  • Tags: CPython 3.12, Windows x86-64
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.13.2

File hashes

Hashes for pycybase-0.2.7-cp312-cp312-win_amd64.whl
Algorithm Hash digest
SHA256 717e2e0dd21a1606fffd574ff23593d82c8ae41c2d128725178123308601f496
MD5 d5f53cd282961786ffd4b9a5bbc7a3ea
BLAKE2b-256 97cd965599ff1d4cd96cf9d1c9ba099786d61e59ef18f463d879c4073b33d270

See more details on using hashes here.

File details

Details for the file pycybase-0.2.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for pycybase-0.2.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 1da6ce9e8131ea9a1e34b3171601d0a8029df06aa9f9d662e9b077f1c597b7ff
MD5 9f7af7428ca34c09565d9f8998739075
BLAKE2b-256 41a1bb5f73c55472a776260d3fcb91aa16ab7451381992a63e27cd553ddfd9bf

See more details on using hashes here.

Release history Release notifications | RSS feed

0.2.9

12 files

0.2.8

12 files

This release

0.2.7 This release

12 files

0.2.6

12 files

0.2.5

12 files

0.2.2

12 files

0.2.1

12 files

0.2.0

12 files

0.1.11.post1

12 files

0.1.11

12 files

0.1.10

12 files

0.1.9

8 files

0.1.8.post5

8 files

0.1.8.post3

4 files

0.1.8.post2

4 files

0.1.8.post1

4 files

0.1.8

4 files

0.1.7.post4

4 files

0.1.7.post2

3 files

0.1.4.post2

4 files

0.1.4.post1

4 files

0.1.3.post1

4 files

0.1.0

4 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page