Skip to main content

Enterprise serialization formats for xwsystem - Parquet, Avro, HDF5, NetCDF, and 14 more heavyweight formats

Project description

xwformats

Many serialization formats, one package. Schema (Protobuf, Parquet, Thrift, …), scientific (HDF5, Feather, Zarr, …), database (LMDB, LevelDB, …), plus binary and text. Same converter style as xwsystem; formats register on import. Pick lite, lazy, full, or dev extras to match how you deploy.

Company: eXonware.com · Author: eXonware Backend Team · Email: connect@exonware.com

Status Python License


Install

Install What you get When to use
pip install exonware-xwformats Lite - exonware-xwsystem + exonware-xwlazy Small install; add formats via [full] or lazy hooks.
pip install exonware-xwformats[lazy] Lazy - xwsystem[lazy] + lazy extras Matches lazy-optional stack in docs.
pip install exonware-xwformats[full] Full - common format deps installed up front Production or CI; on Windows, LevelDB/RocksDB wheels may be missing (see docs/REF_54_BENCH.md).
pip install exonware-xwformats[dev] Dev - pytest, black, mypy, … Contributors

Base install always pulls exonware-xwsystem and exonware-xwlazy. [lazy], [dev], and [full] are extras.


Quick start

from exonware.xwformats import XWFormats

xf = XWFormats()
print(xf.list_formats())                       # All registered formats
data = xf.convert(json_bytes, "json", "yaml")  # Bidirectional conversion
ser = xf.get_serializer("parquet")
ser.encode(value)
ser.decode(data)

Importing xwformats registers its formats with xwsystem; use xwsystem's codec/facade for detection by file extension. See REF_14_DX and REF_15_API.


What you get

Area What's in it
Schema Protobuf, Parquet, Thrift, ORC, Cap'n Proto, FlatBuffers, Arrow. Avro is optional (may be off default import on some platforms - see __init__.py and docs/_archive/KNOWN_ISSUES.md).
Scientific HDF5, Feather, Zarr, NetCDF, MAT.
Database LMDB, LevelDB, RocksDB, GraphDB.
Binary / text BSON, UBJSON; CSV, YAML, TOML, XML, RON.
Integration Same converter/facade as xwsystem; codec registry updated on import.

Lite = few deps. Lazy = optional format wheels on first use. Full = common optionals pre-installed. Platform notes (e.g. RocksDB on Windows): docs/logs/setup/. Known issues: docs/_archive/KNOWN_ISSUES.md and REF_22_PROJECT.md.


Ecosystem functional contributions

xwformats extends format coverage; sibling XW libs provide runtime loading strategy, registration, and downstream consumption layers. You can use xwformats standalone for broad serializer coverage and conversion tasks. Integrating with additional XW libraries is optional and mainly useful for enterprise and mission-critical pipelines that need unified runtime, storage, and query infrastructure.

Supporting XW lib What it provides to xwformats Functional requirement it satisfies
XWSystem Core codec registry/facade and serializer contracts that xwformats registers into. One unified serialization API despite many optional format backends.
XWLazy On-demand dependency loading/install behavior for optional format stacks. Practical deployment of many formats without forcing heavy installs.
XWData Higher-level data transformation workflows that consume xwformats codecs. Real pipeline usage beyond encode/decode-only primitives.
XWStorage Persistence engine integrations that use specialized format serializers. Storage interoperability for scientific/schema/database-oriented payloads.
XWJSON Binary JSON and local data-engine flows that can interoperate with format conversion paths. Efficient local format conversion and transition workflows.
XWQuery Query pipelines over data loaded via format serializers. End-to-end analytics/transform use cases after format normalization.

Competitive edge: instead of isolated serializer plugins, xwformats plugs into a shared runtime and data stack so advanced formats are immediately usable in storage/query/application workflows.


Docs and tests

Aligned with project REFs and docs/GUIDE_01_USAGE.md (see GUIDE_63_README in the monorepo).


License and links

MIT - see LICENSE.

Async Support

  • xwformats is primarily synchronous in its current implementation.
  • Source validation: 0 async def definitions and 0 await usages under src/.
  • This module still composes with async-capable xw libraries at integration boundaries when needed.

Version: 0.9.0.26 | Updated: 04-Apr-2026

Built with ❤️ by eXonware.com - Revolutionizing Python Development Since 2025

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

exonware_xwformats-0.9.0.26.tar.gz (59.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

exonware_xwformats-0.9.0.26-py3-none-any.whl (69.6 kB view details)

Uploaded Python 3

File details

Details for the file exonware_xwformats-0.9.0.26.tar.gz.

File metadata

  • Download URL: exonware_xwformats-0.9.0.26.tar.gz
  • Upload date:
  • Size: 59.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.13

File hashes

Hashes for exonware_xwformats-0.9.0.26.tar.gz
Algorithm Hash digest
SHA256 196496c20f44b81757948e0e921e793ea1d0f906c885dc751e16a71a2ec68ed9
MD5 2f9756d2d94d512b0b1567374b94cbf5
BLAKE2b-256 25f8d60e3a7ca4019043759bc6a8906f01918f09b7a17c70d07aceedc311facd

See more details on using hashes here.

File details

Details for the file exonware_xwformats-0.9.0.26-py3-none-any.whl.

File metadata

File hashes

Hashes for exonware_xwformats-0.9.0.26-py3-none-any.whl
Algorithm Hash digest
SHA256 c24c7df6e79e96714cd5f7c34f6521c89975195156a536f00af76b90385bac82
MD5 5c067acae01783031020ba5193c8f7fc
BLAKE2b-256 79ef58f5eaff1cb81ea9041f1b585335ebe8df1d0bbdf69cd6992ad69297ee73

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page