dynojson 0.2.0

Marshall/unmarshall JSON to/from DynamoDB JSON format — fast Python library backed by Rust

dynojson

Marshall / unmarshall JSON to and from DynamoDB JSON format — a fast Python library backed by Rust.

Convert between regular JSON and DynamoDB JSON format on the terminal or in your Python code. Powered by a Rust core via PyO3 for maximum performance.

• 🔥 Works on all platforms (Linux, macOS, Windows)
• 📄 Convert a file from a given file path
• ✏️ Convert a JSON string directly
• ⛓ Read from stdin — pipe in JSON from another command
• 🍰 Extract and convert only a subset of the JSON
• 🤝 Output can be piped or redirected to other commands
• 🧰 Integrate into your workflow with AWS DynamoDB CLI

Installation

pip install dynojson

Python API

All functions accept JSON strings or Python dicts/lists directly. The return type matches the input type.

from dynojson import marshall, unmarshall, get_property

# ── With dicts (recommended) ──────────────────────────────────────

result = marshall({"name": "Alice", "age": 30})
# {"name": {"S": "Alice"}, "age": {"N": "30"}}  ← returns a dict

result = unmarshall({"name": {"S": "Alice"}, "age": {"N": "30"}})
# {"name": "Alice", "age": 30}  ← returns a dict

# ── With strings (also supported) ─────────────────────────────────

result = marshall('{"name": "Alice", "age": 30}')
# '{"name":{"S":"Alice"},"age":{"N":"30"}}'  ← returns a string

result = unmarshall('{"name":{"S":"Alice"},"age":{"N":"30"}}')
# '{"name":"Alice","age":30}'  ← returns a string

# ── Extract a property ────────────────────────────────────────────

subset = get_property({"Items": [{"type": {"S": "fruit"}}]}, "Items")
result = unmarshall(subset)
# [{"type": "fruit"}]

File & Directory API

Process files and directories directly — Rust handles all I/O, skipping Python-Rust FFI overhead for large payloads.

from dynojson import marshall_file, unmarshall_file, marshall_dir, unmarshall_dir

# ── Single file ─────────────────────────────────────────────────

# File → dict (no output_path)
result = marshall_file("data.json")

# File → file (with output_path)
marshall_file("data.json", "output.json")

# With property extraction
result = unmarshall_file("scan.json", get="Items")

# ── Directory batch processing ──────────────────────────────────

# Mirror mode — one output file per input file
count = marshall_dir("input/", "output/")

# Merge mode — all results into a single JSON array
count = unmarshall_dir("input/", "merged.json", merge=True)

# With property extraction (e.g. DynamoDB scan exports)
count = unmarshall_dir("scans/", "output/", get="Items")

CLI Usage

dynojson <command> [options] <json>

Commands:
  dynojson unmarshall, u    Convert DynamoDB JSON to regular JSON
  dynojson marshall, m      Convert regular JSON to DynamoDB JSON

Options:
  -g <path>                 Extract a property before converting (dot-separated, supports *)
  --version                 Show version
  --help                    Show help

Unmarshall (DynamoDB JSON → regular JSON)

# From a JSON string
$ dynojson u '{"name":{"S":"Alice"},"age":{"N":"30"}}'
{"name":"Alice","age":30}

# From a file
$ dynojson u data.json

# From stdin
$ cat data.json | dynojson u -

# From AWS CLI
$ aws dynamodb get-item --table-name users --key '{"id":{"S":"1"}}' | dynojson u -g "Item" -

Marshall (regular JSON → DynamoDB JSON)

# From a JSON string
$ dynojson m '{"name":"Alice","age":30}'
{"name":{"S":"Alice"},"age":{"N":"30"}}

# From a file
$ dynojson m data.json

# From stdin
$ echo '{"name":"Alice"}' | dynojson m -

Extract a subset of JSON with -g

Use dot-notation to select a property. Supports numeric array indices and * to expand all array items.

# Get a specific property
$ dynojson m -g "fruits.0.benefits" food.json

# Expand all items
$ dynojson m -g "fruits.*.name" food.json

# With AWS CLI scan output
$ aws dynamodb scan --table-name food | dynojson u -g "Items" -

DynamoDB type mapping

JSON type	DynamoDB descriptor
String	{"S": "…"}
Number	{"N": "…"}
Boolean	{"BOOL": …}
Null	{"NULL": true}
Array	{"L": […]}
Object	{"M": {…}}

String sets (SS), number sets (NS), and binary types (B, BS) are also supported during unmarshalling.

Benchmarks

The dict path is faster than the string path because it skips two JSON serialization round-trips (json.dumps + json.loads on the Python side).

Payload	Marshall speedup	Unmarshall speedup
Small (1 item)	1.28x	1.15x
Medium (100 items)	1.20x	1.22x
Large (10,000 items)	1.14x	1.22x

Run the benchmark yourself:

maturin develop --release
python benchmarks/bench_marshall.py

Development

Prerequisites

• Rust (stable)
• Python 3.10+
• uv (recommended) or pip
• maturin

Setup

# Clone the repo
git clone https://github.com/cykruss/dynojson.git
cd dynojson

# Create a virtual environment
uv venv
source .venv/bin/activate

# Install build and test dependencies
uv pip install maturin pytest

# Build and install in development mode
maturin develop

# Run Rust tests
cargo test --lib

# Run Python tests
pytest tests/ -v

Project structure

dynojson/
├── Cargo.toml              # Rust package manifest
├── pyproject.toml           # Python package manifest (maturin build)
├── README.md
├── LICENSE
├── CONTRIBUTING.md
├── src/
│   ├── lib.rs               # PyO3 module + re-exports
│   ├── error.rs             # Custom error types (thiserror)
│   ├── marshall.rs          # Regular JSON → DynamoDB JSON
│   ├── unmarshall.rs        # DynamoDB JSON → regular JSON
│   └── property.rs          # Dot-path property extraction
├── python/
│   └── dynojson/
│       ├── __init__.py      # Public Python API
│       ├── _dynojson.pyi    # Type stubs
│       └── cli.py           # CLI entry point
├── benchmarks/
│   └── bench_marshall.py   # String vs dict path benchmark
└── tests/
    ├── test_marshall.py
    ├── test_unmarshall.py
    ├── test_property.py
    ├── test_file_io.py
    └── test_cli.py

Acknowledgements

This project was inspired by ddbjson by Alexandre Duarte, an excellent Node.js CLI tool for converting DynamoDB JSON. dynojson is a ground-up Rust rewrite with Python bindings, building on the same ideas.

License

MIT