Skip to main content

KORE FileFormat — Python

Version 1.7.3 | PyPI | GitHub

World's fastest human-readable columnar format. .kore v3 opens in Notepad AND reads 12x faster than CSV.

Install

pip install kore-fileformat

.kore v3 — One Format, Everything

KORE2 offset=0000000455      ← jump straight to data
# KORE Format v3.0
# Rows: 100,000  Columns: 3
# Compressed: 28,500 bytes (Rust ZSTD/LZ4)
# Schema:
#   price                F64
#   qty                  I64
# Preview (first 5 rows):
#   [price=10.5 | qty=100]
[binary compressed data — 10x smaller than JSON]

Quick Start

import kore_fileformat as kore

# Write — human-readable header + compressed binary
block = kore.DataBlock()
block.add_column('price', kore.DataType.F64, [10.5, 20.0, 30.75])
block.add_column('qty',   kore.DataType.I64, [100,  200,  300])
kore.write_file('data.kore', block)

# Read — returns array.array (no Python object overhead)
result = kore.read_file('data.kore')
print(result.num_rows, result.num_columns)

# Inspect without loading data
kore.inspect_kore('data.kore')           # prints header
header = kore.kore_header('data.kore')   # returns string
stats  = kore.kore_stats('data.kore')    # {'total_kb', 'overhead_pct', ...}

CLI (installed automatically)

kore inspect data.kore            # show schema + preview (no full read)
kore stats   data.kore            # file size breakdown
kore convert src.kore dst.hkore   # convert formats
kore bench                        # write/read speed benchmark
kore version                      # version string

Benchmark

Format Read Write Size
KORE .kore 79 ns/row 255 ns/row 305 KB
KORE .hkore 28 ns/row 154 ns/row 3,126 KB
JSON 1,096 ns/row 9,576 ns/row 6,786 KB
CSV 1,252 ns/row 3,447 ns/row 3,368 KB
SQLite 1,258 ns/row 1,256 ns/row 3,180 KB

(100K rows × 4 cols, warm OS cache)

API Reference

Function Description
write_file(path, block) Write .kore v3 (compressed + human header)
read_file(path) Read .kore → DataBlock (returns array.array)
write_hybrid(path, block) Write .hkore (raw binary, 28 ns/row read)
read_hybrid(path) Read .hkore → DataBlock
inspect_kore(path) Print text header (no data load)
kore_header(path) Get text header as string
kore_stats(path) Dict: total_kb, header_kb, binary_kb, overhead_pct
DataBlock() Create empty block
block.add_column(name, dtype, data) Add column
block.get_column(name) Get column by name

Data Types

kore.DataType.F64   # 64-bit float
kore.DataType.I64   # 64-bit integer
kore.DataType.STR   # UTF-8 string
kore.DataType.BOOL  # Boolean

Install

pip install kore-fileformat==1.7.3

Or from source (requires Rust):

cargo build --release -p kore-ffi
pip install -e .

Quick Start

import kore_fileformat as kore

# --- Write ---
block = kore.DataBlock()
block.add_column('price',    kore.DataType.F64, [10.5, 20.0, 30.75])
block.add_column('quantity', kore.DataType.I64, [100,  200,  300])
kore.write_file('data.kore', block)

# --- Read ---
result = kore.read_file('data.kore')
print(f'{result.num_rows} rows, {result.num_columns} columns')
price_col = result.get_column('price')
print(price_col.data)   # [10.5, 20.0, 30.75]

# --- CRC32 checksum ---
checksum = kore.crc32(b'hello kore')
print(f'crc32 = {checksum:#010x}')   # 0x4b029b4b

API Reference

Function Description
write_file(path, block) Write DataBlock to .kore binary
read_file(path) Read .kore binary into DataBlock
crc32(data: bytes) CRC32 checksum
DataBlock() Create empty block
block.add_column(name, dtype, data) Add a column
block.get_column(name) Get column by name

Data Types

kore.DataType.I64       # 64-bit integer
kore.DataType.F64       # 64-bit float
kore.DataType.STR       # UTF-8 string
kore.DataType.STR_DICT  # Dictionary-encoded string (compressed)
kore.DataType.BOOL      # Boolean

Run Tests

python -m pytest test_kore_fileformat.py -v
python test_phase3.py

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

kore_fileformat-1.7.3.tar.gz (35.2 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

kore_fileformat-1.7.3-py3-none-any.whl (31.8 kB view details)

Uploaded Python 3

File details

Details for the file kore_fileformat-1.7.3.tar.gz.

File metadata

  • Download URL: kore_fileformat-1.7.3.tar.gz
  • Upload date:
  • Size: 35.2 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.12.13

File hashes

Hashes for kore_fileformat-1.7.3.tar.gz
Algorithm Hash digest
SHA256 641804924226ba7cbad43bd5c2fe0252ef491b8489de665331d5b83e33a37652
MD5 acd3ae03c96a48980b28b639ad930415
BLAKE2b-256 85aae542a2fcae85f61a1301cfacfde4fc5898c34e98472035b0c6b824a9d994

See more details on using hashes here.

File details

Details for the file kore_fileformat-1.7.3-py3-none-any.whl.

File metadata

File hashes

Hashes for kore_fileformat-1.7.3-py3-none-any.whl
Algorithm Hash digest
SHA256 79b448d85184423b0cede0cd34fcdf9784eb73a56e58c71768324959bfa6fd4a
MD5 9cdf65499c9b72cbe44cf62c8c9820da
BLAKE2b-256 048390e957132d6533ec7890a573fdb6da07c80a0d07f07ebc09ba89a291e678

See more details on using hashes here.

Release history Release notifications | RSS feed

1.8.0

2 files

This release

1.7.3 This release

2 files

1.6.7

2 files

1.6.6

2 files

1.6.5

2 files

1.6.0

2 files

1.5.1

3 files

1.5.0

3 files

1.3.3

1 file

1.3.2

3 files

1.3.1

5 files

1.3.0

1 file

1.2.9

10 files

1.2.8

3 files

1.2.3

3 files

1.2.2

3 files

1.2.1

3 files

1.2.0

3 files

1.1.6

3 files

1.1.5

3 files

1.1.4

3 files

1.1.3

1 file

1.1.2

1 file

1.1.1

1 file

1.1.0

1 file

0.1.0

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page