KORE FileFormat — Python
World's fastest human-readable columnar format. .kore v3 opens in Notepad AND reads 12x faster than CSV.
Install
pip install kore-fileformat
.kore v3 — One Format, Everything
KORE2 offset=0000000455 ← jump straight to data
# KORE Format v3.0
# Rows: 100,000 Columns: 3
# Compressed: 28,500 bytes (Rust ZSTD/LZ4)
# Schema:
# price F64
# qty I64
# Preview (first 5 rows):
# [price=10.5 | qty=100]
[binary compressed data — 10x smaller than JSON]
Quick Start
import kore_fileformat as kore
# Write — human-readable header + compressed binary
block = kore.DataBlock()
block.add_column('price', kore.DataType.F64, [10.5, 20.0, 30.75])
block.add_column('qty', kore.DataType.I64, [100, 200, 300])
kore.write_file('data.kore', block)
# Read — returns array.array (no Python object overhead)
result = kore.read_file('data.kore')
print(result.num_rows, result.num_columns)
# Inspect without loading data
kore.inspect_kore('data.kore') # prints header
header = kore.kore_header('data.kore') # returns string
stats = kore.kore_stats('data.kore') # {'total_kb', 'overhead_pct', ...}
CLI (installed automatically)
kore inspect data.kore # show schema + preview (no full read)
kore stats data.kore # file size breakdown
kore convert src.kore dst.hkore # convert formats
kore bench # write/read speed benchmark
kore version # version string
Benchmark
| Format | Read | Write | Size |
|---|---|---|---|
| KORE .kore | 79 ns/row | 255 ns/row | 305 KB |
| KORE .hkore | 28 ns/row | 154 ns/row | 3,126 KB |
| JSON | 1,096 ns/row | 9,576 ns/row | 6,786 KB |
| CSV | 1,252 ns/row | 3,447 ns/row | 3,368 KB |
| SQLite | 1,258 ns/row | 1,256 ns/row | 3,180 KB |
(100K rows × 4 cols, warm OS cache)
API Reference
| Function | Description |
|---|---|
write_file(path, block) |
Write .kore v3 (compressed + human header) |
read_file(path) |
Read .kore → DataBlock (returns array.array) |
write_hybrid(path, block) |
Write .hkore (raw binary, 28 ns/row read) |
read_hybrid(path) |
Read .hkore → DataBlock |
inspect_kore(path) |
Print text header (no data load) |
kore_header(path) |
Get text header as string |
kore_stats(path) |
Dict: total_kb, header_kb, binary_kb, overhead_pct |
DataBlock() |
Create empty block |
block.add_column(name, dtype, data) |
Add column |
block.get_column(name) |
Get column by name |
Data Types
kore.DataType.F64 # 64-bit float
kore.DataType.I64 # 64-bit integer
kore.DataType.STR # UTF-8 string
kore.DataType.BOOL # Boolean
Install
pip install kore-fileformat==1.7.3
Or from source (requires Rust):
cargo build --release -p kore-ffi
pip install -e .
Quick Start
import kore_fileformat as kore
# --- Write ---
block = kore.DataBlock()
block.add_column('price', kore.DataType.F64, [10.5, 20.0, 30.75])
block.add_column('quantity', kore.DataType.I64, [100, 200, 300])
kore.write_file('data.kore', block)
# --- Read ---
result = kore.read_file('data.kore')
print(f'{result.num_rows} rows, {result.num_columns} columns')
price_col = result.get_column('price')
print(price_col.data) # [10.5, 20.0, 30.75]
# --- CRC32 checksum ---
checksum = kore.crc32(b'hello kore')
print(f'crc32 = {checksum:#010x}') # 0x4b029b4b
API Reference
| Function | Description |
|---|---|
write_file(path, block) |
Write DataBlock to .kore binary |
read_file(path) |
Read .kore binary into DataBlock |
crc32(data: bytes) |
CRC32 checksum |
DataBlock() |
Create empty block |
block.add_column(name, dtype, data) |
Add a column |
block.get_column(name) |
Get column by name |
Data Types
kore.DataType.I64 # 64-bit integer
kore.DataType.F64 # 64-bit float
kore.DataType.STR # UTF-8 string
kore.DataType.STR_DICT # Dictionary-encoded string (compressed)
kore.DataType.BOOL # Boolean
Run Tests
python -m pytest test_kore_fileformat.py -v
python test_phase3.py
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
kore_fileformat-1.7.3.tar.gz
(35.2 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file kore_fileformat-1.7.3.tar.gz.
File metadata
- Download URL: kore_fileformat-1.7.3.tar.gz
- Upload date:
- Size: 35.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.12.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
641804924226ba7cbad43bd5c2fe0252ef491b8489de665331d5b83e33a37652
|
|
| MD5 |
acd3ae03c96a48980b28b639ad930415
|
|
| BLAKE2b-256 |
85aae542a2fcae85f61a1301cfacfde4fc5898c34e98472035b0c6b824a9d994
|
File details
Details for the file kore_fileformat-1.7.3-py3-none-any.whl.
File metadata
- Download URL: kore_fileformat-1.7.3-py3-none-any.whl
- Upload date:
- Size: 31.8 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via:
twine/7.0.0 CPython/3.12.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
79b448d85184423b0cede0cd34fcdf9784eb73a56e58c71768324959bfa6fd4a
|
|
| MD5 |
9cdf65499c9b72cbe44cf62c8c9820da
|
|
| BLAKE2b-256 |
048390e957132d6533ec7890a573fdb6da07c80a0d07f07ebc09ba89a291e678
|