kycli 0.4.0

**kycli** is a high-performance Python CLI toolkit built with Cython for speed.

kycli — The Microsecond-Fast Key-Value Toolkit

kycli is a high-performance, developer-first key-value storage engine. It bridges the gap between the simplicity of a flat-file database and the blazing speed of in-memory caches like Redis, all while remaining completely serverless and lightweight.

Built with Cython and linked directly to the Raw SQLite C API (libsqlite3), kycli is optimized for local development, CLI productivity, and high-throughput Python backends.

---

⚡ Performance: Real-World Stats

kycli is designed to be the fastest local storage option available for Python. By bypassing standard abstraction layers and moving critical logic to C, we achieve microsecond-level latency.

Benchmark Results (Average of 1,000 calls)

Operation	Implementation	Avg Latency	vs. Standard Python
L1 Cache Hit	Cython LRU	1.5 µs	Near-In-Memory
Key Retrieval (Get)	Direct C API	2.8 µs	150x Faster
Batch Save	Atomic C-Loop	15 µs / item	Extreme Throughput
History Lookup	Indexed C API	5.0 µs	Instant Auditing

Why so fast? Standard Python storage tools use network sockets (Redis) or heavy wrappers (SQLAlchemy). kycli uses direct memory pointers to an embedded C engine, removing 99% of the overhead.

Scaling: In a benchmark of 10,000 class records with Pydantic validation, kycli maintains a sub-2ms write latency and microsecond-fast reads. See PERFORMANCE.md for full details.

---

🚀 Installation

Install the latest version from PyPI:

pip install kycli

Validate The Install

Run the end-to-end command matrix from the repo root:

bash scripts/validate_install.sh

The validator auto-resolves a usable Python executable, prefers the active python3 runtime, and runs stateful sections in isolated home directories so queue, profile, and policy checks do not interfere with each other.

---

💻 CLI Command Reference

kycli provides a set of ultra-short commands for maximum terminal productivity.

📂 Workspace Management

Command	Description	Example
kyuse	Switch/Create workspace	kyuse project_alpha
kyws	List workspaces	kyws
kydrop	Delete workspace	kydrop old_ws
kymv	Move key to workspace	kymv api_key prod

📝 Basic Operations

Command	Description	Example
kys	Save Key/Value	kys session "active" --ttl 360 kys secret "pass" --key "k1"
kyg	Get Value	kyg session kyg secret --key "k1"
kypatch	Patch JSON	kypatch user '{"age": 30}'
kyl	List Keys	kyl "user.*"
kyd	Delete Key	kyd host
kypush	Push to List	kypush logs "error"
kyrem	Remove from List	kyrem tags "old"

🧱 Queues & Stacks Operations

Command	Description	Example
kypush	Push to Queue/Stack	kypush "task" --priority 10
kypop	Pop item (Atomic)	kypop
kypeek	Peek next item	kypeek
kycount	Count items	kycount
kyclear	Clear implementation	kyclear
kyws	Create Typed WS	kyws create q --type queue
kyack	Ack leased item	kyack <receipt_id>
kynack	Requeue leased item	kynack <receipt_id> --delay 30s

🔍 Search & Utility

Command	Description	Example
kyg -s	Search Values	kyg -s "db_pass"
kyshell	Open Interactive TUI	kyshell
init	Shell Setup	kycli init
kyfo	Optimize Index	kyfo
kyh	Help	kyh

🛠️ Advanced & Recovery

Command	Description	Example
kye	Export Data	kye backup.json json
kyi	Import Data	kyi data.csv
kyr	Restore Deleted Key	kyr host
kyrt	Point-in-Time Recovery	kyrt "2023-10-27 10:00:00"
kyc	Execute Command	kyc hello
kyco	Compact DB	kyco 7
kyrotate	Rotate Master Key	kyrotate --new-key "newpass" --old-key "oldpass" --backup
kyttl	Workspace TTL policy	kyttl set 1h
kyprofile	Manage profiles	kyprofile save dev
kyacl	Workspace ACL/policy	kyacl readonly on
kyaudit	Export audit history	kyaudit export audit.json json
kystats	Show workspace stats	kystats --json
kybackup	Create/restore backup	kybackup snapshot.db
kymetrics	Local metrics endpoint	kymetrics 8765

---

📂 Workspace Management (Multi-Tenancy)

kycli supports isolated contexts called Workspaces. Each workspace is a separate SQLite database.

kyuse — Switch / Create Workspace

kyuse project_alpha

kyws — List Workspaces

Shows all available workspaces. The active one is marked with ✨.

kyws
# Result: 📂 Workspaces:
#    default
# ✨ project_alpha
#    temp_test

kymv — Move Key

Moves a key (and its value) from the current workspace to another.

kymv my_api_key project_beta
# Result: ✅ Moved 'my_api_key' to 'project_beta'.

kydrop — Delete Workspace

Permanently deletes an entire workspace and its database file.

kydrop temp_test
# Result: ⚠️ DANGER: Are you sure you want to PERMANENTLY delete workspace 'temp_test'? (y/N): y
# Result: ✅ Workspace 'temp_test' deleted.

🧭 Advanced Features

Queue Reliability

• Batch Queue Ops: kypush --file and kypop --n 100 for throughput-oriented flows.
• Delayed Jobs: kypush --delay 30s schedules visibility in queue workspaces.
• Visibility Timeout: kypop --lease 30s with kyack and kynack supports retry flows.
• Workspace TTL Policies: kyttl set/get applies default TTL behavior per workspace.

Usability & DX

• Config Profiles: kyprofile use prod switches saved CLI defaults.
• Output Formatting: --json and --pretty are available on structured read paths.
• Interactive TUI: kyshell provides an interactive terminal workflow on top of the same engine.

Security & Compliance

• Workspace ACLs: kyacl readonly on|off|status supports write protection.
• Workspace Access Keys: kyacl key set|get|clear gates writes with KYCLI_ACCESS_KEY.
• Audit Export: kyaudit export supports JSON or CSV output with optional time filters.

Observability

• Stats Command: kystats exposes workspace type, counts, TTL usage, archive size, and DB size.
• Metrics Endpoint: kymetrics starts a local HTTP endpoint on 127.0.0.1.

Data Management

• Namespace/Prefix Views: kyws view <prefix> filters large keyspaces.
• Backup/Restore: kybackup creates and restores encrypted snapshots.

Queue Reliability

# Push from a file
kypush --file tasks.txt

# Delay visibility for 30 seconds
kypush "email:user_123" --delay 30s

# Lease one item for 30 seconds
kypop --lease 30s --json

# Acknowledge or requeue
kyack <receipt_id>
kynack <receipt_id> --delay 15s

Workspace Policies

# Set a default TTL for all new values in this workspace
kyttl set 1h
kyttl get

# Enable read-only mode
kyacl readonly on

# Protect a workspace with an access key
kyacl key set my-secret
KYCLI_ACCESS_KEY=my-secret kys protected value

Profiles, Stats, and Backups

kyprofile save dev
kyprofile list
kyprofile use dev

kystats --json
kyaudit export audit.json json --since "2026-01-01 00:00:00"
kybackup snapshot.db
kybackup restore snapshot.db
kymetrics 8765

Maintenance & Reliability

• Lock Management: Retry with exponential backoff for "Database is locked" in multi-process scenarios.
• Activity Logs: Background logging via Python logging module.
• Atomic Rename Exports: Write-to-temp-then-rename for export safety.
• Compression: Optional large-value compression for storage efficiency.

kyh — The Help Center

Shows the available commands and basic usage instructions.

kyh
# Or use the -h flag on specific commands

kys <key> <value> [--ttl <sec>] — Save Data

Saves a value to a key.

• Auto-Normalization: Keys are lowercased and trimmed.
• Safety: Asks (y/n) before overwriting an existing key.
• TTL (Time To Live): Set an expiration time in seconds.

kys username "balakrishna"
# Result: ✅ Saved: username (New)

kys username "maduru"
# Result: ⚠️ Key 'username' already exists. Overwrite? (y/n):

kys session_id "data" --ttl 1h
# Result: ✅ Saved: session_id (New) (Expires in 1 hour)

🧱 Typed Workspaces: Queues & Stacks

Optimize your workspace for specific data structures with strict concurrency guarantees (BEGIN IMMEDIATE locking).

1. Queue (FIFO)

# Create
kyws create jobs --type queue
kyuse jobs

# Push & Pop (Atomic)
kypush "job_1"
kypop
# Result: job_1

2. Stack (LIFO)

kyws create undo_log --type stack
kypush "cmd1"
kypush "cmd2"
kypop
# Result: cmd2

3. Priority Queue

kyws create triage --type priority_queue
kypush "low" --priority 1
kypush "high" --priority 100
kypop
# Result: high

See docs/QUEUES_STACKS.md for full details.

📂 Advanced JSONPath & Dot-Notation

kycli allows you to treat your key-value store like a document database. You can query and update deep nested structures without retrieving the entire object.

Nested Retrieval:

# Get a specific field
kyg user.profile.email

# Access list items by index
kyg logs[0]

# List slicing (e.g., first 5 logs)
kyg logs[0:5]

Atomic Patching (Partial Updates):

Instead of rewriting a large JSON object, you can update just one field. Use kypatch (not kys) for this — kys always treats dots in a key as a literal flat key name (e.g. for namespacing, like kys ns.alpha 1), while kypatch treats them as a path into the nested object.

# Update just the 'age' field inside the 'user' object
kypatch user.profile.age 25

📦 Collection Operations (Lists & Sets)

Manage lists stored in keys efficiently without manual read-modify-write loops.

# Append to a list
kypush my_list "new_item"

# Append only if the item doesn't exist (Set behavior)
kypush my_tags "python" --unique

# Remove from a list
kyrem my_list "old_item"

---

🔐 Enterprise-Grade Security: Zero-Trust Encryption

kycli takes a Zero-Trust approach by encrypting the Entire Database File.

• Full Database Encryption: The workspace file (.db) is stored as an encrypted binary blob (AES-GCM).
• Opaque File: It cannot be opened by sqlite3, DB Browser, or any other tool. It appears as random noise without the key.
• In-Memory Speed: On access, kycli decrypts the workspace into secure RAM for microsecond-fast operations.

Via CLI:

# Save with encryption (Encrypts entire workspace file)
kys secret_token "ghp_secure" --key "my-master-password"

# Retrieve with encryption
kyg secret_token --key "my-master-password"

Via Environment Variable (Recommended):

This works for the CLI, TUI (kyshell), and Python Library usage.

export KYCLI_MASTER_KEY="my-master-password"

Master Key Rotation

Rotate all stored values to a new master key.

kyrotate --new-key "newpass" --old-key "oldpass" --backup

# Dry-run to see how many values would rotate
kyrotate --new-key "newpass" --old-key "oldpass" --dry-run

export KYCLI_MASTER_KEY="my-master-password"
kyg secret_token

🔑 Using a Custom Key per Command

You can use a specific key for individual commands without setting it for the whole session. This is useful for storing records with different keys in the same workspace.

# Save 'rec1' with Key A
kys rec1 "secret" --key "KeyA"

# Save 'rec2' with Key B
KYCLI_MASTER_KEY="KeyB" kys rec2 "secret"

# To read 'rec1', you must provide Key A
kyg rec1 --key "KeyA"

[!IMPORTANT] If you attempt to retrieve an encrypted key without the correct master_key, kycli will return a masked message: [ENCRYPTED: Provide a master key to view this value] instead of raw ciphertext.

⏳ Value-Level TTL (Time To Live)

Like Redis, you can set keys to expire automatically. kycli implements Soft Expiration: when a key hits its TTL, it is moved to the Archive table (not deleted) and can be recovered within 15 days using the kyr command.

# Expire in 60 seconds
kys temp_code "1234" --ttl 60

# Expire in 1 day
kys daily_report "data" --ttl 1d

# Expire in 1 month (30 days)
kys monthly_archive "data" --ttl 1M

---

kyg [-s] <key_or_query> — Get & Search

• Get Key: kyg <key> retrieves a value.
• Search: kyg -s <query> performs a Google-like search across all values using FTS5.

# Get exact key
kyg username
# Result: maduru

# Search for "admin" anywhere in the database
kyg -s "admin"
# Result:
# {
#   "user_profile": { "name": "balu", "role": "admin" }
# }

kyl [pattern] — List Keys

Lists all keys or those matching a pattern.

kyl
# Result: 🔑 Keys: username, user_id, env

kyl "user.*"
# Result: 🔑 Keys: username, user_id

kyv [key | -h] — View History (Audit Log)

kycli never deletes your old values; it archives them.

• kyv -h: Shows the full history of ALL keys in a formatted table.
• kyv <key>: Shows the latest value from the history for that specific key.

kyv -h
# Result: 📜 Full Audit History (All Keys)
# Timestamp            | Key             | Value
# -----------------------------------------------------
# 2026-01-03 13:20:01  | username        | maduru
# 2026-01-03 13:10:00  | username        | balakrishna

kyd <key> — Delete Key (Soft Delete)

Removes a key from the active store.

• Double-Confirmation: Requires you to re-type the key name to prevent accidental loss.
• Tip: Deletion is "soft" in terms of data—it stays in history and can be recovered.

kyd username
# Result: ⚠️ DANGER: To delete 'username', please re-enter the key name: username
# Result: Deleted
# Result: 💡 Tip: If this was accidental, use 'kyr username' to restore it.

kyr <key> — Restore Key

Restores a key from its history back into the active store.

• Note: This works for keys in the Archive table. KyCLI keeps deleted data for 15 days before permanent removal.

kyr username
# Result: ✅ Key 'username' restored from history.

kyrt <timestamp> — Point-in-Time Recovery (PITR)

Reconstruct the entire database state at a specific moment in time. This is a "Time Machine" for your data.

• Mechanism: Clears the current store and repopulates it with the state as of the given timestamp using the audit log.

# Restore to New Year's Day
kyrt "2026-01-01 12:00:00"
# Result: 🕒 Database restored to 2026-01-01 12:00:00

kyco [retention_days] — Database Compaction & Maintenance

Cleanup old history and optimize the database file.

• Retention: History older than retention_days (default 15) is purged.
• Optimization: Runs SQLite VACUUM and ANALYZE to reclaim space and optimize query paths.

# Cleanup everything older than 7 days
kyco 7
# Result: 🧹 Compaction complete: Space reclaimed and stale history purged.

kye <file> [format] — Export Data

Exports your entire store to a file.

• Format: csv (default) or json.

kye backup.csv
kye data.json json

kyi <file> — Import Data

Bulk imports data from a CSV or JSON file.

kyi backup.csv

kyc <key> [args...] — Execute Mode

Run a stored value directly as a shell command.

• Static Execution: Run the command exactly as stored.
• Dynamic Execution: Pass additional arguments that get appended to the stored command.

# Store a command
kys list_files "ls -la"

# Execute it
kyc list_files

# Dynamic execution (appends /tmp)
kyc list_files /tmp

---

kyshell — Interactive TUI Shell

Launch a multi-pane interactive shell to manage your data.

• Auto-completion: Tab-completion for all commands.
• Split View: Real-time audit trail in a separate pane.
• Workspace Aware: Semantic status bar showing active workspace and user.

kycli kyshell

---

📚 Documentation & Guides

Topic	Description	Link
Workspaces	Managing multiple projects/tenants (kyuse, kymv).	docs/WORKSPACES.md
Data Management	Import, Export, and Backups (kye, kyi).	docs/DATA_MGMT.md
Recovery	Time travel (PITR), Restore, and Compaction (kyrt, kyco).	docs/RECOVERY.md

---

⚙️ Global Configuration & Env Vars

kycli is highly configurable. You can change the database location, export formats, and UI themes via environment variables or configuration files.

🌍 Environment Variables (Highest Priority)

The most direct way to configure kycli is via shell environment variables.

• KYCLI_DB_PATH: Sets the root directory for storing usage data (workspace databases).
• KYCLI_MASTER_KEY: Sets the default master key for AES-256 encryption.

  export KYCLI_DB_PATH="/custom/path/to/data_dir/"
  export KYCLI_MASTER_KEY="your-secret-password"
  

  Note: If KYCLI_DB_PATH is a directory, workspaces will be created inside it (e.g. /custom/path/to/data_dir/default.db). If it is a file path, it will be used as a single database overriding workspaces.

📁 Configuration Files

kycli looks for configuration in .kyclirc or .kyclirc.json.

Example .kyclirc (JSON):

{
  "db_path": "~/.kydata.db",
  "export_format": "csv"
}

🐍 Python Library Interface

1. Dictionary-like Interface (Sync)

The easiest way to integrate into any Python script or class.

from kycli import Kycore

# Use as a context manager for automatic cleanup
with Kycore() as kv:
    # Set and Get (Dict-style)
    kv['theme'] = 'dark'
    print(kv['theme'])  # dark

    # Check existence
    if 'theme' in kv:
        print("Settings loaded.")

    # Bulk count
    print(f"Items stored: {len(kv)}")

# 4. Encryption & TTL (Sync)
with Kycore(master_key="secret-pass") as kv:
    # Save with 10-minute expiration
    kv.save("temp_password", "123456", ttl="10m")
    
    # Save with 1-month expiration
    kv.save("persistent_secret", "data", ttl="1M")
    
    print(kv.getkey("temp_password")) # 123456

# 5. Batch Operations (save_many)
with Kycore() as kv:
    items = [("k1", "v1"), ("k2", "v2"), ("k3", "v3")]
    kv.save_many(items, ttl="1h")
    # Result: ⚡ Atomic transaction per batch (extremely fast)

# 5. Queues & Stacks (API)
use `set_type` to separate queues from KV stores.
```python
from kycli import Kycore

# Initialize and set type
with Kycore("jobs.db") as q:
    q.set_type("queue")
    
    # Push (Atomic & Durable)
    q.push("task_1")
    q.push({"id": 2, "action": "email"})
    
    # Pop (Thread-Safe)
    item = q.pop()
    print(item) # task_1

6. Maintenance & PITR

with Kycore() as kv: # Cleanup history older than 30 days kv.compact(retention_days=30)

# Point-in-Time Recovery
kv.restore_to("2026-01-01 00:00:00")

### 2. High-Throughput (Async)
Designed for `asyncio` applications like FastAPI.
```python
import asyncio
from kycli import Kycore

async def run_tasks():
    # Use encryption and TTL in async environments
    with Kycore(master_key="async-vault") as kv:
        await kv.save_async("session:active", "true", ttl=3600)
        current = await kv.getkey_async("session:active")
        print(f"Session active: {current}")

asyncio.run(run_tasks())

3. Schema Validation (Pydantic)

Enforce data integrity by attaching a Pydantic model to your store.

from pydantic import BaseModel
from kycli import Kycore

class UserSchema(BaseModel):
    name: str
    age: int

# Initialize with schema validation
with Kycore(schema=UserSchema) as kv:
    # This will succeed and auto-serialize
    kv.save("user:101", {"name": "Balu", "age": 30})
    
    # This will raise a ValueError (Schema Validation Error)
    kv.save("user:102", {"name": "Invalid"}) 

4. Application / Class Integration

Wrap Kycore inside your classes for persistent state management.

class UserManager:
    def __init__(self):
        self.db = Kycore()

    def update_profile(self, user_id, data):
        self.db.save(f"user:{user_id}", data)

    def close(self):
        self.db.__exit__(None, None, None)

4. FastAPI Web Server Integration

from fastapi import FastAPI, Depends
from kycli import Kycore

app = FastAPI()

def get_db():
    with Kycore() as db:
        yield db

@app.get("/config/{key}")
async def fetch_config(key: str, db: Kycore = Depends(get_db)):
    return {"val": await db.getkey_async(key)}

---

🏗 Architecture & Internal Safety

• Encrypted Single-File Storage: Each workspace is a single AES-GCM-encrypted file. kycli decrypts it into an in-memory SQLite engine on open and re-serializes the whole workspace back to disk on every write.
• Atomic Writes: Every persist (workspace file, exports, audit export) uses a "temp-file then rename" strategy — a concurrent reader always sees a complete file, never a partial/corrupted one.
• Cross-Process Write Safety: Writes from independent kycli processes sharing one workspace file are serialized via an OS-level advisory lock (flock, POSIX). A process reloads the latest on-disk state immediately before mutating it, so sibling processes' writes are never silently overwritten. (Windows: atomic writes still prevent corruption, but cross-process mutual exclusion requires POSIX flock and is not yet supported.)
• Data Integrity: Keys are automatically lowercased and stripped to prevent duplicate-but-slightly-different keys.
• Auto-Purge Policy: Deleted keys are moved to an Archive table and automatically purged after 15 days to keep the database size optimized.
• Embedded C: Core operations are written in Cython, binding directly to native library pointers.

---

📊 Benchmarking

Want to test the speed on your own hardware?

PYTHONPATH=. python3 tests/performance/kycli_benchmark.py

---

👤 Author & Support

Balakrishna Maduru

• GitHub
• LinkedIn
• Twitter