Skip to main content

Professional market data integration with clean package architecture

Project description

Crypto Kline Vision Data

High-performance market data integration with Failover Control Protocol (FCP).

Package: crypto-kline-vision-data | Import: ckvd | Python: 3.13

GitHub Release

Features

  • Failover Control Protocol (FCP): Cache (~1ms) → Vision API (~1-5s) → REST API (~100-500ms) — automatic failover, retry, and gap detection
  • Apache Arrow Cache: Memory-mapped local cache for instant repeated access
  • Binance Vision API: Bulk historical data from AWS S3 (no rate limits, ~48h delay)
  • Binance REST API: Real-time data with built-in rate limit handling (Spot, USDS-M Futures, Coin-M Futures)
  • Polars Engine: Internal Polars LazyFrames + streaming; pandas or Polars output at API boundary
  • AI Agent Introspection: __probe__.py module for stateless API discovery
  • Security: Symbol validation (CWE-22 path traversal prevention)
  • Machine-Parseable Errors: All exceptions carry .details dict

Quick Start

git clone https://github.com/terrylica/crypto-kline-vision-data.git
cd crypto-kline-vision-data
uv sync --dev
from ckvd import CryptoKlineVisionData, DataProvider, MarketType, Interval
from datetime import datetime, timedelta, timezone

manager = CryptoKlineVisionData.create(DataProvider.BINANCE, MarketType.FUTURES_USDT)

end = datetime.now(timezone.utc)
start = end - timedelta(days=7)

df = manager.get_data("BTCUSDT", start, end, Interval.HOUR_1)
print(f"Loaded {len(df)} bars")
manager.close()

Or as a Git dependency in your pyproject.toml:

dependencies = [
    "crypto-kline-vision-data @ git+https://github.com/terrylica/crypto-kline-vision-data.git"
]

For Claude Code Users

This repository is optimized for Claude Code with a hub-and-spoke CLAUDE.md architecture. When Claude Code opens any directory, it automatically loads that directory's CLAUDE.md — giving it domain-specific context, conventions, and deep links to related documentation.

Start here: CLAUDE.md (root hub) — then Claude Code discovers everything else autonomously.

Hub-and-Spoke Architecture

CLAUDE.md (root hub)
├── src/CLAUDE.md         → Package structure, FCP, exceptions, __probe__, security
├── tests/CLAUDE.md       → Test commands, markers, fixtures, mocking patterns
├── docs/CLAUDE.md        → ADRs, skills, benchmarks, troubleshooting
├── examples/CLAUDE.md    → Example conventions, NDJSON telemetry schema
├── scripts/CLAUDE.md     → Dev scripts, mise tasks, cache tools
└── playground/CLAUDE.md  → Experimental prototypes

Each spoke contains only the context relevant to that directory. Claude Code loads them on demand — no context window waste.

What Claude Code Gets

  • Skills in docs/skills/ — progressive disclosure guides for usage, testing, research, FCP monitoring
  • Agents in .claude/agents/ — specialized subagents (API reviewer, test writer, FCP debugger, silent failure hunter)
  • Commands in .claude/commands//review-ckvd, /feature-dev
  • Full API surface via from ckvd.__probe__ import discover_api — JSON-serializable metadata for agent introspection

Tips for Working with Claude Code

  1. Just ask — Claude Code reads the relevant CLAUDE.md files automatically when you work in a directory
  2. Use skills — ask Claude to "fetch BTCUSDT data" or "run tests" and it discovers the right patterns
  3. Use agents@silent-failure-hunter to audit code, @test-writer to generate tests
  4. Use probefrom ckvd.__probe__ import discover_api for programmatic API discovery

Examples

# Via mise tasks
mise run demo:quickstart          # Minimal FCP usage
mise run demo:features            # Feature engineering pipeline
mise run demo:cache               # Cache toggle mechanisms
mise run demo:logging             # Logging configuration
mise run demo:datetime            # Timezone handling
mise run demo:one-second          # 1s interval (SPOT only)
mise run demo:lazy                # Lazy initialization

# Or directly
uv run -p 3.13 python examples/quick_start.py

All examples emit NDJSON telemetry to examples/logs/events.jsonl. See examples/CLAUDE.md for schema and parsing.

API Reference

Core API

from ckvd import CryptoKlineVisionData, DataProvider, MarketType, Interval

# Manager-based (recommended)
manager = CryptoKlineVisionData.create(DataProvider.BINANCE, MarketType.FUTURES_USDT)
df = manager.get_data("BTCUSDT", start, end, Interval.HOUR_1)
manager.close()

# High-level function
from ckvd import fetch_market_data, ChartType
df, elapsed, count = fetch_market_data(
    provider=DataProvider.BINANCE, market_type=MarketType.SPOT,
    chart_type=ChartType.KLINES, symbol="BTCUSDT",
    interval=Interval.HOUR_1, start_time=start, end_time=end
)

Market Types and Symbols

Market Type Symbol Format Example
SPOT {BASE}{QUOTE} BTCUSDT
FUTURES_USDT {BASE}{QUOTE} BTCUSDT
FUTURES_COIN {BASE}USD_PERP BTCUSD_PERP

Output Formats

# Default: pandas DataFrame (backward compatible)
df = manager.get_data("BTCUSDT", start, end, Interval.HOUR_1)

# Opt-in: Polars DataFrame (zero-copy, faster)
df = manager.get_data("BTCUSDT", start, end, Interval.HOUR_1, return_polars=True)

Error Handling

from ckvd.utils.for_core.rest_exceptions import RateLimitError, RestAPIError
from ckvd.utils.for_core.vision_exceptions import VisionAPIError

try:
    df = manager.get_data(...)
except RateLimitError as e:
    print(f"Rate limited, retry after {e.retry_after}s. Details: {e.details}")
except (RestAPIError, VisionAPIError) as e:
    print(f"Error: {e}. Details: {e.details}")

Environment Variables

Variable Purpose Default
CKVD_LOG_LEVEL Log level (DEBUG/INFO/ERROR) ERROR
CKVD_ENABLE_CACHE Enable/disable cache true
CKVD_USE_POLARS_OUTPUT Zero-copy Polars output false

Development

uv sync --dev                    # Install dependencies
mise trust                       # Load environment
uv run -p 3.13 pytest tests/unit/ -v   # Run tests (399 passing)
uv run -p 3.13 ruff check --fix .      # Lint

See CLAUDE.md for full development conventions, commit trailers, and release workflow.

Documentation

Resource Purpose
CLAUDE.md Root hub — start here for Claude Code
docs/INDEX.md Documentation navigation
docs/TROUBLESHOOTING.md Common issues and solutions
docs/GLOSSARY.md Domain terminology
examples/ Runnable examples with NDJSON telemetry
CHANGELOG.md Release history (auto-generated)

License

MIT License — See LICENSE file for details.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

crypto_kline_vision_data-4.3.11.tar.gz (185.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

crypto_kline_vision_data-4.3.11-py3-none-any.whl (229.5 kB view details)

Uploaded Python 3

File details

Details for the file crypto_kline_vision_data-4.3.11.tar.gz.

File metadata

  • Download URL: crypto_kline_vision_data-4.3.11.tar.gz
  • Upload date:
  • Size: 185.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.4 {"installer":{"name":"uv","version":"0.10.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for crypto_kline_vision_data-4.3.11.tar.gz
Algorithm Hash digest
SHA256 5804a58c87d8cada5de63b29120905a1b22d55ab5121e29b49a54bf52ab3e943
MD5 38de6f3acc82b4fff9cb33578e48edc3
BLAKE2b-256 f6a0bbc411f3d2e5a3ca2bac0d786cb917f392b30e89b33656955756fc77c4d7

See more details on using hashes here.

File details

Details for the file crypto_kline_vision_data-4.3.11-py3-none-any.whl.

File metadata

  • Download URL: crypto_kline_vision_data-4.3.11-py3-none-any.whl
  • Upload date:
  • Size: 229.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.10.4 {"installer":{"name":"uv","version":"0.10.4","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"macOS","version":null,"id":null,"libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for crypto_kline_vision_data-4.3.11-py3-none-any.whl
Algorithm Hash digest
SHA256 27bcc767c7860c72bcedd17661c48842f4e718fdbf4060db811f9c02de736913
MD5 2f848afa8c7d61ee22a209945df650b6
BLAKE2b-256 d00aa2b55d4438e6f5ae8c112df4ec7c966e2bf4f06f2a432e7fb63551c8c639

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page