Skip to main content

High-speed cryptocurrency OHLCV data downloader via concurrent API requests

Project description

Intro

  • Downloads crypto kline (candlestick) data fast by making as many concurrent API requests as possible
    • Takes 3 minutes to download 1 month of entire market data (597 symbols), with 5 minutes time interval

Usage

pip install crypto-data-downloader
import asyncio

from crypto_data_downloader.binance import ALL_COLUMNS, CryptoDataDownloader
from crypto_data_downloader.utils import load_pkl, plot_crypto_data

# Refer to https://developers.binance.com/docs/binance-spot-api-docs/rest-api/market-data-endpoints#klinecandlestick-data


x = CryptoDataDownloader()
x.weight_lim = 5000  # Binance request weight limit per minute, max: 6000
x.quote = "USDT"  # Quote asset
x.interval = "5m"  # Kline time interval
x.kline_lim = 1000  # Kline number of data points per request
x.columns = ["open_time", "close"]  # Data columns to include
print(f"All data columns: {ALL_COLUMNS}")
asyncio.run(x.download("2025-07-01", "2025-08-01"))  # Time in UTC

path = f"data/{x.name}_2025-07-01_2025-08-01.pkl"
data = load_pkl(path, gz=True)
plot_crypto_data(data, path)
  • Output
All data columns: ['open_time', 'open', 'high', 'low', 'close', 'volume', 'close_time', 'quote_volume', 'n_trades', 'taker_buy_base_volume', 'taker_buy_quote_volume', 'unused']
weight lim: 5000/6000, USDT symbols: 597, spot: 557, margin: 401
9 intervals * 597 symbols = 5373 requests -> 2.1492 minutes
left: 5373/5373
server time: 1754371957632, my time: 1754371957664, diff: -32 ms, weight used: 21
left: 2884/5373
server time: 1754372028132, my time: 1754372028165, diff: -33 ms, weight used: 1
left: 385/5373
server time: 1754372095736, my time: 1754372095764, diff: -28 ms, weight used: 1
left: 0/5373

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

crypto_data_downloader-0.2.0.tar.gz (6.3 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

crypto_data_downloader-0.2.0-py3-none-any.whl (6.6 kB view details)

Uploaded Python 3

File details

Details for the file crypto_data_downloader-0.2.0.tar.gz.

File metadata

  • Download URL: crypto_data_downloader-0.2.0.tar.gz
  • Upload date:
  • Size: 6.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.11.14

File hashes

Hashes for crypto_data_downloader-0.2.0.tar.gz
Algorithm Hash digest
SHA256 005b18c9754bc80d2495bffabd48ebf56e3b1efad971e5332b1ef6c6867168e8
MD5 b12b6ee583c4122ee65c0b785f53736b
BLAKE2b-256 353531c4ed017429f32a5a8869e1d53e783165a1b25c4294d9f15f8f3b2b2a74

See more details on using hashes here.

File details

Details for the file crypto_data_downloader-0.2.0-py3-none-any.whl.

File metadata

File hashes

Hashes for crypto_data_downloader-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 cce031bf0f28d95134c6fef5f1fc85ed5d106df67d5bfb4477a466b1216e6b5d
MD5 2e2e1b6638cec7b212878e6076191c45
BLAKE2b-256 e09a9038e717e70b05d00bb910431815736b9b2fb2b73a73f830fadbcde35dc8

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page