Skip to main content

High-speed cryptocurrency OHLCV data downloader via concurrent API requests

Project description

Intro

  • Downloads crypto kline (candlestick) data fast by making as many concurrent API requests as possible
    • Takes 3 minutes to download 1 month of entire market data (597 symbols), with 5 minutes time interval

Usage

pip install crypto-data-downloader
import asyncio

from crypto_data_downloader.binance import ALL_COLUMNS, CryptoDataDownloader
from crypto_data_downloader.utils import load_pkl, plot_crypto_data

# Refer to https://developers.binance.com/docs/binance-spot-api-docs/rest-api/market-data-endpoints#klinecandlestick-data


x = CryptoDataDownloader()
x.weight_lim = 5000  # Binance request weight limit per minute, max: 6000
x.quote = "USDT"  # Quote asset
x.interval = "5m"  # Kline time interval
x.kline_lim = 1000  # Kline number of data points per request
x.columns = ["open_time", "close"]  # Data columns to include
print(f"All data columns: {ALL_COLUMNS}")
asyncio.run(x.download("2025-07-01", "2025-08-01"))  # Time in UTC

path = f"data/{x.name}_2025-07-01_2025-08-01.pkl"
data = load_pkl(path, gz=True)
plot_crypto_data(data, path)
  • Output
All data columns: ['open_time', 'open', 'high', 'low', 'close', 'volume', 'close_time', 'quote_volume', 'n_trades', 'taker_buy_base_volume', 'taker_buy_quote_volume', 'unused']
weight lim: 5000/6000, USDT symbols: 597, spot: 557, margin: 401
9 intervals * 597 symbols = 5373 requests -> 2.1492 minutes
left: 5373/5373
server time: 1754371957632, my time: 1754371957664, diff: -32 ms, weight used: 21
left: 2884/5373
server time: 1754372028132, my time: 1754372028165, diff: -33 ms, weight used: 1
left: 385/5373
server time: 1754372095736, my time: 1754372095764, diff: -28 ms, weight used: 1
left: 0/5373

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

crypto_data_downloader-0.1.7.tar.gz (5.6 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

crypto_data_downloader-0.1.7-py3-none-any.whl (6.0 kB view details)

Uploaded Python 3

File details

Details for the file crypto_data_downloader-0.1.7.tar.gz.

File metadata

  • Download URL: crypto_data_downloader-0.1.7.tar.gz
  • Upload date:
  • Size: 5.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.13.2

File hashes

Hashes for crypto_data_downloader-0.1.7.tar.gz
Algorithm Hash digest
SHA256 f854c870d43b93e71d6bc1c36118c14b1bd424285a21e3ee7fb4f5024e844061
MD5 53a6444768d6290f5d6b9813f878ab63
BLAKE2b-256 4bb1e309f647b98fc9bf3005651c0268be32e54969ef3e7b6d56a63d41010521

See more details on using hashes here.

File details

Details for the file crypto_data_downloader-0.1.7-py3-none-any.whl.

File metadata

File hashes

Hashes for crypto_data_downloader-0.1.7-py3-none-any.whl
Algorithm Hash digest
SHA256 6a4ad012e91b68e330b976ff9f1a6203b13ca706af51f0726b261692b70eff73
MD5 d631fc0e28f1656ddeee83f477fda846
BLAKE2b-256 0d5d3b91f0e9b2f34650c4b6c3913bc1fa53b06fa4b71182fe42607db7e940ea

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page