Skip to main content

Self-hosted A-share Parquet data lake with multi-source ingest and daily orchestration

Project description

自托管 A 股研究数据湖

日线 / 复权 / 基本面 / 资金面 / 行业结构 / 宏观舆情等到同一契约,日更编排落盘为带行级溯源的 curated Parquet。
相对拉数库多水位·重试·审计,相对云端宽表可本地续跑;DuckDB / Polars / load() 直接查,无需自建库或通达信客户端。

CLI:asl · 包名:ashare_lake · Python ≥ 3.10 · 只做数据层(回测和信号留给下游)。

为什么用它

AkShare / efinance 解决「怎么拉数」;Tushare 解决「云端宽表」;Qlib / vn.py 解决「研究/交易平台」。
ashare-lake 专做中间层:多源进同一契约,落成可日更、可溯源、可审计的本地 Parquet 湖。

相对同类更突出的几点:

  • 本地可续跑:水位 / 失败重试 / 质量审计,适合挂 cron
  • 行级溯源:每行带 source / data_version / fetched_at,出了问题能查回来源
  • 多源可核验:主源进 curated,备源进 snapshot,可 diff,不静默顶替
  • 研究口径稳定load() 统一复权 / universe / PIT as_of,不用自己拼
  • 纯数据层:不做回测和交易,和拉数库互补、也不绑架整套框架

一条命令跑通真数 demo:

安装与一分钟体验

需要 Python 3.10+(macOS / Linux / Windows)。

pip install ashare-lake
asl demo

通过 TDX 拉几只流动性股票 × 约 30 个交易日,写入 data/ashare-lake-demo/,并打印样例表。需要能访问 TDX 行情主机(大陆出口更稳)。

asl query --config configs/ashare-lake.demo.toml --sql "
  SELECT symbol, trade_date, close, volume, source
  FROM daily_bars
  WHERE symbol = '600519.SH'
  ORDER BY trade_date DESC
  LIMIT 10
"

全量日更(仍不必 clone;在含配置的工作目录执行):

asl config init                              # → configs/ashare-lake.toml(data.root 写为绝对路径)
# 或显式指定:
# asl config init --data-root /data/ashare-lake --force
asl config validate --config configs/ashare-lake.toml
asl init --config configs/ashare-lake.toml
asl run daily --config configs/ashare-lake.toml

asl demo

有什么数据

数据集名即 load() 的第一个参数。字段见 schema,编排元数据见 catalog

类别 数据集
基础参考 instruments · trading_calendar · trading_status(停复牌 / ST)
行情 daily_bars(未复权) · index_bars · adj_factors
公司事件 corporate_actions · announcement_index · earnings_disclosure_schedule
基本面 / 估值 financial_statement_items(PIT) · valuation_metrics · analyst_consensus
资金面 fund_flow · margin_trading · northbound_flows / northbound_holdings · dragon_tiger · block_trades · institutional_holdings
结构 / 行业 sector_members · index_constituents · industry_members
宏观 macro_indicators · market_breadth
舆情 / 轮动 sentiment_scores · hot_rank · sector_bars · sector_fund_flow · news_headlines
风险 share_unlock_schedule · regulatory_events

读数据

from ashare_lake.query import load

bars = load("daily_bars", start="2020-01-01", end="2025-12-31", adjust="hfq")
roe = load("financial_statement_items", items=["roe"], as_of="2024-04-30")

无 extras —— pip install ashare-lake 即装齐所有数据源。

完整文档

详细 schema、runbook、定位对照与合规说明以 GitHub 为准:

代码 MIT。落盘行情 / 公告仍受上游条款约束——本包不附带、也不再分发数据湖。

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ashare_lake-0.3.1.tar.gz (282.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

ashare_lake-0.3.1-py3-none-any.whl (350.0 kB view details)

Uploaded Python 3

File details

Details for the file ashare_lake-0.3.1.tar.gz.

File metadata

  • Download URL: ashare_lake-0.3.1.tar.gz
  • Upload date:
  • Size: 282.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.14.4

File hashes

Hashes for ashare_lake-0.3.1.tar.gz
Algorithm Hash digest
SHA256 901376c2b7ac8b009067dd0768d9d0af747a952fc4b94fc223350b9c57ed1e06
MD5 17543b6981308e84d465eeac22f16c05
BLAKE2b-256 c9db41bac3e7dfdc900e992566616e506d0005b4428114f834f49c5baa553bdb

See more details on using hashes here.

File details

Details for the file ashare_lake-0.3.1-py3-none-any.whl.

File metadata

  • Download URL: ashare_lake-0.3.1-py3-none-any.whl
  • Upload date:
  • Size: 350.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/7.0.0 CPython/3.14.4

File hashes

Hashes for ashare_lake-0.3.1-py3-none-any.whl
Algorithm Hash digest
SHA256 ffa2e7c82ba927d897e3ce9ed280c49fb0327b1351d374d222f6be54520fb1fb
MD5 2c6a76fd2dcc46d128c25955eb9a7b6e
BLAKE2b-256 47567b1144e86f13ef8323b47cce96b89611768b91dba2d962746621436c2e5e

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page