Atabet-Data

A data retrieval library for financial data from multiple sources with token-based authentication.

Authentication

Atabet-Data uses token-based authentication. Token authentication is mandatory for all API calls.

Install authentication support with pip install atabet-data[auth] (requires atabet-token >= 1.2.0 for licensing-server session support).

Token types

Using the AtabetData Class

All API access is through the AtabetData class, which validates authentication tokens:

import os
import socket
from atabet_data import AtabetData

# Licensing JWT (default validation)
client = AtabetData(
    token=os.environ["ATABET_DATA_TOKEN"],
    licensing_server_url=os.environ.get("LICENSING_SERVER_URL", "http://39.101.65.167:5001"),
    device_name=os.environ.get("ATABET_DEVICE_NAME", os.environ.get("ATABET_CLIENT_ID", socket.gethostname())),
)

# Dev HMAC token (offline / local testing)
client = AtabetData(token=dev_token, validation_method="original")

Set environment variables in the same shell that runs Python:

export ATABET_DATA_TOKEN='eyJhbGci...'
export LICENSING_SERVER_URL='http://39.101.65.167:5001'
export ATABET_DEVICE_NAME='my-macbook'

Seats

One install is one seat. AtabetData sends ATABET_DEVICE_NAME (or ATABET_CLIENT_ID) and ATABET_SESSION_TOKEN when those are set, including when the call also passes a hostname, so a script inside Quant Intelligence shares the app's seat. Outside the app, pass device_name (hostname is fine) and the client stores session_token in ~/.atabet/session.json (override with ATABET_SESSION_STORE). Basic allows one seat online and Pro allows three. A new install past that cap takes the seat from the oldest one.

Licenses must be issued with application_id: "com.atabet.chat" (the Quant Intelligence license). A JWT plan claim of basic can use yahoo, localcsv, and tushare. futu, qmt, and ib require plan: professional. A token with no plan claim is a legacy license and keeps Pro connectors.

Note:

• Token authentication is mandatory and validated when creating the AtabetData instance
• If an invalid token is provided, InvalidTokenError is raised during initialization
• Once initialized, all methods can be called without additional authentication
• The token is the Quant Intelligence license (com.atabet.chat), not a separate data license

Yahoo Finance (source='yahoo')

Install the optional extra (pip install atabet-data[yahoo] or pip install yfinance), then pass source='yahoo'. Tickers must be Yahoo symbols (for example AAPL, 700.HK, ^GSPC), not Bloomberg-style identifiers. Hong Kong names are 4-digit on Yahoo (0700.HK); 700.HK is accepted and padded. Supported fields are PX_OPEN, PX_HIGH, PX_LOW, PX_LAST, and PX_VOLUME for snapshot and daily history. get_datasets, get_mem_weights, and run_query are not implemented for this channel.

1-minute intraday: pass freq='1M'. Yahoo limits 1m bars to ~7 calendar days per request and only within the last ~60 days; wider or older ranges raise ValueError before calling yfinance. For multi-exchange intraday requests (e.g. AAPL + 700.HK), atabet_meta['exchange_tz'] is a per-ticker dict when zones differ. With output_tz='UTC', each ticker is converted using its own exchange zone before columns are merged.

Futu OpenAPI (source='futu')

Install the optional extra (pip install atabet-data[futu] or pip install futu-api), run Futu OpenD locally (default 127.0.0.1:11111), then pass source='futu'. Tickers must be Futu codes (HK.00700, US.AAPL, SH.600000), not Yahoo or Bloomberg identifiers. Snapshot uses get_market_snapshot; history uses paged request_history_kline (see Futu API Doc v10.9), once per ticker (OHLCV columns are sliced from that page set, not re-requested per field). Override OpenD with host/port kwargs or FUTU_OPEND_HOST / FUTU_OPEND_PORT. Supported fields include PX_OPEN, PX_HIGH, PX_LOW, PX_LAST, PX_VOLUME, and PX_TURNOVER. Intraday freqs map to Futu KLType (1M→K_1M, 1H→K_60M, …). See samples/futu_example.py.

Interactive Brokers (source='ib')

Install the TWS API 10.48+ Python client (ibapi — do not pip install ibapi from PyPI; that line is 9.81 and does not match current TWS), run TWS or IB Gateway with API access enabled (paper 7497, live 7496), then pass source='ib'. Tickers: US.AAPL, HK.00700, BTC.USD (CRYPTO PAXOS, whatToShow=AGGTRADES), EUR.USD (CASH IDEALPRO, MIDPOINT). Snapshot uses reqMktData(snapshot=True); history uses paged reqHistoricalData (one page-set per ticker; 1-minute windows are fetched in 1-day slices). Override host/port with kwargs or ATABET_IB_HOST / ATABET_IB_PORT. The quote socket uses ATABET_IB_DATA_CLIENTID (default 17) so it does not collide with the trader gateway (ATABET_IB_CLIENTID, usually 1). Supported fields: PX_OPEN, PX_HIGH, PX_LOW, PX_LAST, PX_VOLUME. See samples/ib_example.py.

Big QMT Redis bridge (source='qmt')

Install the optional extra (pip install atabet-data[qmt] or pip install redis), keep the shared Big QMT bridge from atabet-trader running as a live strategy, then pass source='qmt'. This is the MiniQMT sunset path: clients talk Redis only (no xtquant). Tickers use QMT codes (600000.SH, 000001.SZ). Daily OHLCV via bridge get_market_data_ex; financials via get_datasets (Balance/Income/CashFlow/Capital/Pershareindex, or native ASHARE* tables) plus get_divid_factors / get_instrumentdetail over Redis. History and financials are local QMT cache only (数据管理 → 补充数据). Configure Redis with kwargs or QMT_REDIS_HOST / QMT_REDIS_PORT / QMT_REDIS_PASSWORD / QMT_REDIS_DB / QMT_KEY_PREFIX / QMT_RPC_TIMEOUT (defaults match trader: db 5, prefix atabet:qmt). See samples/qmt_example.py.

Tushare Pro (source='tushare')

Install the optional extra (pip install atabet-data[tushare]), then pass source='tushare' with token= or TUSHARE_TOKEN. Tickers are ts_code (600519.SH, 000001.SZ). get_historical_data uses unadjusted pro.daily (no adj= flag; daily only). get_adj_factor returns the daily trade_date / adj_factor series (pro.adj_factor, ~2000 points) for load-time QFQ in atabet-trader. Do not persist pro_bar(adj='qfq') as the kline CSV.

Channel capabilities (native operations)

Channels do not all implement the same method matrix. Discover what each source supports at runtime:

from atabet_data import AtabetData, list_channels, describe_channel

print(list_channels())                 # summary: sources + operation names
print(describe_channel("yahoo"))       # params, ticker convention, field map

client = AtabetData(token=token)
df = client.run_operation(
    "localcsv",
    "get_historical_data",
    {
        "tickers": ["FUT.CBOT.ZC"],
        "flds": ["PX_LAST"],
        "start": "2024-01-01",
        "end": "2024-01-31",
        "csv_path": "/path/to/K_1D",
    },
)

Adapters for blpapi, bql, bqia, s3, and xtdata remain in the package (hidden=True on ChannelCapability) but are omitted from list_channels() / channel_catalog.json until re-enabled.

Calling an unsupported convenience method (e.g. get_datasets on yahoo) raises NotImplementedError with the available operations and a pointer to run_operation. The static catalog is also shipped as atabet_data/channel_catalog.json.

Each channel includes docs_urls (official / primary references) and search_hints (good web_search queries) so agents know where to look before inventing vendor API usage.

Maintaining the channel catalog

Edit here (source of truth): atabet_data/capabilities.py — CHANNEL_CAPABILITIES entries (operations, params, ticker conventions, docs_urls, search_hints, notes, hidden, requires_pro). Set hidden=True to omit a channel from the public catalog without deleting the adapter.

Do not hand-edit atabet_data/channel_catalog.json; it is generated.

# from atabet-data repo root (expects sibling ../atabet-quant-intelligence by default)
python scripts/sync_channel_catalog.py
# optional: --qi-path /path/to/atabet-quant-intelligence   or   --skip-qi
pytest tests/test_capabilities.py

Commit both repos when the catalog changes:

• atabet-data: capabilities.py + channel_catalog.json
• atabet-quant-intelligence: src/lib/data-channels/channel-catalog.json

Do not hardcode vendor doc URLs only in the QI skill — the capability catalog is the source of truth. See also scripts/sync_channel_catalog.py.

Index and timezone contract

Historical outputs use a consistent multi-index layout across channels:

• Daily data: index levels (FLDS, DATE) — naive calendar dates at midnight (no timezone).
• Intraday data: index levels (FLDS, DATETIME) — exchange-local timestamps by default; pass output_tz='UTC' to convert (exchange_tz may be required for naive sources such as localcsv). Yahoo and localcsv multi-exchange intraday apply UTC conversion per ticker before columns are merged.
• Parameters: get_historical_data(..., freq='1M', output_tz='UTC') — freq selects bar size ('1D' daily default; intraday e.g. '1M', '1H' for Yahoo); output_tz converts intraday timestamps when supported. For localcsv, exchange_tz may be a single IANA string or a {ticker: zone} dict. Crypto quotes such as BTC.USD infer UTC.
• Metadata: returned DataFrames include df.attrs['atabet_meta'] with freq, index_semantics (calendar_date or timestamp), output_tz, exchange_tz (string or per-ticker dict for multi-exchange Yahoo/localcsv intraday), and frame_schema.

Channel matrix (historical)

Passing freq or output_tz to daily-only qmt is safe — the wrapper strips unsupported kwargs before calling the channel.

Available Methods

The AtabetData class provides:

Channel-native (preferred for full coverage)

• list_channels() — summarize registered channels and operations
• describe_channel(source) — full params / ticker conventions / field map
• run_operation(channel, operation, params) — execute a native operation

Convenience facades (best-effort OHLCV / datasets where supported)

• get_data() — snapshot
• get_historical_data() — historical OHLCV
• get_datasets() — datasets
• get_mem_weights() — index member weights
• run_query() — raw vendor query (channel-specific)

Module-level helpers (no auth): list_channels, describe_channel, catalog_as_dict.

Technical Analysis (ta accessor)

Atabet-Data ships with a built-in technical analysis module. Any OHLCV DataFrame can compute indicators directly via the .ta accessor (no external dependency required):

from atabet_data import AtabetData

client = AtabetData(token=your_token)
df = client.get_historical_data(
    tickers=["AAPL US Equity"],
    flds=["PX_OPEN", "PX_HIGH", "PX_LOW", "PX_LAST", "PX_VOLUME"],
    start="2024-01-01", end="2024-12-31", source="yahoo",
)
ohlcv = df.ext.to_ohlcv("AAPL US Equity")

# Individual indicators
ohlcv.ta.sma(length=20)        # Simple Moving Average
ohlcv.ta.rsi(length=14)        # Relative Strength Index
ohlcv.ta.macd()                # MACD (returns DataFrame with line/signal/histogram)
ohlcv.ta.bbands(length=20)     # Bollinger Bands
ohlcv.ta.atr(length=14)        # Average True Range
ohlcv.ta.obv()                 # On-Balance Volume

# Append results to the DataFrame
ohlcv.ta.sma(length=50, append=True)
ohlcv.ta.rsi(length=14, append=True)

# Batch: run a preset strategy (Common or All)
ohlcv.ta.strategy("Common")

# Or define a custom strategy
from atabet_data.ta.strategy import Strategy
my_strat = Strategy(name="Quick", ta=[
    {"kind": "ema", "length": 9},
    {"kind": "rsi", "length": 7},
    {"kind": "bbands", "length": 20, "std": 2.5},
])
ohlcv.ta.strategy(my_strat)

Indicator categories

Functional API

All indicators are also available as standalone functions:

from atabet_data.ta import sma, rsi, macd
sma(ohlcv["close"], length=20)
rsi(ohlcv["close"], length=14)

Custom indicators

Register your own indicators at runtime:

from atabet_data.ta.custom import bind, create_dir, import_dir

# Scaffold a directory tree, write custom modules, then import them
create_dir("~/my_indicators")
import_dir("~/my_indicators")

See samples/ta_example.py for a complete walkthrough.

Environment preparations

Quant Intelligence users: create the combined stack with ../atabet-quant-intelligence/environment.yml (atabet_env) instead of a data-only env.

conda create -n py39_atabet_data python=3.9
conda activate py39_atabet_data
(py39_atabet_data_test) conda install conda-forge::cython
(py39_atabet_data_test) conda install anaconda::pandas
(py39_atabet_data_test) conda install anaconda::setuptools # Windows

Packaging

# Basic usage - will cythonize all .py files including __init__.py
python atabet-data/run_packaging.py

# Remove source files from the wheel
python atabet-data/run_packaging.py --remove-source

# Specify a different module name, version, or distribution directory
python run_packaging.py --module="my-module" --version="2.0.0" --dist-dir="dist"

# Combine options
python run_packaging.py --remove-source --module="my-module" --version="2.0.0"

Test

Unit tests run offline with mocked channels (no Bloomberg, Yahoo, or licensing server required):

conda activate atabet_data_env
pytest tests/ -v

Integration tests (real licensing server) are marked @pytest.mark.integration and excluded by default. Run them explicitly when the server is available:

pytest tests/ -m integration -v

GitHub Actions runs unit tests on every push/PR via .github/workflows/test.yml. Wheel builds remain gated by [build ci] in .github/workflows/build-wheels.yml.

Sample scripts live in samples/ — see samples/README.md for channel demos. Downloading history into atabet-trader backtest folders: samples/LLM_TRADER_HANDOFF_GUIDE.md (+ trader samples/download_atabet_data_kline.py).

GitHub Actions: tests, wheels, and PyPI

Unit tests: .github/workflows/test.yml runs pytest on Ubuntu for every push and pull request (integration tests excluded by default).

Wheel builds: .github/workflows/build-wheels.yml. Wheels are built with cibuildwheel; options live in pyproject.toml under [tool.cibuildwheel].

When jobs run (push): the Linux / Windows / macOS matrix runs only if the head commit message includes one of:

Plain pushes (no tag above) skip the matrix so CI does not run on every commit.

Manual runs: Actions → Build wheels → Run workflow runs the build matrix only; publishing is by push with [pub pypi] or [pub test.pypi] in the commit message.

Authentication (publish): the workflow uses Trusted Publishing (OIDC) by default — no API token in GitHub secrets. Register the GitHub repo and workflow file on each index (project Manage → Publishing). Linux uses gh-action-pypi-publish. That action only runs on Linux, so Windows and macOS exchange the same OIDC token and upload with twine. See the comment block at the top of build-wheels.yml for OIDC vs API-token options.

Private repositories are supported for OIDC the same as public ones.