jing

jing downloads and organizes stock market data. The examples below use China A-share stocks with BaoStock.

Install

pip install jing

Download CN data

BaoStock is the default data source for CN. Stock codes must include an exchange prefix, such as sh. or sz..

import jing

d = jing.D("cn")
d.download("sz.000807")  # Download one stock
d.download()             # Download the stocks in the default list

Batch downloads use ~/data/jing/list/cn_baostock.txt by default. On first use, jing copies the bundled list there; edit this copy to maintain your stock list.

Download US data

US stocks use Yahoo Finance. Yahoo blocks many datacenter and mainland-China IPs with 403/429, so on such hosts route the requests through a proxy:

export https_proxy=http://127.0.0.1:7890 http_proxy=http://127.0.0.1:7890 all_proxy=socks5://127.0.0.1:7890

import jing

d = jing.D("us")
d.download("AAPL")   # one stock
d.download()         # batch from ~/data/jing/list/us.txt

Batch runs resume per stock: each CSV is the ledger, so a re-run after an interruption only fetches what is still missing.

File locations

The default data directory is ~/data/jing:

~/data/jing/
├── list/cn_baostock.txt
└── raw/baostock/cn/sz.000807.csv

Set JING_DATA to use another directory:

export JING_DATA=/data/stocks

Or set the data directory from Python:

from jing.data_paths import set_data_root

set_data_root("/data/stocks")

The stock list and CSV files will then be stored under /data/stocks/list/ and /data/stocks/raw/baostock/cn/.

Upload to COS (optional)

Uploading to Tencent COS is opt-in — most users have no COS bucket, and nothing in the library touches COS unless you ask it to.

pip install jing[cos]        # the Tencent SDK is not installed by default
export JING_SYNC_COS=1       # upload each market after a batch download

Credentials come from a flat ~/.cos.yaml holding only the secrets:

cos_secret_id: <SecretId>
cos_secret_key: <SecretKey>

Or drive the uploader directly, which is also the only way to feed it credentials from somewhere else (an object built with your own client):

from jing import COSUploader

COSUploader(markets=["cn"]).upload()   # python -m jing.uploader_cos also works

A local file never overwrites a better copy on COS: if it is less than half the remote size, or its last date is older, it is skipped and listed in alerts/alert-<date>.txt on COS. JING_COS_SHRINK_GUARD tunes the ratio and 0 turns the guards off.

After the run, each market also gets a manifest at meta/<market>.json on COS: generated_at plus the key, byte size and last trading date of every file whose COS copy is known to be identical to the local one. A consumer syncing from COS can read the manifest instead of listing objects and probing tails. Files blocked by the guards, or whose upload failed, are deliberately left out, so the consumer probes those — and any missing manifest — itself.

Run daily on a server

deploy/ holds the cron entry point. It is a deployment script, not part of the jing package: jing only downloads and uploads, deploy/daily.py schedules that.

sudo mkdir -p /opt/jing /data/jing /var/log/jing
sudo git clone <this repo> /opt/jing
cd /opt/jing && python3 -m venv .venv && .venv/bin/pip install -e ".[cos]"

sudo install -m 600 deploy/env.example /etc/jing/env
sudo $EDITOR /etc/jing/env          # fill in JING_COS_SECRET_ID / _KEY

.venv/bin/python deploy/daily.py us   # smoke test on one market
crontab deploy/jing.cron              # weekdays 20:00 Asia/Shanghai

run_daily.sh loads /etc/jing/env (cron has no login shell), takes a flock so a slow run cannot overlap the next one, and writes /var/log/jing/<date>.log (kept 14 days).

Check on it any time with the status command — no network needed unless you add --cos:

python3 deploy/status.py          # running? how far along? data up to which day?
python3 deploy/status.py --cos    # also count the objects on COS

Catch up from COS before downloading

deploy/cos_sync.py reconciles the bucket against the local CSVs and pulls the newer copy of each file, so a machine that fell behind — or a fresh one that would otherwise spend days backfilling history — starts from what the bucket already has.

python3 deploy/cos_sync.py --dry-run    # report only, write nothing
python3 deploy/cos_sync.py              # cn + us + hk
python3 deploy/cos_sync.py cn hk        # only these markets

JING_COS_SYNC=1 makes deploy/daily.py do this automatically before each market's download, so the resume logic decides what to fetch against current data. It is off by default: the server is normally the bucket's source (it downloads, then uploads), so the reconcile usually just reports that there is nothing to pull. Turn it on after wiping the disk, on a new machine, or when another host is ahead.

Which copy wins is decided per file from the last trading date in each:

A sync never moves a market backwards. Each market reports its own state first:

us 本地最新 2026-06-05，COS 最新 2026-10-02，待拉 1516 跳过 0 无法比较 0

Deciding costs about 2KB per file (jing.resume.TAIL_BYTES), not a download — the bucket's last date comes from a ranged GET of the object's tail and the size from the listing, so there is no HEAD per file. Only the files that win are fetched whole. JING_COS_SHRINK_GUARD tunes the truncation ratio for this direction too.

Even cheaper, each market is first gated by its manifest: cos_sync fetches meta/<market>.json — one small GET — and compares the signature of its files list with the one recorded in cos_sync_state.json under the data root. If it is unchanged, the whole market is skipped: no listing, no per-file probes. The signature ignores generated_at, so a no-op upload that only rewrites the timestamp does not trigger probing. A missing or unreadable manifest (an uploader older than 0.6.0) falls back to probing. The signature is recorded only after a run without pull failures, and never on --dry-run, which bypasses the gate entirely — it is the inspection tool for finding damaged files after a forced kill, so it always probes everything. Files the manifest omits (blocked or failed uploads) are probed again as soon as the manifest next changes.

This lives in deploy/ rather than the library: it is a deployment policy, not a capability of jing. Like the uploader, credentials come from /etc/jing/env, not ~/.cos.yaml.

Notes:

• Interrupted runs resume. Each stock's CSV is the ledger, so the next run only downloads what is missing. Running out of the time budget is a normal outcome, not a failure.
• Concurrency: JING_MAX_WORKERS overrides the built-in worker count (3 for BaoStock, 10 otherwise). The server sets it to 1 — one request at a time from one IP is the cheapest insurance against being rate limited. D.download(max_workers=N) overrides it per call.
• akshare timeouts: akshare's requests carry no timeout, so jing patches requests.sessions.Session.request to inject one (JING_HTTP_TIMEOUT, default 30s, 0 disables). socket.setdefaulttimeout() cannot do this — urllib3 passes its own None down to sock.settimeout(), overriding it.
• cn uses BaoStock only. cn_ak (AKShare) covers the same A-shares; running both every day doubles the request load for no new data.