aicorpusx

English | 中文

English

Resumable, multi-key concurrent translation for CSV / XLSX corpora. Any service that speaks OpenAI-compatible /chat/completions works: pass apis, model, and base_url.

import aicorpusx

aicorpusx.trans(
    "terms.xlsx",
    source_column="中文",
    targets={"ar": "阿拉伯语", "en": "英语"},
    apis=["key-1", "key-2"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

The default output is terms_translated.xlsx. CSV input produces CSV; XLSX input produces XLSX.

Set base_url to the API root (for example https://api.deepseek.com or https://api.openai.com/v1). The library appends /chat/completions. If you already pass a full endpoint, it is used as-is.

Install

python -m pip install aicorpusx

From a local checkout:

python -m pip install .

Examples

Worksheet, source language, and content mode:

aicorpusx.trans(
    "corpus.xlsx",
    sheet_name="Data",
    source_column="原文",
    source_language="zh",
    targets={"ar": "译文"},
    mode="sentence",  # auto / term / sentence / text
    apis=["key-1", "key-2"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

Pass a glossary dict:

aicorpusx.trans(
    "corpus.csv",
    source_column="原文",
    targets={"ar": "阿拉伯语"},
    glossary={
        "阴阳": "اليِن واليانغ",
        "五行": "العناصر الخمسة",
    },
    glossary_mode="strict",  # strict / prefer / off
    apis=["key-1"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

A multilingual glossary file can use columns such as source,ar,en,de:

aicorpusx.trans(
    "corpus.xlsx",
    source_column="原文",
    targets={"ar": "阿拉伯语", "en": "英语", "de": "德语"},
    glossary="glossary.xlsx",
    apis=["key-1", "key-2"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

Map irregular column names explicitly:

aicorpusx.trans(
    "corpus.xlsx",
    source_column="原文",
    targets={"ar": "阿拉伯语"},
    glossary="glossary.xlsx",
    glossary_source_column="中文术语",
    glossary_target_columns={"ar": "阿语标准译名"},
    apis=["key-1"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

Terms are matched in-place, longest first. strict re-requests when a matched target term is missing from the result; prefer only constrains the prompt; off ignores the glossary.

Scheduling and resume

• strategy="dynamic" (default): every live API pulls from a shared queue; faster keys take more work.
• strategy="balanced": work is split evenly at start, and each API shows a fixed total. Unfinished tasks from a dead key are still handed to others.
• The default request gap is sleep=0.2 seconds.
• 429, 5xx, timeouts, and connection errors back off exponentially with jitter. 401/403 disable that key. Ordinary 400 errors are not retried forever.
• checkpoint=True (default) stores successes and failed tasks. Re-runs with overwrite=False skip cells already filled in the output or checkpoint.
• Rich progress labels workers API 1, API 2, and so on. Logs and checkpoints never store API keys.

Main resilience knobs and defaults:

aicorpusx.trans(
    "corpus.csv",
    source_column="source",
    targets={"en": "English"},
    apis=["key-1", "key-2"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
    max_retries=5,
    backoff_base=1,
    max_backoff=60,
    api_failure_threshold=5,
    api_cooldown=30,
    max_api_cooldown=300,
)

Non-compatible APIs

Requests use the OpenAI-compatible protocol by default. Extra fields such as temperature go in provider_options.

If the service is not /chat/completions, pass an object with translate(...) as provider.

---

中文

面向 CSV / XLSX 语料表的可恢复、多 API Key 并发翻译。任意走 OpenAI 兼容 /chat/completions 的服务，只需提供 apis、model、base_url：

import aicorpusx

aicorpusx.trans(
    "terms.xlsx",
    source_column="中文",
    targets={"ar": "阿拉伯语", "en": "英语"},
    apis=["key-1", "key-2"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

默认输出到 terms_translated.xlsx。CSV 输入自动输出 CSV；XLSX 输入自动输出 XLSX。

base_url 写到 API 根路径即可（例如 https://api.deepseek.com 或 https://api.openai.com/v1），库会自动补上 /chat/completions。若已写成完整 endpoint，则按原样请求。

安装

python -m pip install aicorpusx

本地源码安装：

python -m pip install .

常用示例

指定工作表、源语言与内容模式：

aicorpusx.trans(
    "corpus.xlsx",
    sheet_name="Data",
    source_column="原文",
    source_language="zh",
    targets={"ar": "译文"},
    mode="sentence",  # auto / term / sentence / text
    apis=["key-1", "key-2"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

直接传术语表：

aicorpusx.trans(
    "corpus.csv",
    source_column="原文",
    targets={"ar": "阿拉伯语"},
    glossary={
        "阴阳": "اليِن واليانغ",
        "五行": "العناصر الخمسة",
    },
    glossary_mode="strict",  # strict / prefer / off
    apis=["key-1"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

多语言术语文件可使用 source,ar,en,de 这样的列：

aicorpusx.trans(
    "corpus.xlsx",
    source_column="原文",
    targets={"ar": "阿拉伯语", "en": "英语", "de": "德语"},
    glossary="glossary.xlsx",
    apis=["key-1", "key-2"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

列名不规则时可以显式映射：

aicorpusx.trans(
    "corpus.xlsx",
    source_column="原文",
    targets={"ar": "阿拉伯语"},
    glossary="glossary.xlsx",
    glossary_source_column="中文术语",
    glossary_target_columns={"ar": "阿语标准译名"},
    apis=["key-1"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
)

术语在句中按最长优先做局部匹配。strict 会在返回后校验匹配到的目标术语，未满足时按重试规则重新请求；prefer 只向模型提供约束；off 忽略术语表。

调度与恢复

• strategy="dynamic"（默认）：所有可用 API 从公共队列抢任务，快的 key 自动多处理。
• strategy="balanced"：开始时平均切分，每个 API 显示固定进度；key 失效后，未完成任务仍会交给其他 API。
• 正常请求间隔默认为 sleep=0.2 秒。
• 429、5xx、超时和连接错误会指数退避并加入 jitter；401/403 会停用该 key；普通 400 不会无限重试。
• checkpoint=True（默认）会保存成功结果和失败任务。重新运行且 overwrite=False 时，已有输出和 checkpoint 中的结果都会跳过。
• Rich 进度只显示 API 1、API 2 等编号，日志和 checkpoint 都不会保存 API key。

主要容错参数及默认值：

aicorpusx.trans(
    "corpus.csv",
    source_column="source",
    targets={"en": "English"},
    apis=["key-1", "key-2"],
    model="deepseek-chat",
    base_url="https://api.deepseek.com",
    max_retries=5,
    backoff_base=1,
    max_backoff=60,
    api_failure_threshold=5,
    api_cooldown=30,
    max_api_cooldown=300,
)

非兼容接口

默认按 OpenAI 兼容协议发请求。temperature 等额外字段可用 provider_options 传入。

若服务不是 /chat/completions，可传入实现 translate(...) 的对象作为 provider。