Common research utilities for our NUIST-GenAI-Lab and CCGM tasks
Project description
cfskit
cfskit 是一个轻量科研代码库,提供日志、TensorBoard 写入、进程信号控制,以及一个可运行的实验工程示例:
- nlab-template: 使用连续编号 pipeline 表达数据准备、训练和评估的 CIFAR-10 工程。
模板用于复现、二次开发或新实验起步;运行逻辑、配置、checkpoint 和验证证据都保留在普通代码与文档中。
本仓库推荐使用 OpenSpec 管理非平凡变更。根目录 openspec/ 用于讨论和设计 cfskit 工具模块迭代、模板规范化与维护变更;各模板工程内部的 openspec/ 只服务对应 demo 项目的本地科研变更示范。
| 工具实例 | 来源 | 说明 |
|---|---|---|
accelerator |
accelerate.Accelerator |
由项目 Runtime 显式创建并持有 |
logger |
cfskit.log_util |
基于 loguru 的全局代理,tqdm 兼容,单次初始化 |
tb |
cfskit.tb_util |
基于 TensorBoard 的全局代理,多进程自动适配 |
ipc |
cfskit.ipc_util |
基于 SIGUSR1/SIGUSR2 的进程内单例,支持优雅控制训练流程 |
基础 API
from accelerate import Accelerator
from cfskit import ipc, logger, setup_logger, setup_tensorboard, tb
accelerator = Accelerator()
setup_logger("output/demo/logs", name="nlab")
setup_tensorboard("output/demo/runs", name="nlab", accelerator=accelerator)
ipc.register_signal_handler()
accelerator.wait_for_everyone()
logger.info("experiment started")
tb.scalar("Loss/train", 0.1, step=1)
if ipc.get_s1():
ipc.switch_s1(0)
推荐统一使用 accelerator.xxx、logger.xxx、tb.xxx、ipc.xxx。旧的
register_signal_handler()、get_s1()、switch_s1() 等 IPC 函数仍保留为兼容 wrapper,
但新代码不再使用散函数调用。
安装
python -m pip install "cfskit>=0.1.3"
如果需要使用 setup_tensorboard(),请安装 torch extra:
python -m pip install "cfskit[torch]>=0.1.3"
如果还需要 tb.image_grid() 等 torchvision 能力:
python -m pip install "cfskit[torch,tb-vision]>=0.1.3"
仓库内直接运行 examples/nlab-template/ 时,使用 editable 安装以确保调用当前源码:
cd examples/nlab-template
python -m pip install -e "../..[torch,tb-vision]"
nlab-template 当前要求 cfskit>=0.1.3。复制为独立工程后应安装已发布版本,不再依赖仓库相对路径。
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file cfskit-0.1.3.tar.gz.
File metadata
- Download URL: cfskit-0.1.3.tar.gz
- Upload date:
- Size: 26.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
8c22532e94e133045c997d5cc928af339b5b543a379f0b4978ebf7228d28e958
|
|
| MD5 |
a3f86a246cc923d6847c9dc523f18dc4
|
|
| BLAKE2b-256 |
f6fd7acffa2f3dd560e7c24c538e13dd9238904bba1a3d05e0dfcc137d151562
|
File details
Details for the file cfskit-0.1.3-py3-none-any.whl.
File metadata
- Download URL: cfskit-0.1.3-py3-none-any.whl
- Upload date:
- Size: 11.9 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/6.2.0 CPython/3.12.13
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
4edd68bb92e49c56b27eef5cf0169f1af9ad14e396caf2690f07224699bf89b4
|
|
| MD5 |
7d59412a65368cfe43dbd2e12dbedb82
|
|
| BLAKE2b-256 |
b396d8ca054c6c0472fe8156a91989522deae70f4cbbceed8b449b968055240f
|