Skip to main content

HFS v3 - SET 索引优化版本

基于 v2 的优化版本,主要改进:

优化内容

1. SET 索引优化

  • 新增 hfs:project:{proj}:spaces:active SET 索引
  • 新增 hfs:account:{user}:spaces:active SET 索引
  • Lua 脚本原子维护 SET(bind/transition)
  • get_account_space_count 从 O(S) 降到 O(1)

2. Admin CLI Pipeline 优化

  • space list / account list 使用 Pipeline 批量获取
  • 减少网络往返,提升响应速度

3. 性能提升

  • select_account: 26s → 3s
  • space list (db6): 18s → 14s
  • 支持 Space 数量: ~100 → 10000+

目录结构

v3/
├── hfs/                    # 核心代码
│   ├── state.py            # SET 索引 Lua 脚本
│   ├── account.py          # O(1) 账号查询
│   ├── admin/              # CLI 命令
│   └── ...
├── tools/                  # 实用工具
│   ├── view_space_logs.py  # 查看 HF 原生日志
│   ├── chaos_test.py       # 混沌测试
│   └── multi_project_test.py  # 多项目测试
├── publish-tools/          # 发布脚本
└── docs/                   # 文档

使用

环境配置

# v3 使用独立的 Redis db
export HFS_REDIS_URL="redis://...../7"  # db7

迁移 SET 索引

# 首次部署需要构建 SET 索引
python tools/migrate.py --redis-url $HFS_REDIS_URL set-index

# 验证一致性
python tools/migrate.py --redis-url $HFS_REDIS_URL verify

运行测试

# 混沌测试
python tools/chaos_test.py

# 多项目测试
python tools/multi_project_test.py

与 v2 的区别

特性 v2 v3
Redis DB db6 db7
账号查询 O(S) scan O(1) SET
CLI 性能 逐个 GET Pipeline
数据结构 无索引 SET 索引

发布

# Git 发布
cd publish-tools && ./publish-git.sh hfs-v3 v0.1.0

# PyPI 发布
./publish-obfuscated.sh

Metadata

Release files for mp-hfs 3.0.5

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for mp-hfs 3.0.5
File Size Uploaded
mp_hfs-3.0.5.tar.gz 54.2 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for mp-hfs 3.0.5
File Interpreter ABI Platform
mp_hfs-3.0.5-py3-none-any.whl Python 3 none any Details

Total release size: 119.0 kB

Release files / mp_hfs-3.0.5.tar.gz

Download URL mp_hfs-3.0.5.tar.gz
Size 54.2 kB
Tags Source
SHA-256 checksum
How to use checksums
5ea43f4c1003ee81caa30cc5a0879482c90239b676f64fc06bba3d4ea4d60e8a
BLAKE2b-256 checksum
How to use checksums
cb86ea1186479e0a6a929e1f914fd85638cd45a27dc9e170cd43dacf84fea409
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.7

Release files / mp_hfs-3.0.5-py3-none-any.whl

Download URL mp_hfs-3.0.5-py3-none-any.whl
Size 64.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
7867584b2a6453d911008f11b31bdfc64ffae346dae5813744d1ad4baff4a97e
BLAKE2b-256 checksum
How to use checksums
deb8cdbef197a7039bbee57181efea1e03a29a737c439487f04353d321c28a37
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.2.0 CPython/3.13.7

Release history Release notifications | RSS feed

This release

3.0.5 This release

2 release files

3.0.4

2 release files

3.0.3

2 release files

3.0.2

2 release files

3.0.1

2 release files

3.0.0

2 release files

0.2.13

1 release file

0.1.21

2 release files

0.1.18

2 release files

0.1.17

2 release files

0.1.16

2 release files

0.1.15

2 release files

0.1.14

2 release files

0.1.12

2 release files

0.1.11

2 release files

0.1.10

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page