Skip to main content

Lightweight multi-storage routing and sync abstraction layer

Project description

SyncGate 🎯

轻量级多存储路由与同步抽象层

License: MIT Python 3.10+ Tests Coverage

🎯 产品定位

SyncGate 是轻量级多存储路由与同步抽象层,统一管理本地文件、HTTP 链接和多种云存储。

核心功能

  1. 🔗 统一路径 - 多种存储一个命令访问
  2. 🌳 虚拟文件系统 - 不移动源文件,只管理链接
  3. ✅ 链接验证 - 自动验证链接有效性
  4. 📊 监控统计 - 跟踪使用情况和健康评分
  5. 🔄 批量操作 - 批量创建/删除/验证
  6. ⚡ 错误处理 - 重试机制和熔断器

支持的存储

存储类型 状态 协议
本地文件 local:/path/to/file
HTTP/HTTPS https://example.com/file
AWS S3 s3://bucket/key
WebDAV webdav://server.com/path
FTP ftp://server.com/path
SFTP sftp://user@server.com/path

🚀 快速开始

安装

pip install syncgate

演示

python3 demo_complete.py

输出:

🎯 SyncGate Complete Demo
============================================================

📝 Step 1: 创建链接
  ✅ Local:  /docs/notes.txt
  ✅ HTTP:   /online/api.json
  ✅ S3:     /cloud/data.csv

📂 Step 2: 虚拟文件系统结构
virtual/
├── 📁 docs/  → local:/path/to/files
├── 📁 online/ → https://api.example.com
└── 📁 cloud/  → s3://my-bucket/data

🔍 Step 3: 链接验证
  ✅ /docs/notes.txt
  ✅ /online/api.json

✅ 演示完成!

使用

# 创建链接
syncgate link /docs/a.txt local:/path/to/file.txt local
syncgate link /online/api.json https://api.example.com/data.json http
syncgate link /cloud/data.csv s3://my-bucket/data/2024.csv s3

# 列出目录
syncgate ls /docs

# 树形结构
syncgate tree /

# 验证链接
syncgate validate /docs/a.txt

# 查看状态
syncgate status /docs/a.txt

📖 文档

🛠️ 功能模块

CLI 命令

命令 说明
link 创建链接
unlink 删除链接
ls 列出目录
tree 树形结构
validate 验证链接
status 查看状态

REST API

# 启动 API 服务器
python -m syncgate.api --port 8080
端点 说明
GET /api/stats 获取统计
GET /api/links 获取所有链接
POST /api/links 创建链接
DELETE /api/links/{path} 删除链接

高级功能

from syncgate.batch import BatchOperations
from syncgate.monitor import StatsCollector
from syncgate.error_handler import retry, CircuitBreaker

# 批量操作
batch = BatchOperations(vfs_root="virtual")
batch.batch_create([...])

# 监控统计
stats = StatsCollector(vfs_root="virtual")
stats.record_create("local")
health = stats.get_health_score()

# 错误处理
@retry(max_retries=3, delay=1.0)
def fragile_operation():
    ...

🧪 测试

# 运行所有测试
pytest tests/ -v

# 运行性能测试
python3 tests/benchmark_streaming.py

测试覆盖: 168+ 测试用例

📈 性能

场景 性能
初始化 (5000 文件) 0.04ms
首屏加载 2ms (50 条)
流式滚动 流畅无卡顿

🏗️ 开发

# 克隆项目
git clone https://github.com/cyydark/syncgate.git
cd syncgate

# 安装开发依赖
pip install -e ".[dev]"

# 运行演示
python3 demo_complete.py

# 运行测试
pytest tests/ -v

🤝 贡献

欢迎提交 Issue 和 PR!

反馈: GitHub Issues

📄 许可

MIT License - see LICENSE


SyncGate - 统一管理散落在各处的文件 📁🌐☁️

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

syncgate-1.0.0.tar.gz (91.1 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

syncgate-1.0.0-py3-none-any.whl (53.9 kB view details)

Uploaded Python 3

File details

Details for the file syncgate-1.0.0.tar.gz.

File metadata

  • Download URL: syncgate-1.0.0.tar.gz
  • Upload date:
  • Size: 91.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for syncgate-1.0.0.tar.gz
Algorithm Hash digest
SHA256 9abbd86687b79c19b9f08d8d77219cb8f5811bcbe15bad9b7df61c7c2ef1f300
MD5 9c131b16b25902d1f21248ba0d451fb1
BLAKE2b-256 ec51aa6dd823f21316b386124f7131523e63561f5b1f0149bbc24ee2d4aead19

See more details on using hashes here.

File details

Details for the file syncgate-1.0.0-py3-none-any.whl.

File metadata

  • Download URL: syncgate-1.0.0-py3-none-any.whl
  • Upload date:
  • Size: 53.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for syncgate-1.0.0-py3-none-any.whl
Algorithm Hash digest
SHA256 f745ca0f4fd161335fca771d4526ac645e057d17aafcf16567bbda73b7681291
MD5 29557c23b0870980522ad2c746d92ab2
BLAKE2b-256 981038f8b07b2cbe4ad0617e2ddd42c65b2162a27947c2f73c18772e7d8e3669

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page