Chumoli

O‘zbekiston ma’lumot manbalari uchun engil EL vosita
dlt (Apache 2.0) ustida · UI + CLI

SQL, REST, fayl/S3, MoySklad, Bitrix24 va O‘zbekiston manbalaridan (Click, Payme, Uzum, Didox — beta) ma’lumotni olib DuckDB, PostgreSQL, lokal fayl yoki S3 ga yuklang. Brauzerda boshqaring yoki CLI orqali ishga tushiring.

Tez boshlash

Talab: Python 3.11+

pip install chumoli
chumoli ui

• Dashboard: http://127.0.0.1:8000/
• API docs: http://127.0.0.1:8000/api/docs
• To‘xtatish: Ctrl+C

Brauzersiz: chumoli ui --no-open yoki chumoli-api.

Ishlab chiqish (repo)

git clone https://github.com/farrux05-ai/chumoli.git
cd chumoli
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev]"
chumoli ui

Birinchi pipeline

1. + Yangi yoki Ulagichlar
2. Manba tanlang → ulanishni tekshiring → saqlang
3. Run → Preview

UI dagi demo: SQL → DuckDB, REST (CBU valyuta kurslari) → DuckDB, Volume (sintetik).

Nima bor (v0.1)

UZ connectorlar — beta

Click, Payme, Uzum Market va Didox kod bazasida bor va UI da beta deb belgilangan. Ular hali barcha merchant/sandbox muhitlarida to‘liq tasdiqlanmagan. Ishlatishdan oldin o‘z API kalitingiz bilan sinang; xato topsangiz Issues oching.

Destinationlar

Lokal fayl va S3 katalogda alohida. dlt _dlt_* metadata yashirin staging da qoladi; foydalanuvchi papkasiga faqat toza jadvallar chiqadi. Preview: papka yo‘li, fayllar, CSV namunasi.

Ma’lumot qayerda

~/.chumoli/                 # tizim (yashirin)
  chumoli_control.db
  master.key
  api.key
  pipelines/                # dlt state
  fs_staging/               # dlt filesystem + _dlt_* (yashirin)

~/chumoli-data/             # foydalanuvchi ko‘radigan
  <pipeline>.duckdb
  warehouse.duckdb          # path ko‘rsatilmagan DuckDB uchun fallback
  examples/
  exports/<pipeline>/       # toza CSV/Parquet

export CHUMOLI_HOME=/var/lib/chumoli
export CHUMOLI_DATA=/var/lib/chumoli-data
chumoli ui

master.key ni control DB bilan birga zaxiralang — kalitsiz secrets ochilmaydi.

CLI

chumoli --help
chumoli ui                 # dashboard + brauzer
chumoli ui --port 8080
chumoli list
chumoli run <pipeline>

Docker

docker compose up --build
# http://127.0.0.1:8000/

• Bind: 127.0.0.1:8000 (tashqi tarmoq uchun reverse proxy + CHUMOLI_API_KEY + TLS)
• Volume: chumoli_data → /data (CHUMOLI_HOME + CHUMOLI_DATA)
• Bitta uvicorn worker — scheduler va run lock shu processda

Muhit o‘zgaruvchilari

Telegram: dashboard Sozlamalar → bot token + chat id.

Ishlab chiqish

pip install -e ".[dev]"
PYTHONPATH=src python -m pytest tests/ -q

CI (GitHub Actions): har push/PR da Python 3.11 va 3.12 da testlar + Docker image build.

Litsenziya

Apache-2.0 · GitHub