docmesh-kbms
문서 원본과 메타데이터는 dms-core가 관리하고, 이 프로젝트는 문서 내용을
추출·chunking·embedding하여 Milvus에 저장하는 지식 저장 관리 시스템이다.
외부 공개 진입점은 kbms.KnowledgeManagement 하나다.
계층
- Domain: Python
- Document management: dms-core
- ORM/RDB: SQLAlchemy with SQLite or PostgreSQL
- Vector store: Milvus
- Embedding provider: Ollama
KnowledgeManagement의 문서 작업은 모두 async API이며 await로 호출한다.
upload_document()은 dms-core에 원본 문서와 메타데이터를 저장한 뒤,
텍스트 문서만 추출·chunking하여 Ollama/Milvus로 지식화한다. SQLAlchemy engine과
dms-core, Ollama, Milvus client의 생성·종료 및 commit/rollback은 host가 맡는다.
비동기 경로를 사용하려면 AsyncEngine(sqlite+aiosqlite 또는 async PostgreSQL),
Ollama AsyncClient, pymilvus AsyncMilvusClient를 주입한다. 이 경우
dms-core의 AsyncDocumentManagementSDK와 async SQLAlchemy/Milvus 호출을 사용하며,
동기 클라이언트를 주입하면 기존 호환 경로를 사용한다.
host 권한 정책은 생성자에 access_policy로 주입하고, 각 문서 작업에는 dms-core의
AccessContext를 전달해 사용자·그룹 권한을 적용한다.
검증
uv run pytest -q
uv run ruff check kbms test_kbms
SQLite memory/disk와 Milvus local 검사는 항상 실행된다. PostgreSQL, Milvus server, Ollama 연결 검사는 다음 환경변수가 설정된 경우 실행된다.
export KBMS_POSTGRES_URL='postgresql+psycopg://docmesh:postgres@postgres:5432/kbms'
export KBMS_MILVUS_URI='http://milvus:19530'
export KBMS_OLLAMA_HOST='http://192.168.219.106:11434'
export KBMS_OLLAMA_EMBEDDING_MODEL='bge-m3'
export KBMS_OLLAMA_GENERATION_MODEL='llama3.2'
uv run pytest -q -m integration
실환경 전체 흐름(DMS + MinIO + PostgreSQL + Ollama + Milvus)은 다음 테스트가 검증한다. MinIO 설정을 생략하면 devcontainer 기본값을 사용한다.
export KBMS_MINIO_ENDPOINT='milvus-minio:9000'
export KBMS_MINIO_ACCESS_KEY='minioadmin'
export KBMS_MINIO_SECRET_KEY='minioadmin'
export KBMS_MINIO_BUCKET='kbms-e2e'
uv run pytest -q -m real_integration test_kbms/test_real_integration.py
1 maintainer
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file docmesh_kbms-0.1.0.tar.gz.
File metadata
- Download URL: docmesh_kbms-0.1.0.tar.gz
- Upload date:
- Size: 12.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
022a7ac33619ad7491f989537016354729496327db2ca302ac036921dc94b011
|
|
| MD5 |
749b07e01db52d27e07c283dc75a0fbc
|
|
| BLAKE2b-256 |
6d4e2b812778549131fbc406e441c0309852fe607fab1f09c4cd1645c3b67748
|
Provenance
The following attestation bundles were made for docmesh_kbms-0.1.0.tar.gz:
Publisher:
python-publish.yml on kyundae-kim/docmesh-kbms
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
docmesh_kbms-0.1.0.tar.gz -
Subject digest:
022a7ac33619ad7491f989537016354729496327db2ca302ac036921dc94b011 - Sigstore transparency entry: 2732493630
- Sigstore integration time:
-
Permalink:
kyundae-kim/docmesh-kbms@3337d80ae95f666c6a38323430ac148776d83ef6 -
Branch / Tag:
refs/tags/v0.1.0 - Owner: https://github.com/kyundae-kim
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
python-publish.yml@3337d80ae95f666c6a38323430ac148776d83ef6 -
Trigger Event:
release
-
Statement type:
File details
Details for the file docmesh_kbms-0.1.0-py3-none-any.whl.
File metadata
- Download URL: docmesh_kbms-0.1.0-py3-none-any.whl
- Upload date:
- Size: 15.7 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via:
twine/7.0.0 CPython/3.13.14
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
ca01656d6c0bec8f36b8a59a23c439ad120ff5ad46083161a6bf0b54d6463f0c
|
|
| MD5 |
a745bac3a1e9ad5009e80cc1b318bfd3
|
|
| BLAKE2b-256 |
e9e2f5e8aead84944a13e32aabc7aa5971ac735f2ee9f9a6435967b6352320fb
|
Provenance
The following attestation bundles were made for docmesh_kbms-0.1.0-py3-none-any.whl:
Publisher:
python-publish.yml on kyundae-kim/docmesh-kbms
-
Statement:
-
Statement type:
https://in-toto.io/Statement/v1 -
Predicate type:
https://docs.pypi.org/attestations/publish/v1 -
Subject name:
docmesh_kbms-0.1.0-py3-none-any.whl -
Subject digest:
ca01656d6c0bec8f36b8a59a23c439ad120ff5ad46083161a6bf0b54d6463f0c - Sigstore transparency entry: 2732493690
- Sigstore integration time:
-
Permalink:
kyundae-kim/docmesh-kbms@3337d80ae95f666c6a38323430ac148776d83ef6 -
Branch / Tag:
refs/tags/v0.1.0 - Owner: https://github.com/kyundae-kim
-
Access:
public
-
Token Issuer:
https://token.actions.githubusercontent.com -
Runner Environment:
github-hosted -
Publication workflow:
python-publish.yml@3337d80ae95f666c6a38323430ac148776d83ef6 -
Trigger Event:
release
-
Statement type:
