Skip to main content

Augur (de-augur)

점술관. 자연어를 SQL로 옮기되, 근거(스키마)를 대고, 틀리면 왜 틀렸는지 센다.

Parquet 레이크에 자연어로 질문한다. 스키마 RAG(테이블·컬럼·샘플값 검색) → LLM → DuckDB 실행. 모든 호출은 trace(JSONL)로 남고, augur eval은 골든 SQL과 실행 결과를 대조해 실패 모드를 자동 분류한다. 벡터 DB 없음, SDK 없음(httpx), 임베딩 없음(어휘 검색). 필요해지면 retrieve.Scorer 구현체 하나를 추가한다.

question ─▶ retrieve(catalog) ─▶ prompt ─▶ LLM ─▶ SQL ─▶ guard(SELECT only) ─▶ DuckDB
                │                                            │ error? → repair ×1
                └── SchemaDoc (table, path, columns, samples) └──────────────▶ Trace

사용

pip install de-augur
augur auth set anthropic                 # 숨김 입력 → ~/.config/augur/credentials.json (0600)
augur auth status                        # env | file | none — 값은 앞 4자리만

augur index eval/sources.yaml            # {tables: {orders: './data/orders/*.parquet'}} → catalog.json
augur ask "How many orders are paid?"    # SQL은 stderr, 결과는 stdout(jsonl)
augur eval eval/cases.yaml               # 실패 모드별 집계 + Parquet/JSONL 기록
augur report                             # run 간 추이
gladius query "SELECT failure_mode, count(*) FROM './data/augur/eval/*.parquet' GROUP BY 1"

키는 ANTHROPIC_API_KEY/OPENAI_API_KEY 환경변수 → augur auth set으로 저장한 파일 → (TTY라면) 실행 시 숨김 프롬프트 순으로 찾는다. 인자로는 받지 않는다 — 셸 히스토리에 남는다. 파일 위치는 AUGUR_HOME으로 바꿀 수 있다. 키는 trace·로그·repr에 절대 찍히지 않는다.

실패 모드 (자동 분류, 단계 순)

모드 판정 원인 층
RETRIEVAL_MISS 케이스의 tables가 검색 결과에 없음 retrieve
NON_SELECT 쓰기 문장 — 실행 전 차단 prompt/model
UNANSWERABLE 모델이 답 불가 선언(정답은 answerable) prompt/model
HALLUCINATED_TABLE 카탈로그에 없는 경로 참조 prompt/model
HALLUCINATED_COLUMN DuckDB "not found in FROM clause" schema render/model
EXEC_ERROR 그 외 실행 실패(문법 등) model
WRONG_SHAPE 실행됐지만 컬럼 수/행 수가 골든과 다름 semantic (집계 축·필터)
WRONG_RESULT 모양은 같은데 값이 다름 semantic (조건·리터럴)
CORRECT 집합 비교 일치 —

앞 단계 실패가 뒷 단계 실패를 만든다. 그래서 RETRIEVAL_MISS가 결과 일치보다 우선한다 — "우연히 맞은 것"은 개선 신호가 아니다.

케이스 파일

cases:
  - id: l3-001
    question: Total paid revenue per product name.
    golden_sql: SELECT ... JOIN ...
    tables: [orders, products]     # 검색이 반드시 맞혀야 하는 테이블
    answerable: true               # false면 UNANSWERABLE이 정답
    notes: 자유 텍스트 (수동 라벨, 함정 설명)

동봉 eval/cases.yaml 34건 = L1 단일 집계 · L2 필터/그룹 · L3 JOIN · L4 시간 · L5 함정 (답 불가, 없는 리터럴, DML 유도, 한국어). eval/make_fixture.py가 seed 고정 합성 데이터를 만든다.

하지 않는 것

  • ORDER BY 의미 검증(집합 비교만). 순서가 답인 케이스는 notes에 표시.
  • 스키마 수백 개 규모의 검색 품질 보장 — 어휘 검색의 한계이며 측정 대상.
  • 자동 재시도 2회 이상. 1회 repair로 못 고치면 실패로 센다.

Metadata

Release files for de-augur 0.5.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for de-augur 0.5.3
File Size Uploaded Provenance Actions
de_augur-0.5.3.tar.gz 28.6 kB With provenance Details for de_augur-0.5.3.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for de-augur 0.5.3
File Interpreter ABI Platform Actions
de_augur-0.5.3-py3-none-any.whl Python 3 none any Details for de_augur-0.5.3-py3-none-any.whl

Total release size: 55.4 kB

Release files / de_augur-0.5.3.tar.gz

Download URL de_augur-0.5.3.tar.gz
Size 28.6 kB
Tags Source
SHA-256 checksum
How to use checksums
2322523a90240ce621b1e28aca645d06be8d52182f62095ec01e6429fc1ea631
BLAKE2b-256 checksum
How to use checksums
504182b4e95378b60ef80faa08656ccc041dc8e47fdd457653df6fb04e12bbf3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / de_augur-0.5.3-py3-none-any.whl

Download URL de_augur-0.5.3-py3-none-any.whl
Size 26.8 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b939c147c19cfc32833edfab91726450c27ab048fd3f96484977cc60e8b9b919
BLAKE2b-256 checksum
How to use checksums
aee3400187c7134c13ddf7cea9fd1878013871d2b21194dd27ef85935cc2f68a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Security for de-augur 0.5.3

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for de_augur-0.5.3.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page