softgnn-advisor 0.1.5

Graph-guided, runtime-proven, LLM-assisted PR test generation for Python projects

SoftGNN Advisor

Graph-guided, runtime-proven, LLM-assisted PR testing

Know what changed. Know what tests hit it. Generate what is missing.

---

Install

Recommended for CLI use:

pipx install softgnn-advisor

Or install into your current environment:

pip install softgnn-advisor

Optional extras:

pip install "softgnn-advisor[llm]"   # Gemini / OpenAI-compatible generation
pip install "softgnn-advisor[gnn]"   # PyTorch Geometric ranking
pip install "softgnn-advisor[all]"   # full stack

Then run:

softgnn setup /path/to/your-repo --project my-app
softgnn apply --project my-app

For local development from source:

git clone https://github.com/minhquang0407/softgnn-advisor.git
cd softgnn-advisor
python -m venv .venv
.venv\Scripts\activate  # Windows
pip install -e ".[all]"
softgnn --help

---

What is SoftGNN?

SoftGNN Advisor is an experimental CLI that combines a code graph, runtime test graph, and LLM test generation to help you understand PR impact and generate missing pytest tests.

Most AI testing tools stop at:

read changed file -> ask LLM for tests -> run pytest

SoftGNN aims for a stronger loop:

scan PR -> find impacted code -> map tests that actually execute it -> generate missing tests -> verify -> refresh runtime proof

Core thesis: a generated test is not truly useful until it passes pytest and proves it hits the intended code.

---

The pipeline

flowchart LR
    A[PR Diff] --> B[Code Graph]
    B --> C[Impact Scan]
    C --> D[Missing Runtime Coverage]
    D --> E[LLM Semantic Test Generation]
    E --> F[Schema + Safety Validation]
    F --> G[Transactional Patch]
    G --> H[Pytest Verify]
    H --> I[Runtime Test Mapping]
    I --> J[PR Scan Confirmation]

    H -- failure --> K[LLM Repair]
    K --> F

---

Why it is different

Capability	Naive LLM test generation	SoftGNN Advisor
Reads changed code	✅	✅
Generates pytest tests	✅	✅
Validates structured LLM output	❌	✅
Patches transactionally	❌	✅
Rolls back failed generated tests	❌	✅
Maps tests to functions at runtime	❌	✅
Confirms PR coverage after generation	❌	✅
Supports Gemini/OpenAI-compatible LLMs	varies	✅

---

Features

• PR impact scanning between Git revisions.
• Code graph extraction from Python source files.
• Runtime test mapping from pytest execution to source functions.
• Missing runtime coverage detection for impacted functions.
• LLM-assisted semantic pytest generation.
• Native Gemini provider and OpenAI-compatible provider.
• Structured JSON validation before writing tests.
• Safety validation against unsafe generated code patterns.
• Transactional patching with generated block markers.
• Pytest verification and bounded generated-test repair loop.
• Runtime refresh after successful generation.
• PR scan confirmation after runtime refresh.

---

Verified demo

On a local social-link-prediction repo, SoftGNN used Gemini to generate behavior tests for:

FUNC:is_edge_index_sorted

Result:

pytest: 6 passed
runtime mode: per-test
runtime edges: 336
persisted: True
missing coverage before: 0
missing coverage after: 0

Fallback without an LLM produced only a shallow smoke test:

assert callable(is_edge_index_sorted)

Gemini-assisted generation produced behavior checks for sorted edges, unsorted source order, unsorted target order, single-edge input, and invalid-shape errors.

Read the full demo: docs/examples/social-link-demo.md

---

Install

git clone https://github.com/minhquang0407/softgnn-advisor.git
cd softgnn-advisor
python -m venv .venv

Windows PowerShell:

.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Linux/macOS:

source .venv/bin/activate
pip install -r requirements.txt

PyTorch / PyTorch Geometric installs can be platform-specific. If installation fails, follow the official PyTorch and PyG installation guides for your environment.

---

Configure an LLM provider

Gemini

$env:SOFTGNN_LLM_PROVIDER="gemini"
$env:SOFTGNN_LLM_MODEL="gemini-3-flash"
$env:SOFTGNN_LLM_API_KEY="YOUR_GEMINI_API_KEY"

If your API account uses another model ID:

$env:SOFTGNN_LLM_MODEL="gemini-2.5-flash"

OpenAI-compatible endpoint

$env:SOFTGNN_LLM_PROVIDER="openai-compatible"
$env:SOFTGNN_LLM_BASE_URL="http://localhost:11434/v1"
$env:SOFTGNN_LLM_MODEL="qwen2.5-coder:7b"
$env:SOFTGNN_LLM_API_KEY="optional-if-your-endpoint-needs-it"

Generation strategies:

template  -> deterministic templates only
llm       -> require configured LLM unless fallback is allowed
auto      -> try LLM first, fallback to templates when unavailable

---

Quickstart

After setup, a single command runs the full workflow:

python softgnn.py setup C:\repo\my-app
python softgnn.py apply --project my-app

apply does everything automatically:

detect changes (git / filesystem snapshot / full-scan)
run pr-scan
rank missing coverage targets
generate tests (LLM or template)
patch test files
run pytest
repair if failing
rollback if still failing
run runtime map
run post-scan confirmation

Nothing is modified unless pytest passes.

Want to review proposed tests before patching? Use plan first:

python softgnn.py setup C:\repo\my-app
python softgnn.py plan --project my-app   # review output
python softgnn.py apply --project my-app  # apply reviewed plan

Template-only generation (no LLM):

python softgnn.py apply --project my-app --strategy template

Advanced commands are still available (prepare, pr-scan, generate-tests, test-map).

Mental model:

setup/prepare need the repo path once
everyday commands use --project

Daily commands after setup:

Goal	Command
Generate + verify tests	python softgnn.py apply --project my-app
Review before patching	python softgnn.py plan --project my-app
Inspect change impact	python softgnn.py scan --project my-app
Runtime test map	python softgnn.py map --project my-app
Health check	python softgnn.py doctor --project my-app
Impact of one symbol	python softgnn.py impact --project my-app FUNC:foo
Developer triage	python softgnn.py triage --project my-app "bug description"

More details: docs/quickstart.md

The full guide covers:

simple CLI workflow
plan cache and apply-from-plan
Git PR workflow
no-Git filesystem snapshot workflow
first-run full-scan workflow
explicit target workflow
runtime coverage mapping
patch/verify/repair/rollback flow

---

Safety model

SoftGNN is conservative by default:

writes tests/ only
wraps generated code in markers
validates LLM output before patching
runs pytest before accepting generated tests
rolls back failed generated edits by default
never requires committing API keys

Generated test blocks are marked:

# <softgnn-generated target="FUNC:example" start>
...
# <softgnn-generated target="FUNC:example" end>

Recommended workflow:

run on a feature branch
start with --mode plan
use --mode patch after review
inspect git diff before commit

---

CLI highlights

python softgnn.py pr-scan --project social-link --repo-path "C:\path\to\repo" --base main --head HEAD

python softgnn.py test-map --project social-link --repo-path "C:\path\to\repo" --mode per-test --persist

python softgnn.py generate-tests --project social-link --repo-path "C:\path\to\repo" --mode plan --generation-strategy auto

---

Project status

Current release: v0.1.0-alpha

This is a developer preview. Generated tests should be reviewed before commit. Production-code fixes are intentionally out of scope for v0.1.

---

Roadmap

Short version:

M4  Runtime-Proven Test Generation
M5  Graph Impact Report / Dashboard
M6  Learned Test Prioritization / GNN Ranking
M7  Multi-Agent Quality Swarm
M8  Large-scale repo automation
M9  Controlled production-code fixes

Read more:

• ROADMAP.md
• docs/future-milestones.md

---

License

MIT License. See LICENSE.