SCOPE API

Unified DB access layer across SCOPE. It consolidates the fragmented data access patterns currently spread across scope-backend, DAL, and workflow components into one single library and provides an optional REST API, for use by the downstream components.

Tech stack

• Python 3.13+
• SQLAlchemy — ORM / database toolkit (SQLite & ClickHouse backends)
• clickhouse-connect — native ClickHouse driver
• FastAPI + Uvicorn — async HTTP server

Quick start

# 1. Create a virtual environment & install dependencies
uv venv
source .venv/bin/activate
uv sync

2. Run the server

export CLICKHOUSE_HOST=localhost
export CLICKHOUSE_PORT=8123
export CLICKHOUSE_USERNAME=scopeclickhouseuser
export CLICKHOUSE_PASSWORD=scopeclickhousepassword
export CLICKHOUSE_DATABASE=scope
uv run dev

The API will be available at http://127.0.0.1:8000.

OpenAPI specification docs are available at http://127.0.0.1:8000/docs.

Running tests

# Recreate a dedicated test environment from the project metadata
uv venv .venv_tests
source .venv_tests/bin/activate
uv sync --active --dev

# Run the test suites
python -m pytest tests/ -v

The uv sync --active --dev step installs both the main project dependencies and the test-only dependencies declared in pyproject.toml, including vendored wheel dependencies such as mce-core, mce-provider-native, and mce-provider-sdk.

No additional uv pip install commands are required for metric compute in local dev or CI when syncing with --group dev.

Vendored wheel maintenance (important for Docker builds)

The API image build uses Docker context: api, so uv.lock cannot contain sources outside that directory (for example ../mce/...). Keep [tool.uv.sources] and api/uv.lock pointing to api/vendor/*.whl paths.

When updating mce-core or MCE providers:

# From repo root
uv build mce/mce-core
uv build mce/mce-provider-native
uv build mce/mce-provider-sdk

cp mce/dist/mce_core-2.0.0rc1-py3-none-any.whl api/vendor/
cp mce/dist/mce_provider_native-0.1.0-py3-none-any.whl api/vendor/
cp mce/dist/mce_provider_sdk-0.1.0-py3-none-any.whl api/vendor/

uv lock --project ./api

Docker

Build the image

docker build -t scope-api -f Dockerfile .

or use docker compose to build as part of the full stack:

docker compose -f deploy/docker-compose.yml --build scope-api

Run the container standalone

docker run --rm -p 8000:8000 \
  -e CLICKHOUSE_HOST=host.docker.internal \
  -e CLICKHOUSE_PORT=8123 \
  -e CLICKHOUSE_USERNAME=scopeclickhouseuser \
  -e CLICKHOUSE_PASSWORD=scopeclickhousepassword \
  -e CLICKHOUSE_DATABASE=default \
  scope-api

The init script (data/clickhouse/otel_traces.sql) runs automatically on first startup, creating the scope database.

Test HTTP REST API

make test-rest-api

Using as a Python library

Besides running as a REST server, scope-api can be imported directly into any Python project as a library.

Quick example

from scope import LocalClient

with LocalClient() as client:
    apps     = client.get_application_names()
    sessions = client.get_sessions(app_name="my-app", limit=10)

Specifying database credentials

LocalClient accepts an optional db argument — a Connector instance that you configure with explicit credentials. When db is omitted, the backend is chosen automatically through the SCOPE_LOCAL environment variable (see Environment-variable mode below).

Option A — ClickHouse (via clickhouse-connect)

from scope import LocalClient
from scope.connectors.clickhouse import ClickHouseConnector

db = ClickHouseConnector(
    host="clickhouse.example.com",
    port=8123,
    username="scope_user",
    password="s3cret",
    database="scope",
)

with LocalClient(db=db) as client:
    apps = client.get_applications()

Option B — PostgreSQL (via SQLAlchemy)

from scope import LocalClient
from scope.connectors.sqlalchemy import SQLAlchemyConnector

# PostgreSQL
db = SQLAlchemyConnector(url="postgresql://user:pass@host:5432/scope")

with LocalClient(db=db) as client:
    sessions = client.list_sessions(limit=20)

Option C — Neo4j (for KG, metrics, and concepts operations)

KG, metrics, and concepts methods require a Neo4j-backed connector:

from scope import LocalClient
from scope.connectors.neo4j import Neo4JConnector

db = Neo4JConnector(
    host="neo4j.example.com",
    port=7687,
    username="neo4j",
    password="s3cret",
    database="neo4j",
)

with LocalClient(db=db) as client:
    state  = client.get_state_by_session_id("session-abc")
    metrics = client.get_session_metrics("session-abc", hops=2)
    concepts = client.get_app_concepts("my-app")

Environment-variable mode

When you create LocalClient() without a db argument, the backend is selected by the SCOPE_LOCAL environment variable:

Neo4j settings (used by the REST server's dependency injection) are read from:

Example:

export SCOPE_LOCAL=false
export CLICKHOUSE_HOST=clickhouse.example.com
export CLICKHOUSE_PORT=8123
export CLICKHOUSE_USERNAME=scope_user
export CLICKHOUSE_PASSWORD=s3cret
export CLICKHOUSE_DATABASE=scope

from scope import LocalClient

# Automatically connects to the ClickHouse instance above
client = LocalClient()

Available client methods

Type hints & return types

All methods return Pydantic models (e.g. TracesResponse, ApplicationsResponse, SpanDetailsResponse) defined in scope.models.otel_traces. Results are fully typed and serializable:

from scope import LocalClient

with LocalClient() as client:
    resp = client.get_applications()
    for app in resp.applications:
        print(app.applicationName, app.cost)

    # Serialize to dict / JSON
    print(resp.model_dump_json(indent=2))

For dependency injection or testing, type-hint against the abstract base class:

from scope import SCOPEAPIClient, LocalClient

def analyze(client: SCOPEAPIClient):
    ...

analyze(LocalClient())