mlflow-langsmith

Imports a LangSmith run into MLflow as a native MLflow trace. Run tree reconstructed as MLflow spans, source timestamps preserved, raw payload stored as an artifact, lookup tags written.

Use it when you want a LangSmith run to outlast LangSmith retention, sit next to your native MLflow runs in one UI, or act as a stable baseline for an eval suite.

Install

pip install mlflow-langsmith
# or
uv add mlflow-langsmith

Python 3.10, 3.11, 3.12, or 3.13. Depends on langsmith>=0.8, mlflow>=3.12, pydantic>=2.11.

Configure

LANGSMITH_API_KEY=...
MLFLOW_TRACKING_URI=http://127.0.0.1:5000
LANGSMITH_PROJECT=my-project            # optional default for list-langsmith-runs
MLFLOW_EXPERIMENT_NAME=agent-baselines  # optional default for imports

CLI flags and Python kwargs override the env.

CLI

Find a recent LangSmith run:

mlflow-langsmith list-langsmith-runs --limit 10 --success-only

Import it into MLflow:

mlflow-langsmith import-run \
    --langsmith-run-id 019e5a23-a0b8-7491-848f-b251a011e23a \
    --mlflow-experiment-name agent-baselines \
    --baseline-key nightly-regression

Look it back up later:

mlflow-langsmith list-imports  --mlflow-experiment-name agent-baselines
mlflow-langsmith show-import   --langsmith-run-id 019e5a23-a0b8-7491-848f-b251a011e23a

The import shows up in the MLflow UI as a normal trace.

Python

from mlflow_langsmith import import_langsmith_run

result = import_langsmith_run(
    "019e5a23-a0b8-7491-848f-b251a011e23a",
    mlflow_experiment_name="agent-baselines",
    baseline_key="nightly-regression",
    tags={"owner": "eval-platform"},
)

print(result.mlflow_run_id, result.mlflow_trace_id)

Already have a LangSmith Run object (e.g. inside your own eval harness)? Skip the second API call:

from mlflow_langsmith import import_langsmith_run_object
result = import_langsmith_run_object(run, mlflow_experiment_name="agent-baselines")

Resolve a prior import by baseline key. No LangSmith call needed:

from mlflow_langsmith import find_imported_run, load_imported_payload

ref = find_imported_run(baseline_key="nightly-regression")
payload = load_imported_payload(ref.mlflow_run_id)

What gets written

One MLflow run plus one MLflow trace per import, in your chosen experiment.

Spans: LangSmith chain, llm, chat_model, tool, retriever, parser, embedding, agent runs map to the matching MLflow span types. Parent-child relationships and start/end timestamps come from LangSmith, so step durations are real and not import wall-clock time.

Tags: source run id, source trace id, source project, baseline key. Indexable in MLflow search, and what find_imported_run queries against.

Artifacts under langsmith/:

• raw-run.json (full LangSmith payload, audit copy)
• normalized-run-tree.json
• span-manifest.json
• import-summary.json
• messages.json, tool-calls.json, token-usage.json, attachments-manifest.json when present

Per-span attributes: token usage, model name, model provider, cost, status. Pulled from LangSmith fields where they exist.

Re-imports

duplicate_mode="skip" is the default. Importing the same LangSmith run twice returns the existing MLflow run. Pass "raise" to fail, "import" to force a new MLflow run.

Sensitive data

Imported artifacts contain the same prompts, responses, tool inputs, tool outputs, and metadata as the original LangSmith trace. Don't import into an MLflow store you wouldn't keep that data in.

For contributors

src layout. uv, pytest, ruff, mypy, pre-commit. CI runs the test suite on Python 3.10, 3.11, 3.12, 3.13.

uv sync
uv run pytest
uv run mypy src
uv run ruff check .

Local MLflow server (sqlite:///experiments/mlflow/mlflow.db):

bash scripts/start_mlflow_dev.sh

End-to-end against a real LangSmith run:

uv run --env-file .env mlflow-langsmith import-run --langsmith-run-id <RUN_ID>

Deeper docs:

• docs/langsmith-to-mlflow-trace-strategy.md: span mapping, timestamp handling, normalization rules
• docs/local-mlflow-dev.md: local tracking store layout
• docs/pypi-release.md: TestPyPI / PyPI release workflow