Fabric-RLM

Run verifiable data tasks inside Microsoft Fabric.

fabric-rlm gives a model a Python workspace next to your Fabric data. It can read Lakehouse files, analyze Delta tables, query Power BI semantic models, calculate results, and write new artifacts. Output contracts and validators decide whether the work is accepted. Failed checks go back into the run for another attempt.

The runtime uses a CPython subprocess inside the notebook session. pandas, DuckDB, Polars, openpyxl, PyMuPDF, and other installed packages remain available. Large files stay on disk. The model sees previews, summaries, and computed results rather than every raw byte.

Quick start in Fabric

Install fabric-rlm in a Fabric notebook. The Python notebook experience is recommended for this example.

%pip install "fabric-rlm[analytics]"

After installation, restart the session. The example below uses the roughly 140 MB, 1.57-million-row IMF CSV created by the flagship notebook. You can replace it with another large CSV in Lakehouse Files.

from fabric_rlm import FabricLM, File, RLM

result = RLM.task(
    task="""
    Analyze the complete IMF price dataset. Inspect its schema and confirm the
    dimensions used. Select the monthly, all-items, year-over-year CPI series:
    INDEX_TYPE='CPI', COICOP_1999='_T',
    TYPE_OF_TRANSFORMATION='YOY_PCH_PA_PT', and FREQUENCY='M'. For the latest
    complete year, calculate average inflation by country and return the 10
    countries with the highest average. Report the matched row count, source
    columns, and filters applied.
    """,
    inputs={
        "prices": File("/lakehouse/default/Files/imf_cpi.csv"),
    },
    outputs={
        "year": int,
        "top_countries": list,
        "rows_analyzed": int,
        "source_columns": list,
        "filters_applied": list,
    },
    lm=FabricLM("gpt-5.1"),
    skills=["data_exploration"],
    max_turns=8,
).run()

print(result.top_countries)
print(result.filters_applied)

FabricLM uses the model endpoint available to the Fabric capacity. The notebook identity supplies authentication. There is no API key to place in the notebook and no separate Azure OpenAI resource to configure. You can see the list of supported models here. You can also use models from OpenAI, Anthropic, Foundry, and OpenRouter through LiteLLM.

Why this works beyond the context window

A CSV with 1.57 million rows cannot be placed in a model prompt. The File input passes its Lakehouse path into the Python worker instead. The model can write DuckDB, Polars, or pandas code to inspect the schema, filter rows, and calculate aggregates. Only bounded execution feedback, such as schema details, previews, aggregates, and errors, enters the next model call. The raw dataset does not enter the model prompt.

This pattern also works with wide Excel workbooks, Parquet files, JSONL streams, PDFs, and combinations of those sources. File size is constrained by the libraries and compute available in the notebook session rather than the model's context window.

What happens during a run

1. You provide a task, named inputs, and an output contract.
2. The model writes Python for the task.
3. The Python runs in a persistent subprocess with access to mounted Lakehouse files and installed packages.
4. Execution output returns to the model. It can inspect results and revise the code.
5. The run calls SUBMIT(...).
6. Type checks, skill verifiers, and your validators inspect the submission.
7. A rejected submission returns with specific repair feedback.
8. An accepted submission becomes an RLMResult with the payload and full trajectory.

flowchart LR
    A["Task + Fabric data"] --> B["Model writes Python"]
    B --> C["CPython subprocess"]
    C --> D["Inspect and revise"]
    D --> E["SUBMIT"]
    E --> F{"Contract passes?"}
    F -- "No" --> D
    F -- "Yes" --> G["Result + trajectory"]

Why Fabric developers use it

Work with Fabric data in place

Bind individual Lakehouse files with File(...), discover Delta tables and Files with LakehouseSource(...), publish generated files with FileDestination(...), or connect Power BI semantic models with SemanticModel(...). A single task can combine these handles. DAX runs in the tabular engine, Delta reads honor the transaction log, file processing runs in Python, and generated artifacts can be written back to Files/ without giving the isolated worker OneLake credentials.

In Fabric Jupyter runtimes where SemPy's automatic token service is unavailable, opt into refreshable user-identity authentication without moving the token into the worker payload:

model = SemanticModel(
    "<semantic-model-id>",
    workspace="<workspace-id>",
    credential_provider="notebookutils",
)

This calls notebookutils.credentials.getToken("pbi") in the process that uses the model. The token itself is never serialized.

Learn a reusable, source-bound package when the same approved sources support multiple tasks:

from fabric_rlm import FabricLM, File, RLM, load_knowledge

knowledge = RLM.learn(
    sources={
        "orders": File("/lakehouse/default/Files/orders.parquet"),
    },
    store="/lakehouse/default/Files/knowledge/sales.json",
)

result = RLM.task(
    "Revenue by region for the latest complete month",
    knowledge=knowledge,
    outputs=["answer"],
    lm=FabricLM("gpt-5.1"),
).run()

knowledge = load_knowledge(
    "/lakehouse/default/Files/knowledge/sales.json",
    sources={
        "orders": File("/lakehouse/default/Files/orders.parquet"),
    },
)

RLM.learn(...) deterministically profiles bounded source metadata, keeps runtime paths and authorization handles outside the persisted package, and can save locally or to a canonical OneLake abfss://.../Files/... path. Loading requires fresh, exact source aliases and rejects source drift before binding. Every knowledge-enabled task preflights the current sources before the model is called. For a SemanticModel, learning also registers one bounded semantic_model.measure.v1 capability from visible measures and columns. The RLM may select that operation through a strict scalar JSON plan; the host validates the allowlisted measure, group-by, and up to three filters, executes SemanticModel.measure(...), audits row/column/byte bounds, and gives the model only the compact fingerprinted result packet for synthesis. The model never supplies arbitrary DAX. CSV, Parquet, and Delta profiles also register a compiler-owned tabular.aggregate.v1 operation. Exact Lakehouse Delta catalogs register bounded aggregate operations, plus a two-fact operation that pre-aggregates each fact at the shared key grain before joining. The model selects only typed scalar parameters; it never supplies SQL or file-reader expressions. Inexact Lakehouse file catalogs and stale source snapshots fail closed rather than entering registered execution.

Reviewing a Fabric Data Agent

The review is experimental. It lives under fabric_rlm.experimental because its questions, grading and suggestions are still being refined against real agents. The sweep, the brief, the reports and the KPIs do not depend on it; what they share with it, the modelling of a source (schemas, the tables an agent selected, keys and time columns, measure columns, joins, attribute paths, vocabulary), is fabric_rlm.source_model. The old import path fabric_rlm.data_agent_review keeps working and resolves every name to its new home.

fabric_rlm.experimental.data_agent_review turns the same machinery on a Data Agent: read its sources, instructions, descriptions and few-shots; profile the sources with RLM.learn; check the setup against the documented guidance (schema names belong in data-source instructions, references must exist, definitions must not conflict between levels, descriptions drive routing, instructions truncate past about 4,600 characters); generate questions from the schemas, phrased the way a business user asks them with the words the agent's own instructions use, across the skills an author wants tested (aggregation, ranking, change, counts, a filter, a KPI from the definitions, the right measure column, an ambiguous total, a channel abbreviation, an out-of-scope topic the agent should decline), plus the driver-based questions a real user asks once a discovery step has pulled real names and periods from the data (why the measure dropped between two months and which products drove it, how a named reseller performs over time and on a category, shares of the total, a count above a threshold, churned entities, a comparison of two entities, the leading category per territory), the time expressions users write (the same month last year, the month before, last month, the week after Thanksgiving, winter, year to date, the last 30 days, a quarter; a relative period read another way is right when the answer says which dates it took), and a check of whether the answers follow the instructions' own rules (stating the period and the channel, rank and trend formats, currency format, the partial-year caveat, no personal data, the join and calendar rules), and compute each reference by executing a query against the source. What moved in the data is a separate tool over the source itself: fabric_rlm.reports.report takes a lakehouse or a semantic model and a question in plain words ("trend of revenue by product category", "why did reseller sales fall in December 2013 by territory", "top movers by customer year over year", "weekly recap", "what changed in July 2025 for product xyz and was it in line with the trend"), reads it against the source's own vocabulary (the kind of report; the measure; the groupings after "by", "per" or "which"; the periods; a filter on any value the source holds, looked up before it is applied and pushed into every query: "for tiktok", "where customer state = SP", "for the Night shift"; and a check of the named month against the trend and season of each measure when the question asks whether it was in line), measures every figure with the source's engine (SQL over OneLake, DAX for a model), recomputes each reported figure with an independent query, and renders a dashboard with trend, waterfall and driver scatter charts (examples/notebooks/rlm_what_moved.ipynb). The page lists how the question was read, the filter with its row count, and the words that named nothing; a value that matches nothing or too many things is said, not silently widened. No model touches the numbers. fabric_rlm.brief.brief(source, ["revenue by region", "orders"]) is the Monday Morning Brief: last week for the metrics you name, against the week before, the same week last year, the recent averages and the seasonal expectation, with level shifts, the drivers of the move, the volume and rate split, the day-of-week pattern and what moved together. kpis=["new customers", "churned customers over 4 weeks", "average order value = sales amount / order quantity", "top 3 share of revenue by reseller"] adds KPIs built from the source's structure: the entity is discovered and its choice explained, and every definition is printed next to its number. It also asks the data what nobody would think to put in the instructions (keys with no match in their dimension, dimension keys that repeat, values spelled in several cases in a case-sensitive lakehouse, the vocabulary of small attributes so "bikes" maps to the Bikes category, partial years, negative measures, snowflaked join paths, personal data on joined tables) and proposes each as an instruction line the author confirms or turns into a data fix; ask the agent the same questions through its Responses endpoint, whose run steps carry the query it executed and the source it routed to, grade that query or its prose, classify the failures (misrouting included on multi-source agents); and render suggested instructions, descriptions and few-shots into a report. examples/notebooks/rlm_data_agent_review.py is the Fabric Python notebook (runtime 3.12) that runs the whole loop; applying the suggestions to the agent's draft stage is its last, explicit cell. The notebook scopes a lakehouse to the tables the agent has selected (read from the datasource's elements tree, and resolved under either OneLake layout, with or without schemas) and raises the profile limits, since a lakehouse catalog is many tables rather than one file. A ReviewContext carries what the reviewer knows beyond the agent's configuration: the scope in plain words, priorities that order the questions, definitions declared to the RLM, the reviewer's own questions with a query or an answer as ground truth (graded first), and notes every RLM task receives. When nothing can be evaluated, the report says why (year discovery, fact tables, the time axis). The generator works from the shape of the data, not from one sample's names: a fact is a table with measures and a time axis, the time axis is a date dimension with a year column, a date or timestamp column on the fact, or one on a joined header table (order lines through their order), joins follow key and id columns to the table that carries them (irregular plurals and prefixed dimension tables included), grouping columns may sit on the fact itself, an integer or text date and a period written as text (2024/Q1) are time axes too, and when the column names say nothing the column types the profile recorded decide what is a measure, a time column or a grouping column. The generator was checked blind, with no agent and no instructions, on a SaaS schema, a bakery chain, a flat retail file and ARR tables keyed by quarter, as well as on AdventureWorks. Topics the agent's instructions put out of scope are not asked about, and a decline on one is graded as policy. The report renders as Markdown or as a self-contained HTML page and includes what RLM.learn recorded, read the way the review uses it: the selected tables as the RLM sees them (fact or dimension, time axis, measures, personal-data columns, whether an aggregate operation covers them), the operations by kind and the lessons. With a language model deepen adds RLM-proposed questions whose references come from two blind solves that must agree, plus an explanation and a proposed change for every question that is not correct; those solves run with evidence capture, the package learns from them through RLM.enrich, the report lists the lessons that appeared, and the enriched package is returned as report.learned_knowledge for saving.

A package can also carry what earlier runs learned about a source. Every turn records its source calls as typed telemetry (the grain a semantic-model query or a Lakehouse query asked for, the estimated group count, whether it ran, was rejected or timed out, how long it took, which measures came back identical) and never the data values; a registered operation the host executed for the run is recorded the same way, so CSV, Parquet, Delta and Lakehouse sources learn from their operations as semantic models learn from worker queries. Declared facts (grain, period column, units, definitions) are lessons from the start and reach every task on the source. capture_evidence=True turns that telemetry, together with what actually checked the run's answer (a validator or skill verifier that executed and accepted it, never configuration alone) and the analytical-integrity status, into result.evidence; RLM.enrich promotes evidence into structured lessons by a per-kind policy and returns a new package without touching the saved one. Evidence keeps the schema fingerprints the run executed against; a record that does not match the package it is enriched into is dropped and noted as an event, never relabelled:

knowledge = RLM.learn(
    sources={"production": "production.csv"},
    store=store,
    # what the profile cannot infer, stated by the source owner
    declared={
        "production": {
            "grain": ["line_id", "reporting_period"],
            "period_column": "reporting_period",
            "units": {"produced_units": "units"},
            "definitions": {"reporting_complete": "the row is final; exclude incomplete rows from totals"},
        }
    },
)

result = RLM.task(question, knowledge=knowledge, lm=lm, capture_evidence=True).run()
knowledge = RLM.enrich(knowledge, [result], store=store, overwrite=True)

for lesson in knowledge.package.lessons:
    print(lesson.kind, lesson.status, lesson.confidence, lesson.subject)

Lesson kinds include time_semantics (a boolean current-period flag in a period-like table of a semantic model is active from learn() on, labelled as inferred from schema names; a name match alone in such a table is only a candidate, and "current" elsewhere in the schema produces nothing), context_requirement (a derived measure that collapsed to its base measure under an unfiltered context; it stays a candidate, however often that recurs, until the same pair is compared under a period filter or a period grouping and comes out distinct), expensive_grain (proved by a cardinality preflight at once, by timeouts only in two separate runs), valid_grain (a grain that executed and returned rows in two separate runs, rendered as an observed feasible query grain; verified runs raise its confidence), preferred_strategy (only when the trajectory shows a restrict_to_candidate_tuples step between the coarse and the fine query, in runs whose answer passed an actual verifier and the integrity screen), invalid_path, and query_behavior for two measures whose values coincided across filtered contexts, recorded with semantic_equivalence: false: equal values never promote a metric_equivalence lesson. Candidates are never shown to the model. When a task falls through to the live source, the active lessons relevant to it are rendered into a short "Learned source guidance" section after the inputs, each line tagged with the source it was learned on; the registered-operation planner sees only the lessons for the sources its operations read. The source stays bound, so learning narrows the search and never removes the cold path. A package with no evidence and no lessons serializes exactly as before. Each lesson carries a dependency scope: a schema change stales the schema-scoped lessons that depend on the changed source, and a data-only change stales the snapshot- and operational-scoped ones (grains, costs, strategies) while leaving the schema facts. Today the typed source-call telemetry that feeds richer lessons comes from SemanticModel (and Lakehouse SQL timings); file sources contribute run outcomes only, so the behavioural learner is not yet equally deep across source types.

The development notebook benchmarks/notebooks/rlm_knowledge_benchmark_matrix.py runs seeded, cache-disabled cold-versus-learned trials across these paths and records correctness, operation selection, audit status, turns, token usage, LM/worker/ host/wall time, provenance, and drift rejection. KnowledgeBenchmarkReport also records source calls, failed calls, source seconds, the first useful query turn, verifier repairs, integrity status and injected lessons, and cold_parity() states the release rule: learned correctness must not fall below cold, overall and on every task.

from fabric_rlm import FabricLM, FileDestination, LakehouseSource, RLM

lakehouse = LakehouseSource(
    "abfss://<workspace-id>@onelake.dfs.fabric.microsoft.com/<lakehouse-id>"
)

with FileDestination(
    "abfss://<workspace-id>@onelake.dfs.fabric.microsoft.com/"
    "<lakehouse-id>/Files/reports"
) as destination:
    result = RLM.task(
        task=(
            "Create a formatted revenue workbook. Save it to a staged file, "
            "verify it by reopening it, then publish it through destination."
        ),
        inputs={"lakehouse": lakehouse, "destination": destination},
        outputs={"workbook_path": str, "summary": dict, "sources_used": list},
        skills=["excel_modify", "delta_lakehouse"],
        lm=FabricLM("gpt-5.1"),
    ).run()

Worker code uses destination.stage("revenue.xlsx") for the local openpyxl path and destination.publish(staged) after verification. The trusted parent performs the final copy and returns a manifest containing path, name, and size. Publishing refuses path traversal, files outside the private staging area, oversized files, and accidental overwrites. The context manager removes local staging files whether the run succeeds or fails. Pass overwrite=True to destination.publish(...) only when replacing an existing OneLake file is intentional.

Use the Python packages already in the notebook

The subprocess runs the same Python environment as the notebook. Generated code can import native packages and work with real file paths. This is the main difference from the Deno and Pyodide interpreter used by DSPy's standard RLM.

Check the work before accepting it

Output mappings enforce runtime types. output_validator can enforce business rules. output_validator_context can inspect files and other side effects. Markdown skills can include their own verifier. A failed check becomes feedback for the next attempt.

Keep domain rules beside the data

Skills are Markdown playbooks. They capture field definitions, procedures, tripwires, and executable checks. Store custom skills in Lakehouse Files and load them with SkillLoader. Bundled and custom skills can be used together.

Inspect each run

RLMResult includes the submitted payload, executed turns, errors, timings, token usage, validation repairs, and a deterministic report. Trajectories can be saved and replayed without calling the model again.

Where it fits

Use fabric-rlm for tasks that need one or more of these:

• Data that is too large to place in a model prompt
• Exact calculation across many rows or files
• Power BI semantic model queries mixed with Lakehouse files
• Workbook, report, or document generation
• Multi-step analysis with checkable outputs
• Reusable domain instructions and validation rules
• An execution trail for debugging and review

A direct model call is usually a better choice for short questions, rewriting, or judgment over text that already fits in context. The runtime adds Python execution and iterative checks, so each run takes longer than a single call.

Start with the API tour

Import examples/notebooks/rlm_api_tour.ipynb into a Fabric workspace. It covers task construction, typed outputs, files, custom Lakehouse skills, validators, result inspection, and worker controls.

The other notebooks cover PDF work, Spark log analysis, spreadsheet editing, semantic models, and multi-source tasks.

Measured results

The benchmarks are included for readers who want the evaluation setup, costs, and caveats. They are not required to use the library.

xychart-beta
    title "SpreadsheetBench V1 Verified-400 pass rate (percent)"
    x-axis ["fabric-rlm + MiniMax M3", "WPS AI", "GPT for Excel", "Data Analysis Agent", "Qingqiu Agent"]
    y-axis "pass rate" 0 --> 100
    bar [82.5, 91.25, 92.5, 96.5, 98.25]

Fabric data sources

Fabric notebooks already provide mounted Lakehouse storage, sempy, notebook identity, and the Python analytics stack. fabric-rlm exposes those resources to the run through typed handles. The task can combine semantic model queries with CSV, PDF, Excel, Parquet, and JSONL files without building a separate ingestion path.

Every source is bound as an input and the model decides which one answers which part of the brief. The semantic models are queried with DAX in the tabular engine, so aggregation happens where the data is and only the result comes back. The files are read in the subprocess. The workbook is written straight to Files/.

Installation

Python 3.10 to 3.12 is required; 3.13 is not supported yet. In Fabric, select the Python 3.12 (jupyter_python) kernel. Install the package and restart the session:

%pip install fabric-rlm

If imports fail on the Synapse PySpark kernel, see docs/fabric-runtime-deps.md.

Optional extras install packages used by specific workloads:

Other Python environments

The runtime also works on a laptop, in CI, or in an Azure Function. Use OpenAILM or AnthropicLM with the matching environment variable:

from fabric_rlm import File, OpenAILM, RLM  # set OPENAI_API_KEY in your environment

rlm = RLM.task(
    task="Sum every integer from 1 to 1,000,000 that is divisible by 3 or 5.",
    outputs=["answer"],
    lm=OpenAILM("gpt-4o-mini"),
)
print(rlm.run().answer)

RLM.task(...) is the short constructor; RLM.from_task(...) is the explicit form. OpenAILM, AnthropicLM, and FabricLM are thin wrappers over dspy.LM, so any OpenAI, Anthropic, Azure, or local Ollama model works.

Use a mapping when an output needs a runtime type contract:

result = RLM.task(
    task="Return the highest-revenue region and its revenue.",
    inputs={"sales": File("sales.csv")},
    outputs={"result": dict},
    lm=OpenAILM("gpt-4o-mini"),
).run()

If SUBMIT(result=...) receives the wrong type, the submission is rejected and the model gets repair feedback. Name-only lists such as outputs=["answer"] remain supported and do not add type enforcement.

Inputs and worker API

Values and files

Bind values, including large files, as inputs. Files arrive inside the worker as File(...) handles with .path, .read_text(), .read_bytes(), and .exists(), so a Lakehouse path or a local path is just a file path.

Lakehouses

LakehouseSource builds a metadata catalog in the parent Fabric notebook, then passes that catalog and the direct OneLake paths to the isolated worker. The model can choose relevant Delta tables and Files without you naming each table:

from fabric_rlm import FabricLM, LakehouseSource, RLM

source = LakehouseSource(
    "abfss://workspace-id@onelake.dfs.fabric.microsoft.com/lakehouse-id",
    tables="Tables",
    files="Files/data",
)

result = RLM.task(
    task="Which customer segments have declining usage and rising support demand?",
    inputs={"lakehouse": source},
    outputs={"answer": dict, "sources_used": list},
    lm=FabricLM("gpt-5.1"),
).run()

The root may use workspace and Lakehouse names or GUIDs. You can also pass a path ending in /Tables, /Tables/<schema>, /Tables/<schema>/<table>, or /Files/<path> to narrow discovery. Multiple LakehouseSource objects can be nested in input lists or dictionaries. Automatic Delta schema discovery requires fabric-rlm[analytics]. Outside a Fabric notebook, or when you already maintain a catalog, pass catalog=[...] to bypass discovery.

Inside the worker, use the resolved catalog helpers instead of trying to call Fabric discovery APIs again:

lakehouse.list_sources(kind="delta")
lakehouse.find_sources("usage")
lakehouse.find_sources("customer_id", kind="delta")

summary = lakehouse.query(
    """
    SELECT c.region, SUM(s.mrr) AS active_mrr
    FROM companies AS c
    JOIN subscriptions AS s USING (company_id)
    WHERE s.status = 'active'
    GROUP BY c.region
    ORDER BY active_mrr DESC
    """,
    sources={
        "companies": "dbo.companies",
        "subscriptions": "dbo.subscriptions",
    },
)

Catalog searches match source names, paths, columns, and data types. They do not widen the Tables or Files scopes supplied by the caller. query() runs in the trusted parent process against only the named catalog entries and returns bounded JSON-safe rows, so Fabric storage credentials are never exposed to the isolated model-generated worker. Queries are parsed before execution: every relation must resolve to a supplied alias or a CTE derived from one, and user-authored table functions, dynamic SQL, external paths, and unrecognized or side-effecting functions fail closed. The parent also applies a 30-second deadline, a 256 MiB DuckDB memory limit with temporary spill disabled, a 10,000-row ceiling, and a 5 MiB serialized-result ceiling. Results are fetched and sized one row at a time so an oversized scalar or row is rejected before the complete result is materialized.

Semantic models

A Power BI semantic model binds the same way and arrives as a connected handle:

from fabric_rlm import FabricLM, RLM, SemanticModel

RLM.task(
    task="Which product line has the highest recurring revenue?",
    inputs={"arr": SemanticModel("ARR Model SF (79)")},
    outputs=["answer"],
    lm=FabricLM("gpt-5.1"),
).run()

Inside the run, arr.schema() returns formatted schema text. For programmatic inspection, arr.metadata() returns ordinary pandas DataFrames with stable snake-case columns for tables, columns, measures, and relationships. arr.dax("EVALUATE ...", normalize_columns=True) returns an ordinary pandas DataFrame with predictable names such as period_year and arr, avoiding SemPy's bracketed result-column conventions. The raw metadata methods and raw arr.dax("EVALUATE ...") behavior remain available. arr.measure(name, groupby=[...], filters={...}) evaluates a model measure without authoring DAX. Pass workspace= for a model outside the attached workspace.

For measures by dimensions, prefer arr.aggregate(...). It validates measure and column names against the model, estimates how many groups the request would produce, and refuses to run a query whose estimate exceeds the safe limit (10,000 groups by default) or cannot be produced within thirty seconds. The error names the grouping, the measures, and concrete ways to narrow the query, so the model recovers in one turn instead of waiting out the worker timeout.

arr.aggregate(
    measures=["ARR $", "New $"],
    groupby=["Products[Line Of Business]", "Sold To[Sold_To Region]"],
    filters={"Period[YearQuarter]": "2026/Q2"},
    order_by="ARR $",
    top=100,
)

SemanticModel("...", max_groups=50_000) or FABRIC_RLM_SEMANTIC_MAX_GROUPS raises the ceiling for a model that handles wide grains well, and FABRIC_RLM_SEMANTIC_PREFLIGHT_TIMEOUT adjusts the estimate budget in seconds. arr.query_telemetry records the estimate, timing, and outcome of each call. arr.dax(...) is unchanged and runs whatever it is given.

Bind several at once and the model routes between them:

inputs={
    "mfg": SemanticModel("Manufacturing Ops"),
    "arr": SemanticModel("ARR Model SF (79)"),
}

This needs a Fabric notebook, where sempy ships in the runtime. The dataset name is checked when you construct SemanticModel, so a typo fails on that line rather than several turns into a run.

The handle gives generated code a clear entry point. Across two semantic models and two model families, tasks scored 18-19/19 and 13/15 with the handle. The same tasks scored 7/19 and 5/15 when they only named the semantic model.

Analytical integrity

The same rules apply whether a number came from a File, a LakehouseSource, or a SemanticModel. Three helpers are predefined in the sandbox and exported from fabric_rlm:

is_material_change(current, baseline, absolute_tolerance=1000, direction="decrease")
restrict_to_candidate_tuples(history, candidates, keys=["product", "region", "group"])
validate_analysis_integrity(ranking={...}, requested_grain=[...], actual_grain=[...], claims=[...])

is_material_change never treats float noise as a trend and carries no business threshold of its own; the analysis states the rule. The tuple helper keeps multidimensional candidates as compound identities instead of independent per-dimension lists. validate_analysis_integrity runs whichever checks have inputs: ranking concept versus metric, grain, materiality, candidate identity, provenance, and cross-source period, unit, definition, entity, and contradiction reconciliation.

Before accepting a SUBMIT, the runtime also screens the answer: prose that contradicts its own numbers, a "rank by impact" task whose ranking that reaches the answer sorted by something else or whose answer hides the impact metric, and code that consumed independent per-dimension value lists from a candidate frame together (a cartesian filter, whether as .isin chains or aggregate(filters=...)) without restoring the compound identity on those dimensions afterwards, are sent back with the reason. The code detectors are high-confidence and best-effort: they read pandas, polars, pyspark, sorted and SQL ORDER BY spellings and follow variable lineage from SUBMIT, and they stay silent when they cannot tell. In the default "repair" mode this happens at most twice, then the answer is accepted and the findings are exposed as result.integrity_problems with result.integrity_ok false. In "strict" mode a submission with findings is never accepted. Pass analytical_integrity=False or set FABRIC_RLM_ANALYTICAL_INTEGRITY=0 to turn the screen off.

Cross-source reconciliation (entities, metric definitions, periods, units, contradictions) is not enforced automatically, because the runtime has no structured claims to check. It is available three ways: as guidance the prompt injects whenever two or more evidence inputs are bound, in the analytical_integrity skill, and as validate_evidence_lineage / validate_analysis_integrity(claims=...) for an analysis that declares its claims, sources, joins, and disclosures.

Submission contract

The runtime injects SUBMIT(...). Call it with keyword arguments matching the declared outputs, or with positional arguments in the same order. After a valid submission, result.payload holds the dictionary and each field is also available as an attribute such as result.answer.

Inspect a run

In a Fabric or Jupyter notebook, render an interactive turn timeline:

result.inspect()

Each turn expands to show the observable model response, executed code, output, errors, validator feedback, submitted payload, timing, and token usage. Slow, error, repair, and submission turns are labeled in the timeline. Model-provider private chain-of-thought is not exposed. The inspector renders open with each turn collapsed, and the turn list scrolls after 15 rows. Use result.inspect(visible_turns=10) to change the viewport or result.inspect(expanded=False) when the whole inspector should start collapsed.

The inspector is dependency-free and escapes trajectory content before rendering. Save the same view as a standalone file when you need to share or archive it:

result.inspect().save_html("rlm-run.html")

Submitted fields normally remain available as result attributes. If an output is named inspect, result.inspect is that submitted value; use RLMResult.inspect(result) to open the run inspector for that result.

Nested model calls

Inside its Python, the model can call a nested model with predict_sync("english -> french", english=phrase) (or the async predict), optionally routed to a cheaper sub_lm=.

Engines

RLM ships with three stable engines, plus the experimental adaptive:

The default core skill carries a PLAN / VERIFY / REFLECT contract: plan before running code, self-check before SUBMIT, and carry prior-attempt failures into retries. It is on by default. Set FABRIC_RLM_PVR=0 to turn it off for token-sensitive batch runs on trivial tasks.

Skills

Skills are Markdown playbooks that tell the model how to do a kind of work properly: which library to reach for, the traps to avoid, and what to check before submitting. Eleven ship with the package. Name the ones a task needs and they are prepended to the prompt. The keyword router can also select them from the task and input names.

from fabric_rlm import RLM, File, FabricLM

rlm = RLM.task(
    task="Rebuild the summary tab from the raw export and flag any variance over 5 percent.",
    inputs={"workbook": File("/lakehouse/default/Files/finance/q3.xlsx")},
    outputs=["answer"],
    lm=FabricLM("gpt-5.1"),
    skills=["excel_modify", "data_exploration"],   # load as many as the task needs
)
print(rlm.run().answer)

The eight domain skills are keyword-routed. For example, data_exploration activates when a task or input name mentions logs or CSV files. The core and utility skills provide planning, validation, and error recovery.

Writing your own

A skill is one Markdown file with a small frontmatter block. Put it anywhere the notebook can read, including Lakehouse Files, and point a SkillLoader at that folder:

from fabric_rlm import RLM, SkillLoader, FabricLM

loader = SkillLoader(skill_dir="/lakehouse/default/Files/skills")
print(loader.list_skills())        # your skills plus the bundled ones

rlm = RLM.task(
    task="Extract the vendor totals from this invoice.",
    inputs={"doc": File("/lakehouse/default/Files/invoices/2026-07.pdf")},
    outputs=["totals"],
    lm=FabricLM("gpt-5.1"),
    skill_loader=loader,
    skills=["invoice_rules", "pdf_document_analysis"],
)

Your folder layers over the bundled skills rather than replacing them, so you can mix your own with the shipped ones in the same skills=[...] list. Pass several folders as a list if you keep them apart, and a file named after a bundled skill overrides it. If you want only your own, pass include_packaged=False.

Contributed skills

contrib-skills/ in the repository holds playbooks that are not installed with the package, either because they are narrower than the bundled ones or because the measurements behind them are thinner than a default install should carry. Point a loader at the folder to use one:

from fabric_rlm import RLM, File, SkillLoader

loader = SkillLoader(skill_dir="contrib-skills")

rlm = RLM.task(
    task="What was Boeing's FY2022 core operating loss? Report the figure with its sign.",
    inputs={"filing": File("BOEING_2022_10K.pdf")},
    outputs=["answer"],
    skill_loader=loader,
    skills=["pdf_document_analysis", "financial_documents"],
)

financial_documents is the first of these: reporting conventions for 10-K, 10-Q and earnings releases, covering parentheses as negative, scale stated in a header, fiscal against calendar year, adjacent period columns and subtotal rows. It is scoped to financial reporting on purpose, since parentheses mean something else in legal and scientific documents. docs/contrib-skills.md records what it was measured on and what the measurement does not support.

If a skill needs a library fabric-rlm does not depend on, install it in the notebook (%pip install python-docx) before running. The sandbox blocks pip and subprocess, so a skill cannot install its own dependencies. See docs/authoring-skills.md.

This is how house rules stop being tribal knowledge: your chart of accounts, the naming conventions your reports use, the columns that are always dates. Write it once, store it beside the data, and every run reads from the same copy.

Start from docs/skill-template.md; the structure is documented in docs/authoring-skills.md.

Security

This library runs model-generated code. The default SecurityPolicy scrubs secret-bearing environment variables from the worker, screens generated code, and blocks destructive Lakehouse operations such as notebookutils.fs.rm and notebookutils.fs.mv. The worker remains inside the notebook trust boundary. Read SECURITY.md before using untrusted prompts with sensitive data or credentials.

CLI

fabric-rlm --version
fabric-rlm run examples/simple_math/task.json      # run a task from JSON
fabric-rlm trace inspect path/to/trajectory.jsonl  # summarize and diagnose a saved run

Documentation

• QUICKSTART.md: step-by-step guide covering install, first run, Fabric notebook usage, sub-LM calls, traces, and skill authoring.
• docs/fabric-runtime-deps.md: read this if a Fabric notebook fails at import time (Sentinel, yarl.Query, aiohttp.ConnectionTimeoutError).
• docs/lossless-submit-payloads.md: how final payloads avoid namespace-snapshot truncation.
• examples/: what a user runs. examples/notebooks/ holds the ready-to-import Fabric recipes: start with rlm_vs_plain_llm_imf_cpi.ipynb (the with-and-without comparison) and rlm_api_tour.ipynb, then the PDF workflows, the Spark-log root-cause analysis, what moved and the Data Agent review. examples/simple_math/ is the quickstart task and examples/semantic_model/ the multi-source example.
• benchmarks/: the benchmark harnesses and the notebooks that reproduce the published SpreadsheetBench and knowledge-learning figures.
• docs/replaying-trajectories.md: record a run once, replay it with no model call, keep it as a regression test.
• CHANGELOG.md: release history.

Develop

git clone https://github.com/pawarbi/fabric-rlm-core.git
cd fabric-rlm-core
pip install -e ".[dev]"
pytest -q

See CONTRIBUTING.md for the development workflow.

Acknowledgments

fabric-rlm builds on the following work:

• The Recursive Language Model paradigm comes from the paper Recursive Language Models by Alex L. Zhang, Tim Kraska, and Omar Khattab (MIT CSAIL), which showed that letting a model programmatically examine and recursively query its own prompt beats stuffing everything into context.
• DSPy provides the RLM predictor and the interpreter protocol this library plugs into, and dspy.LM powers every model backend here.
• Predict-RLM by Trampoline AI, a production-focused RLM runtime built on DSPy signatures, inspired the direction of this project.

License

MIT. See LICENSE.