sympypl

SymPy expressions as Polars DataFrames.

sympypl lets you build symbolic math expressions with SymPy, serialize them to a Polars DataFrame for storage or transport, and evaluate them against any DataFrame with matching columns — all without per-row Python overhead.

Installation

pip install sympypl

scipy is an optional dependency. Install it to enable gamma, factorial, and binomial:

pip install sympypl scipy

Primary use cases

Build an expression and apply it to a DataFrame

import sympy as sp
import polars as pl
from sympypl import sympy_to_pl_expr

age = sp.Symbol('age')
income = sp.Symbol('income')

expr = sp.log(income) + sp.sqrt(age) / 10

df = pl.DataFrame({'age': [25.0, 40.0, 55.0], 'income': [40000.0, 80000.0, 120000.0]})

result = df.select(
    sympy_to_pl_expr(expr, {age: 'age', income: 'income'}).alias('score')
)

Serialize an expression to a DataFrame and restore it

from sympypl import to_polars, from_polars

# Serialize — each row is one node of the expression tree
tree_df = to_polars(expr)

# Restore — identical expression object
restored = from_polars(tree_df)

# Apply the restored expression exactly as before
result2 = df.select(
    sympy_to_pl_expr(restored, {age: 'age', income: 'income'}).alias('score')
)

tree_df's schema — node_id, node_type, str_value, float_value, args (one row per expression-tree node) — is available as sympypl.sp_schema if you need to declare or check it independently of calling to_polars.

Positional column references with $ indices

When a DataFrame has awkward column names — numeric strings like '0', '1', auto-generated headers, or anything else that clashes with SymPy's parser — use col_symbol and index_bindings to refer to columns by position instead of by name.

from sympypl import col_symbol, index_bindings, sympy_to_pl_expr

# col_symbol(i) returns a Symbol named '$i'
x0, x1 = col_symbol(0), col_symbol(1)

expr = x0 * sp.log(x1) + sp.Float(1.0)

# DataFrame with integer-string column names
df = pl.DataFrame({'0': [1.0, 2.0, 3.0], '1': [10.0, 100.0, 1000.0]})

# index_bindings maps col_symbol(i) → df.columns[i]
result = df.select(
    sympy_to_pl_expr(expr, index_bindings(df.columns)).alias('v')
)

$-prefixed names are unambiguous to SymPy's parser — sympify('$0') returns a Symbol, never an integer — so expressions using col_symbol survive string serialization via srepr / sympify as well as the standard to_polars / from_polars round-trip.

Parse an expression string safely

parse_expr_safe validates a string against an AST allowlist before evaluation, so it is safe to use with untrusted input.

from sympypl.safe_parse import parse_expr_safe

expr = parse_expr_safe('log(income) + sqrt(age) / 10', ['age', 'income'])

result = df.select(
    sympy_to_pl_expr(expr, {age: 'age', income: 'income'}).alias('score')
)

Supported functions

Arithmetic

Operator	Example
+, -, *, /, **	x**2 + y / 3
% (modulo)	sp.Mod(x, y)
Unary -	-x

Trigonometric

SymPy	Notes
sp.sin, sp.cos, sp.tan
sp.asin, sp.acos, sp.atan

Hyperbolic

SymPy	Notes
sp.sinh, sp.cosh, sp.tanh
sp.asinh, sp.acosh, sp.atanh

Exponential and logarithmic

SymPy	Notes
sp.exp
sp.log	One-arg form is natural log; sp.log(x, b) for base b
sp.sqrt

Other unary

SymPy	Notes
sp.Abs
sp.sign	Returns −1, 0, or 1
sp.floor, sp.ceiling
sp.erf	Error function — pure Polars, no scipy
sp.erfc	Complementary error function — pure Polars, no scipy; max absolute error < 1.5×10⁻⁷

Variadic

SymPy	Notes
sp.Max, sp.Min	Any number of arguments
sp.Heaviside	Optional second arg sets value at zero (default 0.5)

Combinatorial — require scipy

SymPy	Notes
sp.factorial	Continuous extension via Γ(n+1)
sp.gamma
sp.binomial	Computed as Γ(n+1) / (Γ(k+1)·Γ(n−k+1))

Piecewise and relational

sp.Piecewise, sp.Eq, sp.Ne, sp.Lt, sp.Le, sp.Gt, sp.Ge, sp.And, sp.Or, sp.Not are supported for use as piecewise conditions.

expr = sp.Piecewise(
    (sp.log(income), income > 0),
    (sp.Float(0.0), sp.true),
)

Categorical mapping

CatMap maps a string-valued column to numeric values. Unknown categories produce null.

Dict form — the usual constructor:

from sympypl import CatMap, CatEntry

region = sp.Symbol('region')
score = CatMap.from_dict(region, {'north': 1.0, 'south': 2.0, 'east': 3.0})

df = pl.DataFrame({'region': ['north', 'east', 'unknown']})
result = df.select(
    sympy_to_pl_expr(score, {region: 'region'}).alias('score')
)
# [1.0, 3.0, null]

Entry form — explicit CatEntry objects, useful when building programmatically:

score = CatMap(region, CatEntry('north', 1.0), CatEntry('south', 2.0))

Each CatEntry's category key is stored internally as a StrLiteral — a string-valued SymPy leaf node (sp.Basic, not sp.Expr, so it can't be composed into arithmetic on its own). You won't normally construct one directly; it shows up if you introspect a CatMap's SymPy tree (e.g. CatMap.from_dict(...).args).

Composing with arithmetic — CatMap subclasses sp.Expr so it composes directly with other symbolic operations:

x = sp.Symbol('x')

# Multiply the categorical score by a numeric column, then add a bias
expr = CatMap.from_dict(region, {'north': 1.0, 'south': 2.0, 'east': 3.0}) * x + 0.5

df = pl.DataFrame({
    'region': ['north', 'south', 'east'],
    'x': [10.0, 20.0, 30.0],
})
result = df.select(
    sympy_to_pl_expr(expr, {region: 'region', x: 'x'}).alias('score')
)
# [10.5, 40.5, 90.5]

CatMap expressions round-trip through to_polars / from_polars like any other node.

Multi-output log-weights

LogWeights wraps N per-category log-weight expressions as a single node — useful for a categorical outcome, where each class needs its own function of the inputs and you want one expression object (and one string) instead of N separately-tracked ones. Unlike CatMap, LogWeights deliberately does not subclass sp.Expr: a vector of per-class logits isn't something you'd add or multiply into another expression, so composing it with arithmetic is disallowed by construction.

from sympypl import LogWeights, sympy_to_pl_exprs, col_symbol, index_bindings

x0, x1 = col_symbol(0), col_symbol(1)

# One log-weight expression per class (3 classes here)
lw = LogWeights(
    x0 * 0.5,               # class 0
    x1 - 1.0,                # class 1
    sp.sin(x0) + x1,         # class 2
)

df = pl.DataFrame({'0': [1.0, 2.0], '1': [3.0, 4.0]})
bindings = index_bindings(df.columns)

# One pl.Expr per term, in order
exprs = sympy_to_pl_exprs(lw, bindings)
logits = df.select([e.alias(f'class_{i}') for i, e in enumerate(exprs)])

# Softmax-normalize + sample a class (see silverknockoff's SynthesizeY for
# the full pattern, including the RNG draw)
import numpy as np
lw_np = logits.to_numpy()
lw_np -= lw_np.max(axis=1, keepdims=True)
probs = np.exp(lw_np)
probs /= probs.sum(axis=1, keepdims=True)

sympy_to_pl_exprs is the multi-output counterpart to sympy_to_pl_expr: it returns a list[pl.Expr], one element per LogWeights term (or a single-element list for an ordinary expression, so it's safe to call unconditionally). This mirrors gradient/gradient_df's existing list-of-expressions convention.

LogWeights round-trips through both serialization paths, like any other node:

from sympypl import to_polars, from_polars, expr_to_str
from sympypl.safe_parse import parse_sympypl_str

# DataFrame tree round-trip
restored = from_polars(to_polars(lw))

# String round-trip — only col_symbol-based expressions round-trip through
# strings (same constraint as ordinary expr_to_str/parse_sympypl_str); the
# string form is logweights(term0, term1, ...)
s = expr_to_str(lw)
restored2 = parse_sympypl_str(s)

Symbolic differentiation

partial_derivative, gradient, and gradient_df differentiate an expression symbolically (via SymPy's own diff), rather than numerically -- the result is itself a sympypl expression, so it serializes, composes, and evaluates on a DataFrame exactly like any other.

from sympypl import partial_derivative, gradient, gradient_df

age = sp.Symbol('age')
income = sp.Symbol('income')
expr = sp.log(income) + sp.sqrt(age) / 10

# A single partial derivative -- wrt is a Symbol, or an int column index
# (int -> col_symbol(i))
d_age = partial_derivative(expr, age)
# 1/(20*sqrt(age))

# The full gradient: one sp.Expr per element of wrt, same order
d_age, d_income = gradient(expr, [age, income])

# Evaluate the gradient directly on a DataFrame -- one Float64 column per
# wrt element, named after the corresponding DataFrame column, rows
# aligned with df
df = pl.DataFrame({'age': [25.0, 40.0, 55.0], 'income': [40000.0, 80000.0, 120000.0]})
grad_df = gradient_df(expr, df, wrt=[age, income], bindings={age: 'age', income: 'income'})

gradient_df's wrt/bindings both default sensibly for the common case of differentiating with respect to every column of df by position: omit them and wrt becomes every column index and bindings becomes index_bindings(df.columns).

Differentiating through CatMap. A CatMap is piecewise-constant in its underlying symbol, so its derivative isn't simply zero-or-undefined in any generally useful sense -- categorical (on all three functions, default 'ignore') controls how it's handled:

• 'ignore' (default): CatMap nodes are treated as constants: their contribution to the derivative is 0.
• 'diff': for a CatMap whose inner symbol equals wrt, the derivative is itself a CatMap -- each category's value becomes value_k - value_ref, where the reference category is the one keyed '0'. This matches the one-hot-encoding drop_first=True convention (the reference category's dummy is implicitly all zeros), so the result is exactly the marginal effect of each non-reference category relative to the reference.

region = sp.Symbol('region')
cm = CatMap.from_dict(region, {'0': 1.0, '1': 2.0, '2': 3.0})

d_region = partial_derivative(cm, region, categorical='diff')

df = pl.DataFrame({'region': ['0', '1', '2']})
result = df.select(sympy_to_pl_expr(d_region, {region: 'region'}).alias('d_score'))
# [0.0, 1.0, 2.0] -- category '0' is the reference (0 by construction),
# '1' and '2' are their value minus the reference's

safe_parse function allowlist

When using parse_expr_safe, the default SAFE_FUNCTIONS allowlist covers all functions in the table above except Heaviside and the piecewise/relational forms (which require string syntax beyond arithmetic). Use Abs (capital A), not abs.

# Two-argument log
parse_expr_safe('log(x, 2)', ['x'])

# Scipy-backed functions require scipy at evaluation time
parse_expr_safe('gamma(x)', ['x'])