OpthaSim

Gymnasium-compatible reinforcement learning environments for ophthalmic treatment optimization.

OpthaSim provides four clinically-grounded simulation environments for training and evaluating RL agents on ophthalmic treatment decisions. Each environment is backed by peer-reviewed physiological models with literature-sourced parameters.

Environments

Environment	Task	Horizon	Action Space	Clinical Domain
GlaucomaIOP-v0	Medication selection for IOP control	365 days	Discrete(6)	Glaucoma
AntiVEGF-v0	Injection timing for wet AMD	24 months	Discrete(3)	Wet AMD
LaserTrabeculoplasty-v0	SLT parameter optimization	36 months	MultiDiscrete([2,3,3])	Glaucoma
DiabeticRetinopathy-v0	Screening and treatment escalation	10 years	Discrete(5)	Diabetic Retinopathy

Quick Start

pip install ophthasim

import gymnasium as gym
import ophthasim
from ophthasim.agents import get_heuristic_agent

# Create environment
env = gym.make("ophthasim/GlaucomaIOP-v0", difficulty="medium")
obs, info = env.reset(seed=42)

# Use clinical guideline-based agent
agent = get_heuristic_agent("ophthasim/GlaucomaIOP-v0", target_iop=info["target_iop"])

total_reward = 0.0
terminated, truncated = False, False
while not (terminated or truncated):
    action = agent.act(obs)
    obs, reward, terminated, truncated, info = env.step(action)
    total_reward += reward

print(f"Episode reward: {total_reward:.2f}")
print(f"Final IOP: {info['true_iop']:.1f} mmHg (target: {info['target_iop']:.1f})")
env.close()

Physiological Models

IOP Dynamics (Goldmann Equation)

Intraocular pressure follows the Goldmann ODE with circadian modulation:

dIOP/dt = (1/K) * [F_in(t) - C * (IOP - P_e) - F_u]
F_in(t) = F_in_mean * (1 + A_circ * cos(2*pi*(t - phi)/24))

Parameters sourced from Brubaker (1991), Bill (1966), Liu (1998). Medication effects modeled for prostaglandin analogs, beta-blockers, alpha-agonists, and carbonic anhydrase inhibitors.

Anti-VEGF Pharmacokinetics/Pharmacodynamics

One-compartment vitreous PK with first-order elimination:

dC_vit/dt = -k_el * C_vit    (k_el = ln2 / t_half)
VEGF_free = VEGF_basal * IC50 / (C_drug + IC50)
dCST/dt = alpha * VEGF_free - beta * (CST - CST_normal)
VA = VA_baseline + gamma * ln(CST_baseline / CST)

Drug library includes ranibizumab (Gadkar 2015), aflibercept (Kaiser 2021), and bevacizumab (Stewart 2012).

Diabetic Retinopathy Progression

Five-state Markov chain {None, Mild NPDR, Moderate NPDR, Severe NPDR, PDR} with HbA1c-modulated quarterly transitions from WESDR/UKPDS data. Treatment modifiers for PRP and anti-VEGF.

SLT Outcome Model

Probabilistic success model with exponential durability decay:

P_success = sigmoid(-2.0 + 0.08 * IOP_baseline + 3.0 * (coverage/360))
IOP_reduction(t) = delta_IOP * exp(-0.023 * t_months)

Difficulty Tiers

Each environment supports three difficulty levels:

• Easy: Stable patient, good prognosis, high compliance
• Medium: Typical clinical case, moderate variability
• Hard: Treatment-resistant, comorbidities, low compliance

env = gym.make("ophthasim/GlaucomaIOP-v0", difficulty="hard")

Baseline Agents

Heuristic (Clinical Guidelines)

from ophthasim.agents import get_heuristic_agent

agent = get_heuristic_agent("ophthasim/AntiVEGF-v0")
action = agent.act(observation)

Random

from ophthasim.agents import RandomAgent

agent = RandomAgent(env.action_space, seed=42)
action = agent.act(observation)

PPO (Stable-Baselines3)

pip install ophthasim[train]
ophthasim-train --env ophthasim/GlaucomaIOP-v0 --timesteps 500000

Benchmarking

pip install ophthasim[bench]
ophthasim-bench --episodes 100 --seed 42

from ophthasim.benchmarks import BenchmarkRunner

runner = BenchmarkRunner(n_episodes=100, seed=42)
results = runner.run_all()
runner.print_summary()

Development

git clone https://github.com/HassDhia/ophthasim.git
cd ophthasim
python -m venv .venv
source .venv/bin/activate
pip install -e ".[all]"
pytest tests/ -v

Parameter Validation

Every physiological parameter includes:

• SOURCE comment citing the literature reference
• PARAMETER_RANGES entry with clinically valid bounds
• Runtime assertion enforcing those bounds

Modeling simplifications are documented with SIMPLIFICATION comments.

Citation

@software{dhia2026ophthasim,
  title={OpthaSim: Gymnasium-Compatible RL Environments for Ophthalmic Treatment Optimization},
  author={Dhia, Hass},
  year={2026},
  url={https://github.com/HassDhia/ophthasim},
  version={0.1.0}
}

@article{dhia2026ophthasim_paper,
  title={OpthaSim: Reinforcement Learning Environments for Ophthalmic Treatment Optimization},
  author={Dhia, Hass},
  year={2026},
  journal={arXiv preprint}
}

License

MIT License. See LICENSE for details.