REACHER — Python Backend

FastAPI server, serial communication kernel, and session manager for the REACHER ecosystem

Written by: Joshua Boquiren

Overview

The Python backend is the core of the REACHER system. It provides:

• A FastAPI REST API for session management, hardware control, experiment execution, and data export
• A WebSocket server for real-time event streaming to the browser UI
• A multi-threaded serial communication kernel for bidirectional JSON messaging with Arduino hardware
• A session manager coordinating multiple simultaneous experiment sessions with port locking
• A firmware uploader for flashing Arduino .hex files via avrdude

When running as a standalone executable, the backend also serves the React frontend as static files and opens a browser window automatically.

Role in the REACHER Ecosystem

The Python backend is the bridge between the Arduino hardware and the browser-based UI. It:

1. Communicates with one or more Arduinos over USB serial (115200 baud, JSON messages)
2. Exposes a REST API and WebSocket endpoint on port 6229
3. Serves the React frontend as static files at the root URL
4. Manages experiment sessions — starting, stopping, pausing, and collecting data
5. Handles firmware uploads to Arduino boards via avrdude
6. Logs all serial events and behavioral data for post-experiment analysis

Arduino ◄──USB Serial──► REACHER Kernel ◄──► FastAPI ◄──► React Frontend
                          (threads)          (REST+WS)    (browser)

Architecture

Project Structure

reacher/
├── pyproject.toml              # Package metadata and dependencies
├── src/reacher/
│   ├── __init__.py             # Exports: REACHER, COMMAND_REGISTRY, CommandCode, PARADIGMS
│   ├── __main__.py             # Entry point for `python -m reacher`
│   ├── session_manager.py      # Multi-session coordinator with port locking
│   ├── api/
│   │   ├── app.py              # FastAPI app, CORS, lifespan, static file mount
│   │   └── routers/
│   │       ├── session.py      # Session CRUD
│   │       ├── serial.py       # Port listing and serial connections
│   │       ├── firmware.py     # Paradigm listing and firmware upload
│   │       ├── hardware.py     # Command dispatch and config retrieval
│   │       ├── program.py      # Start/stop/pause and limit configuration
│   │       ├── data.py         # Behavior events, frames, CSV export
│   │       ├── file.py         # Filename and destination configuration
│   │       ├── lifecycle.py    # Graceful shutdown
│   │       └── websocket.py    # Real-time event streaming
│   ├── kernel/
│   │   ├── reacher.py          # Core REACHER class (serial I/O, threading, data)
│   │   └── commands.py         # CommandCode enum, CommandSpec, COMMAND_REGISTRY
│   ├── uploader/
│   │   └── uploader.py         # FirmwareUploader (avrdude wrapper)
│   └── hex/<board>/*.hex       # Committed firmware artifacts (package data, shipped in wheel)
├── firmware/                   # Arduino firmware source (folded in from reacher-firmware)
│   ├── compile.sh             # Builds all 5 paradigms -> ../src/reacher/hex/<board>/
│   ├── fr/ pr/ vi/ omission/ pavlovian/   # Per-paradigm sketches
│   └── libraries/REACHERDevices/          # Shared C++ device library + Commands.h
└── tests/
    ├── test_commands.py
    ├── test_command_parity.py # Asserts Commands.h matches the CommandCode enum
    ├── test_session_manager.py
    ├── test_api.py
    └── core/

Kernel — Multi-Threaded Serial I/O

The REACHER class manages all communication with a single Arduino. Each instance runs three daemon threads:

Thread coordination uses threading.Event flags:

• serial_flag — cleared to read, set to stop
• program_flag — cleared when running, set when paused/stopped
• time_check_flag — monitors limit conditions

Session Manager

The SessionManager coordinates multiple independent REACHER instances:

• Port locking — prevents two sessions from binding to the same COM port
• Session states — idle → uploading → connected → running → paused → stopped
• Session IDs — 12-character hexadecimal identifiers (from uuid4)
• Event broadcasting — state changes are forwarded to connected WebSocket clients

Firmware Uploader

The FirmwareUploader wraps avrdude to flash compiled .hex files onto the Arduino (Mega 2560, ATmega2560; legacy UNO artifacts still ship):

• Async subprocess execution with progress parsing from avrdude stderr
• Hex resolution order: PyInstaller bundle (_MEIPASS/hex/) → REACHER_HEX_DIR → package data (src/reacher/hex/, canonical) → cwd firmware/hex/ → ~/.reacher/hex GitHub cache. Set REACHER_SKIP_HEX_FETCH=1 to disable the network fallback on airgapped hosts.

Firmware Source

Firmware source lives in firmware/ (folded in from the now-archived Otis-Lab-MUSC/reacher-firmware). Five Arduino sketches share the REACHERDevices C++ library. firmware/Commands.h and kernel/commands.py are kept in lockstep — tests/test_command_parity.py fails on drift.

arduino-cli core install arduino:avr   # one-time toolchain install
bash firmware/compile.sh               # recompile -> src/reacher/hex/<board>/ (commit the result)

The compiled hex/<board>/*.hex files are committed package data (pyproject.toml glob hex/**/*.hex) and ship inside the wheel — there is no firmware build step in CI. Firmware version strings (library.properties + each sketch's SendIdentification()) are stamped by scripts/bump-version.py from the package version; recompile hex after bumping. The microscope timestamp pin (INT0) is fixed in firmware and not remappable.

API Reference

REST Endpoints

Sessions (/api/sessions)

Serial (/api/serial)

Firmware (/api/firmware)

Hardware (/api/hardware)

Program (/api/program)

Data (/api/data)

File (/api/file)

Lifecycle (/api/lifecycle)

WebSocket

Event types sent over WebSocket:

Serial Protocol

Communication with Arduino hardware uses the following protocol:

Event code meanings (firmware → backend)

Command code ranges (backend → firmware)

The backend's COMMAND_REGISTRY contains 71 CommandSpec entries with paradigm filtering — commands are only exposed for paradigms that use them.

Installation

From a wheel file

pip install reacher2p-3.1.0-py3-none-any.whl

From source

git clone https://github.com/otis-lab-musc/reacher.git
pip install -e reacher/

Running

CLI command

reacher

Module invocation

python -m reacher

Both start the FastAPI server on http://localhost:6229 and open a browser window. Set the REACHER_STATIC_DIR environment variable to point to a built frontend directory, or run from the labrynth root where web/dist/ will be found automatically.

Port configuration

Set the REACHER_PORT environment variable to change the default port:

REACHER_PORT=8080 reacher

Development

Setting up a development environment

git clone https://github.com/otis-lab-musc/reacher.git
cd reacher
python -m venv .venv
source .venv/bin/activate    # Linux/macOS
# .venv\Scripts\activate     # Windows
pip install -e ".[dev]"

Running tests

pytest

Linting

ruff check .

Building the Standalone Executable

See labrynth for standalone packaging via PyInstaller.

Configuration

Environment Variables

Data Directory

REACHER stores logs and data under ~/REACHER/:

~/REACHER/
├── LOG/
│   └── YYYY-MM-DD_HH-MM-SS/
│       ├── controller_log.json    # JSON events from firmware
│       └── interface_log.log      # Python logging output
└── DATA/                          # Default data export destination

The data export destination can be customized per session via the File API.

Dependencies

Runtime

Optional (tray extra: pip install reacher2p[tray])

Development

License

This project is licensed under the MIT License. See LICENSE for details.

Contact

Joshua Boquiren — thejoshbq@proton.me

GitHub: otis-lab-musc/reacher