PyMAST - Movement Analysis Software for Telemetry

PyMAST (Movement Analysis Software for Telemetry) is a Python toolkit for processing, analyzing, and modeling radio telemetry data. From data import to statistical model export, PyMAST provides a complete solution for aquatic telemetry studies.

Key Features

• Multi-Manufacturer Support - Import from Lotek (SRX600/800/1200), Orion, ARES, and VR2 receivers
• Automated Classification - Naive Bayes classifier removes false positives
• Bout Detection - DBSCAN clustering identifies continuous presence periods
• Overlap Resolution - Signal quality comparison resolves spatial ambiguity
• Movement Filtering - Adjacency filter removes impossible transitions
• Statistical Export - CJS, LRDR, and Time-to-Event formats for Program MARK/R
• HDF5 Database - Fast queries and efficient storage for large datasets
• Comprehensive Documentation - All modules accessible via Python help() system
• Visualization Suite - Network graphs, bout distributions, overlap analysis, 3D fish tracks

Quick Start

New users: Start with GETTING_STARTED.md for a complete walkthrough.

Requirements

• Python 3.9 or higher
• Compatible with Windows, macOS, and Linux

Installation

Standard installation:

pip install pymast

Recommended first step on Windows (improves wheel resolution):

python -m pip install --upgrade pip setuptools wheel

GUI dependencies (optional):

pip install "pymast[gui]"

For Spyder/Anaconda users:

To ensure installation in the correct environment, use:

python -m pip install pymast

To verify you're using the right Python interpreter:

python -c "import sys; print(sys.executable)"

From source:

cd mast
pip install -e .

30-Second Example

from pymast.radio_project import radio_project
import pandas as pd

# Initialize project
proj = radio_project(
    project_dir='C:/my_study',
  db_name='study',
  detection_count=5,
  duration=1,
  tag_data=pd.read_csv('tags.csv'),
  receiver_data=pd.read_csv('receivers.csv')
)

# Import receiver data
proj.telem_data_import(
    rec_id='REC001',
  rec_type='srx1200',
  file_dir='C:/my_study/Data/Training_Files',
  db_dir=proj.db,
    scan_time=2.5,
    channels=1
)

# Process and analyze
proj.make_recaptures_table()

See GETTING_STARTED.md for complete workflows.

Documentation

In-Code Documentation

All modules support Python's help() system:

import pymast
help(pymast.parsers)              # Module overview
help(pymast.overlap_removal.bout) # Class documentation
help(pymast.naive_bayes.calculate_likelihood)  # Function details

System Architecture

Raw receiver files
  -> Parsers (HDF5 import)
  -> Classification (false positive removal)
  -> Recaptures (link detections to locations)
  -> Bout detection (presence periods)
  -> Overlap resolution (multi-receiver conflicts)
  -> Adjacency filter (movement constraints)
  -> Formatter (CJS/LRDR/TTE)
  -> Program MARK / R analysis

See ARCHITECTURE.md for details.

System Requirements

• Python: 3.9 or higher
• RAM: 8+ GB recommended for large datasets
• Storage: HDF5 database typically 20-50% of raw data size
• Disk Space: 10+ GB for large projects with beacon tags
• Operating System: Windows, macOS, or Linux

Required Input Files

MAST requires three CSV files to initialize a project:

1. Master Tag File (tblMasterTag.csv)

Example:

freq_code,pulse_rate,tag_type,rel_date,rel_loc,cap_loc,length,species
164.123 45,3.0,study,2024-05-15 08:30:00,Release Site 1,Capture Site A,450,Atlantic Salmon
164.234 12,5.0,BEACON,2024-05-01 00:00:00,Beacon Location,Beacon Location,NA,NA

2. Master Receiver File (tblMasterReceiver.csv)

Example:

rec_id,rec_type,node,name
R01,srx800,N01,Downstream Array
R02,srx800,N02,Fishway Entrance
R03,orion,N03,Upstream Release

3. Nodes File (tblNodes.csv)

Note: Use arbitrary coordinates for visualization, not actual lat/lon. This creates clearer network diagrams for sinuous river systems.

Example:

node,reach,X,Y
N01,Lower River,100,100
N02,Fishway,200,150
N03,Upper River,300,200

Project Directory Structure

When you create a project, MAST automatically creates this structure:

your_project/
  Data/
    tblMasterTag.csv
    tblMasterReceiver.csv
    tblNodes.csv
    Training_Files/
      [raw receiver files]
  Output/
    Figures/
    recaptures.csv
  your_database.h5

Workflow Steps

1. Project Initialization

from pymast.radio_project import radio_project
import pandas as pd
import os

project_dir = r"C:\MyProject"
db_name = 'my_study'

# Load input files
tags = pd.read_csv(os.path.join(project_dir, 'tblMasterTag.csv'))
receivers = pd.read_csv(os.path.join(project_dir, 'tblMasterReceiver.csv'))
nodes = pd.read_csv(os.path.join(project_dir, 'tblNodes.csv'))

# Create project (detection_count and duration are classifier parameters)
project = radio_project(
    project_dir=project_dir,
    db_name=db_name,
    detection_count=5,      # Number of detections in detection history window
    duration=1,             # Duration (minutes) for noise ratio calculation
    tag_data=tags,
    receiver_data=receivers,
    nodes_data=nodes
)

2. Import Raw Data

# Import data from one receiver
project.telem_data_import(
    rec_id='R01',                    # Receiver ID from tblMasterReceiver
    rec_type='srx800',               # Receiver type
    file_dir=os.path.join(project_dir, 'Data', 'Training_Files'),
    db_dir=os.path.join(project_dir, f'{db_name}.h5'),
    scan_time=1,                     # Channel scan time (seconds) for Orion/Ares
    channels=1,                      # Number of channels for Orion/Ares
    ant_to_rec_dict={'A0': 'R01'}    # Antenna to receiver mapping
)

# Undo import if needed
# project.undo_import('R01')

3. Train Classifier

Training creates a dataset of known true and false positive detections using beacon tags and miscoded detections.

rec_id = 'R01'
rec_type = 'srx800'

# Get all fish detected at this receiver
fishes = project.get_fish(rec_id=rec_id)

# Train on each fish
for fish in fishes:
    project.train(fish, rec_id)

# Generate training summary with visualization
project.training_summary(rec_type, site=[rec_id])

# Undo training if needed
# project.undo_training(rec_id)

4. Classify Detections

Apply the Naive Bayes classifier to identify false positives.

project.reclassify(
    project=project,
    rec_id='R01',
    rec_type='srx800',
    threshold_ratio=1.0,  # 1.0 = MAP hypothesis, >1.0 = more strict, <1.0 = less strict
    likelihood_model=['hit_ratio', 'cons_length', 'noise_ratio', 'power', 'lag_diff']
)

# Undo classification if needed
# project.undo_classification('R01')

Available Likelihood Predictors:

• hit_ratio: Proportion of expected pulses detected
• cons_length: Maximum consecutive detections
• noise_ratio: Ratio of miscoded to correct detections
• power: Signal strength
• lag_diff: Second-order difference in detection intervals

5. Bout Detection (Optional)

Identify discrete presence events at receivers.

import pymast

node = 'N01'
bout_obj = pymast.bout(project, node, eps_multiplier=5, lag_window=2)

# Write DBSCAN presence records
bout_obj.presence()

# Undo if needed
# project.undo_bouts(node)

6. Overlap Removal (Optional)

Remove detections from parent receivers when fish is present at nested child receivers.

import pymast

# Define parent:child relationships (large:small detection zones)
edges = [
    ('R01', 'R02'),  # R01 overlaps R02
    ('R01', 'R03'),  # R01 overlaps R03
    ('R02', 'R04'),  # R02 overlaps R04
]

nodes = ['R01', 'R02', 'R03', 'R04']

# Create overlap object and apply nested doll algorithm
doll = pymast.overlap_reduction(nodes, edges, project)
doll.nested_doll()

# Alternative: unsupervised removal (uses clustering)
# doll.unsupervised_removal()

# Undo if needed
# project.undo_overlap()

7. Create Recaptures Table

Compile all classified, de-overlapped detections into a final recaptures table.

# Create and export recaptures table
project.make_recaptures_table(export=True)

# Output will be saved to: Output/recaptures.csv

# Undo if needed
# project.undo_recaptures()

Statistical Data Formatting

Cormack-Jolly-Seber (CJS) Format

from pymast.formatter import cjs_data_prep

# Define receiver to recapture occasion mapping
receiver_to_recap = {
    'R01': 'R00',  # Release
    'R02': 'R01',  # First recapture
    'R03': 'R02',  # Second recapture
    'R04': 'R03',  # Third recapture
}

# Create CJS formatted data
cjs = cjs_data_prep(
    receiver_to_recap=receiver_to_recap,
    project=project,
    rel_loc='Release Site 1',  # Optional filter
    initial_recap_release=False  # True if starting from first recapture
)

# Export to .inp file for Program MARK
cjs.input_file('my_model', project.output_dir)

Competing Risks / Time-to-Event Format

from pymast.formatter import time_to_event

# Define node to state mapping
node_to_state = {
    'N01': 1,  # Downstream
    'N02': 2,  # Fishway
    'N03': 3,  # Upstream
}

# Create time-to-event data
tte = time_to_event(
  receiver_to_state=node_to_state,
  project=project
)

# Build transition records and export for survival analysis in R
tte.data_prep(project, bucket_length_min=15)
tte.master_state_table.to_csv(os.path.join(project.output_dir, 'time_to_event.csv'), index=False)

Visualization

Fish Movement History

from pymast.fish_history import fish_history

# Create fish history object
fh = fish_history(
    projectDB=project.db,
    filtered=True,        # Use classified data
    overlapping=False     # Exclude overlapping detections
)

# Plot a specific fish in 3D space-time
fh.fish_plot('164.123 45')

Supported Receiver Types

Scientific Background

PyMAST implements validated algorithms for radio telemetry data processing, peer-reviewed and published in Animal Biotelemetry:

Nebiolo, K.P., & Castro-Santos, T. (2024). MAST: Movement Analysis Software for Telemetry data. Part I: the semi-automated removal of false positives from radio telemetry data. Animal Biotelemetry, 12(1), 11. https://doi.org/10.1186/s40317-024-00358-1

False Positive Detection Algorithm

MAST uses a Naive Bayes classifier with the conditional independence assumption:

P(True|X) = P(True) * P(Xi|True)
P(False|X) = P(False) * P(Xi|False)

Where X represents the vector of predictor variables. The classifier computes the posterior probability for each detection and classifies using the Maximum A Posteriori (MAP) hypothesis.

Key Innovation: Laplace smoothing (add-one smoothing) prevents zero probabilities for unseen feature combinations, making the classifier robust to novel detection patterns.

Bout Detection Algorithm

Fits a three-process broken-stick model (piecewise exponential decay) to inter-detection intervals:

1. Process 1: Continuous presence (seconds to minutes)
2. Process 2: Edge milling behavior (minutes to hours)
3. Process 3: True departure and return (hours to days)

The knot locations determine threshold values for classifying detection gaps.

Nested Doll Overlap Removal

Uses a directed acyclic graph (DAG) representing parent:child relationships between receivers. If a fish is present at a child receiver (small detection zone) while also detected at a parent receiver (large detection zone), the algorithm removes the overlapping parent detections.

Analogy: Like nested Russian dolls - if you can place an object in the innermost doll, you know it is not in the outer dolls.

Examples

See the following for complete examples:

• MAST_Project.ipynb - Interactive Jupyter notebook with full workflow
• scripts/mast_complete_project.py - Python script implementation
• Sample data - Check data/ folder for example file formats

Contributing

We welcome contributions. Please see CONTRIBUTING.md for guidelines.

Quick Development Setup:

git clone https://github.com/knebiolo/mast.git
cd mast
python -m venv venv
venv\Scripts\activate  # Windows
# source venv/bin/activate  # macOS/Linux
pip install -e .

Citation

If you use PyMAST in your research, please cite:

@article{nebiolo2024mast,
  author = {Nebiolo, Kevin P. and Castro-Santos, Theodore},
  title = {MAST: Movement Analysis Software for Telemetry data. Part I: the semi-automated removal of false positives from radio telemetry data},
  journal = {Animal Biotelemetry},
  year = {2024},
  volume = {12},
  number = {1},
  pages = {11},
  doi = {10.1186/s40317-024-00358-1},
  url = {https://doi.org/10.1186/s40317-024-00358-1}
}

Software Citation:

@software{nebiolo2025pymast,
  author = {Nebiolo, Kevin P. and Castro-Santos, Theodore},
  title = {PyMAST: Movement Analysis Software for Telemetry},
  year = {2025},
  publisher = {GitHub},
  url = {https://github.com/knebiolo/mast}
}

Additional Key Publications:

• Beeman, J.W., & Perry, R.W. (2012). Bias from False-Positive Detections and Strategies for their Removal in Studies Using Telemetry. American Fisheries Society.
• Sibly, R.M., Nott, H.M., & Fletcher, D.J. (1990). Splitting Behavior into Bouts. Animal Behavior.

License

This project is licensed under the MIT License - see LICENSE.txt for details.

Authors and Acknowledgments

Development Team:

• Kevin P. Nebiolo - Lead Developer (Kleinschmidt Associates)
• Theodore Castro-Santos - Co-Developer (USGS)

Institutional Support:

• Kleinschmidt Associates
• U.S. Geological Survey (USGS)

Community:

• Thanks to the aquatic telemetry community for feedback and testing

Support

• Issues: GitHub Issues
• Email: kevin.nebiolo@kleinschmidtgroup.com
• Documentation: See CHANGELOG.md for version history

Additional Resources

• HDF5 Documentation
• Program MARK
• R survival package

Happy fish tracking!