Research Helpers

Research Helpers is a set of utilities for managing code-intensive research projects and their accompanying papers. Includes shared figure styling, LaTeX table generation and rendering, a build pipeline that keeps a paper's generated tables and figures from drifting out of date, submission packaging for arXiv and Zenodo, structured logging, and a parameter-sweep engine for scheduler job arrays.

Read the documentation

Features

• Generated Tables and Figures: Build the paper's tables and figures from the repository's own data
• Drift Detection: Detect when the paper and the data differ
• Numbers in the Prose: Track every number a sentence states back to the data that supports it, and audit the prose for any untracked values
• Several Documents: Give a technical note or a supplement its own generated tables, figures, and checks
• LaTeX Assembly: Compose tables for LaTeX and also display them in Jupyter
• Shared Figure Styling: Consistent look across LaTeX and Jupyter
• Submission Packaging: Automatic preparation and pre-checking for arXiv and Zenodo
• Parameter Sweeps: Plan a grid, run it as a scheduler job array, resume if necessary, and collect the results
• Campaigns: Put hundreds of sweeps through a few arrays, resume them, and recover from elements that ran twice
• Aggregation: Combine repeated runs into confidence intervals, with no dependencies on the cluster side
• Structured Logging: Console and file output at separate levels, with progress bars and colour output

Installation

The core package has no third-party dependencies. Specifically, the code path that runs on a cluster only depends on the standard library. Planning a sweep, running an array task, checking its status, sizing the array and computing a confidence interval all work on a node with nothing but Python on it. Everything that needs a library is a capability, installed as an extra:

pip install research-helpers[figures]   # matplotlib, seaborn
pip install research-helpers[latex]     # LaTeX table assembly and rendering (stdlib only)
pip install research-helpers[log]       # structlog, colorama, tqdm
pip install research-helpers[sweep]     # pandas, pyarrow (planning, running, and intervals need neither)

Requires Python 3.11 or newer.

Quick Start

Wire up the project

A project's wiring—i.e., where the paper lives, how wide it is, where runs are written—goes in the [tool.research-helpers] section of the pyproject.toml file.

[tool.research-helpers.paper]
main = "tex/paper.tex"
tables-dir = "tex/tables"
figures-dir = "tex/figures"
text-width-pt = 468.0 # as reported by \showthe\textwidth

[tool.research-helpers.figures]
profile = "print"
palette = "colorblind"
dpi = 300

research-helpers doctor shows every setting, the value in force, and where it came from.

Keys and values are described in the configuration reference.

Generate a table from data

import json

from research_helpers.build import TABLES, Registry
from research_helpers.latex import half_up, header, table

tables = Registry(TABLES)


@tables.register('tab:scores')
def scores() -> str:
    """Accuracy on the held-out set."""
    data = json.loads(SCORES_PATH.read_text())
    return table(
        spec='lr',
        header_rows=[[header('System'), header('Accuracy')]],
        body_rows=[[name.title(), half_up(v['accuracy'] * 100)] for name, v in data.items()],
        caption='Accuracy on the held-out set.',
        label='tab:scores',
    )


if __name__ == '__main__':
    raise SystemExit(tables.main())

The label names the file, so this is written to scores.tex and the paper reads it with \input{tables/scores}:

$ python -m demo.tables --install
wrote build/tables/scores.tex
installed tex/tables/scores.tex

Check for drift

$ python -m demo.tables --check
OK: all 1 tables in the paper are current

If the data is changed without rebuilding, the check fails:

$ python -m demo.tables --check
stale, the paper is behind the data: tab:scores
to fix: rebuild and install the tables
$ echo $?
1

For a full example, see adding a generated table.

Style a figure for the page

import matplotlib.pyplot as plt

from research_helpers.figures import apply_style, save

apply_style(profile='print')  # \textwidth across, paper font sizes
fig, ax = plt.subplots()  # the profile's size is already in rcParams
ax.plot(iterations, accuracy, marker='o')
save(fig, 'tex/figures/learning-curve.png')

See details in the figures guide.

Run a parameter sweep

from research_helpers.sweep import Sweep

sweep = Sweep()


@sweep.context
def prepare(manifest, run_dir):
    """Loaded once per array task."""
    return load_corpus(manifest.metadata['corpus'])


@sweep.evaluate
def evaluate(params, corpus):
    """One parameter combination."""
    return {'f1': score(corpus, **params)}


if __name__ == '__main__':
    raise SystemExit(sweep.main())

$ python sweep_demo.py plan --config sweep.toml --run-dir runs/demo --tasks 4
planned 12 combinations over 4 array tasks (~3 per task)
manifest: runs/demo/manifest.json

submit with:
  sbatch --array=1-4 <your sbatch script> runs/demo

The manifest is written once and the tasks read it, so re-submitting the two that were killed reproduces exactly their original slices instead of repartitioning the grid. Resumption is per combination, not per task, so a job that hits its wall-time keeps everything it finished. status and run need only the standard library, so nothing needs to be installed to run a progress check from a login node.

Full details are discussed in the sweeps guide.

Package a submission

$ research-helpers arxiv --dry-run
  paper.tex                       43.1 KB
  paper.bbl                       82.1 KB
  tables/scores.tex                0.3 KB
  figures/learning-curve.png      66.3 KB

  4 files, 0.19 MB
  bbl format 3.3 (TeX Live 2025)

ready to upload: select xelatex and TeX Live 2025

(dry run, nothing written)

The target files are keyed by the path they take inside the submission, so what resolves locally will also resolve at the destination. The checks cover filenames arXiv rejects, a .bbl that is missing, stale, or in a format the selected TeX Live will not read, and microtype font expansion under an engine that has none. --tar packs it reproducibly, so re-packing an unchanged submission gives an identical file.

See the submission guide.

Logging

from research_helpers.log import get_logger, setup_logging

setup_logging()
log = get_logger(__name__)

log.info('scoring', corpus='perseus', sentences=18_000)

[INFO    ] 19:41:47 __main__ scoring (corpus=perseus, sentences=18000)

The console and the log file take separate levels. Colour is applied only when the stream is a terminal, so no escape codes are included in a redirected log.

Details in the logging guide.

Documentation

The full documentation includes:

• Configuration: Every section, key, type, and default, plus units and settings resolution
• Guides: Task-focused walkthroughs
  • Adding a generated table: Full table pipeline
  • Numbers in the prose: Registering, checking, and auditing the numbers sentences state
  • Several documents: A second manuscript with its own artefacts and checks
  • Figures: Profiles, scoped styling, saving versus rendering
  • LaTeX tables: Assembling tables and reading them back
  • Parameter sweeps: Planning, running, resuming, and collecting
  • Many sweeps through few arrays: Plan lists, queues, resuming, and quarantine
  • arXiv and Zenodo: Submission checks and reproducible archives
  • Logging: Setup, levels, colour, and progress bars
• API Reference: Complete API documentation

License

The project is licensed under the MIT License, allowing free use, modification, and distribution.