Skip to main content

sslabdata

PyPI Python License: MIT Tests

sslabdata compiles BibTeX and a little YAML into one schema-specified document — works, people, projects and the links between them — that any website, CV or script can read.

Most academics already keep good BibTeX. What they do not have is that bibliography as data: authors linked to the people in the group, papers linked to the projects they belong to, names normalised, LaTeX resolved to plain Unicode text. sslabdata does that one job and writes the result to a single YAML or JSON file, specified by a published JSON Schema you can check it against.

sslabdata --config lab.yaml --output lab.yml
  • SPEC.md — the normative contract: what the strings are, what order the lists are in, which fields are derived, when the version changes.
  • CHANGELOG.md — what changed at each release, and what it replaced.
  • schema/v6/output.schema.json — the document's JSON Schema. Published versions are immutable and live at their own paths; schema/v3/, schema/v4/ and schema/v5/ are still there.
  • tests/COVERAGE.md — every input case sslabdata supports, and every case it does not, with the fixture and test for each.

What sslabdata is not

sslabdata is not a CMS and not a site generator. It does not build a website, own your pages or manage your content. News, openings, teaching pages, press and galleries are prose with no shared structure to compile, and they belong in your site repository. SPEC.md §8 gives the evidence for that boundary and the destination for each content type it leaves out.

sslabdata emits data. Rendering it is your renderer's job.

Install

pip install sslabdata

To work on sslabdata itself, install from a clone instead:

git clone https://github.com/siddhss5/sslabdata.git
cd sslabdata
pip install -e ".[test]"
pytest

The test extra installs pytest and jsonschema, which the tests need.

Start a new lab

sslabdata init mylab
cd mylab && sslabdata --config lab.yaml --validate --strict

init writes lab.yaml, bib/publications.bib, people.yaml, projects.yaml and collaborators.yaml into mylab/ (the current directory if you name none), each with one fictional record and a comment on each field. Replace them with your own. It never overwrites a file that is already there and names each one it refuses; --force overwrites those files and nothing else. The paths in lab.yaml are relative to the directory you run sslabdata from, so run it from mylab/.

Write lab.yaml

lab:
  name: "My Lab"
  description: "What our lab does"
  university: "University Name"
  website: "https://mylab.example.org"

bib_dir: "data/bib"
bib_files:
  - name: "journal.bib"
    category: "Journal Papers"
  - name: "conference.bib"
    category: "Conference Papers"

pdf_base_url: "https://mylab.example.org/pdfs"  # optional; leave out to guess no PDF links
people_file: "data/people.yaml"       # optional
projects_file: "data/projects.yaml"   # optional
collaborators_file: "data/collaborators.yaml"  # optional

Each bib_files entry's name is a name under bib_dir: it is emitted as the work's source.file, so it may be neither absolute nor leave bib_dir, and sslabdata rejects such a name rather than rewriting it (SPEC.md §5).

Paths are relative to the directory you run sslabdata from. examples/demo/lab.yaml is a complete example, built from the fictional Example Lab in examples/demo/.

Then compile it:

sslabdata --config lab.yaml --validate            # report counts and problems
sslabdata --config lab.yaml --unresolved          # list unmatched author names
sslabdata --config lab.yaml --output lab.yml      # write the document
sslabdata --config lab.yaml --format json --output lab.json
sslabdata --config lab.yaml --validate --strict   # fail on every problem
sslabdata --config lab.yaml --validate --format json   # problems as JSON

--validate exits 0 when it finds no errors and 1 when it does, or when the run fails outright. An author who matched nobody is reported but is not an error; most are external collaborators. It checks the configuration, the people and projects files and every entry, not the output against the JSON Schema. Every problem is reported under a stable code.

--strict turns every coded problem into an error, except those about authors who matched no lab member and redefined @string macros; an error exits 1 and an export then writes nothing. With --validate or --unresolved, --format json prints the problems as one JSON array on standard output. SPEC.md §1 owns the flags, exit codes, streams and precedence when you pass more than one mode, the diagnostic codes with their classes, and the JSON shape.

Inputs

BibTeX (required)

Standard .bib files. These are the fields sslabdata interprets. A field not listed here is carried through in bibtex but is not interpreted and affects nothing else:

Field Becomes
title title, LaTeX converted to plain Unicode text; $...$ math kept as TeX
author authors, one authorship per name, each with its position, a readable name, its given / von / family / suffix parts (or literal for a brace-protected name), equal_contribution, a resolution record, and exactly one of person_id and collaborator_key
editor editors, read by the same machinery. Editing a volume is not an authorship: editors are in nobody's work_ids and produce no collaborator
year year, and the sort order of the works list. null, with a diagnostic, when the entry has none
journal / booktitle / school / institution venue, as {kind, name} — the one place sslabdata normalises across entry types. null when the entry names no container
volume, number, pages, series, edition, publisher, address, organization, chapter, month, howpublished, type Properties of the work, under BibTeX's own names and with BibTeX's own meanings
doi, isbn, issn, eprint + archivePrefix (or eprinttype) identifiers, an open map from scheme to a list of identifiers, plus the links built from them. An eprint's scheme is the repository archivePrefix or eprinttype named, lower-cased, so that field needs no property of its own — and an eprint in a repository other than arXiv gets no arXiv link
abstract abstract
note note
award awards, a list of {name, year} (see below). note is never read for awards
url A link of kind video when its host is YouTube or Vimeo (or a subdomain of either), otherwise of kind url
video A link of kind video, whatever its host, so url can hold the work's website
pdf The work's one link of kind pdf. An entry without it gets pdf_base_url plus its citation key, when pdf_base_url is set
project project_ids (see below)
crossref An error: the field is rejected, not resolved, and the run fails. Write the fields out on the entry itself

The entry is also re-serialized into a bibtex field, so fields sslabdata does not interpret are still carried. It is a re-serialization, not a copy (SPEC.md §5).

Awards

A paper's awards go in its award field. Several are separated by and, as the names in author are, and braces keep an and inside one name. An award may start with the year it was given, as YYYY: and a space, when that is not the paper's year:

award = {2026: Test of Time Award and {Best Systems and Software Paper Award}}

Each award becomes {name, year} in the work's awards, in the order written, with its name converted from LaTeX as title is. An award without a year takes the work's year, or null when the work has none. awards is [] for a work with no award field. An empty award, or a prefix that looks like a year and is not four digits, a colon and a space, is reported (BIB-AWARD-EMPTY, BIB-AWARD-YEAR-MALFORMED); a malformed prefix stays in the name. Awards a person holds, such as fellowships, are not part of the document.

The project tag

sslabdata adds one custom BibTeX field, project, to link a paper to a research project:

@inproceedings{cote2024pantry,
  title     = {Where Does This Go? Object Placement in Unfamiliar Kitchens},
  author    = {C{\^o}t{\'e}, Carol and Davis, Dave and Adams, Alice},
  booktitle = {Proceedings of the Conference on Robot Learning Systems},
  year      = {2024},
  eprint    = {2406.99812},
  archivePrefix = {arXiv},
  award     = {Best Paper Award},
  project   = {homebot}
}

Several projects go in one field, comma-separated: project = {homebot, sharedcontrol}. Each project in the document then back-links the works tagged with it, and the people who wrote them.

People (optional, data/people.yaml)

A list of lab members and alumni. aliases tells sslabdata how to match BibTeX author names to people:

- id: "bbrown"
  name: "Bob Brown"
  aliases: ["B. Brown"]
  role: "phd_student"
  status: "current"
  website: "https://example.org/people/bbrown"
  co_advisor: "Peggy Park"
  start_year: 2021
  bio: |
    Bob Brown is a PhD student advised by Alice Adams and Peggy Park.
    He works on shared control for assistive robot arms.

- id: "iingram"
  name: "Ivan Ingram"
  aliases: ["I. Ingram"]
  role: "phd_student"
  status: "alumni"
  start_year: 2016
  end_year: 2022
  degree: "PhD"
  thesis_title: "Learning Grasp Affordances from Play"
  current_position: "Research Scientist, Example Robotics Inc."

id and name are required. role is any non-empty string, so any lab's roles fit; status is current (the default) or alumni.

bio is a short biography in plain text, not Markdown or HTML: it is emitted as written and a renderer escapes it. Line breaks are kept as YAML reads them, so a block scalar (|) keeps each line, and a renderer may treat a blank line as a paragraph break. A person without one gets bio: null; an empty string is emitted as written, as any other person field is. Keep the structured facts in their own fields: nothing reads degree, years or current_position out of a bio, so an alumni line such as "PhD 2022, now Research Scientist at Example Robotics" is for the renderer to build from those fields.

External co-authors (optional, data/collaborators.yaml)

A list of co-authors outside the lab whose spellings you want grouped together. It only decides which authorships share one collaborators entry; it never makes anyone a lab member and never produces a person_id:

- name: "Priya Patel"
  aliases: ["P. Patel"]

Patel, Priya and Patel, P. are then one collaborator, with grouped_by: declared. A different Patel, Pradeep is not joined, because nothing declares him. A name or alias that a lab member already declares is reported under RESOLVE-COLLABORATOR-ALIAS-IS-MEMBER and left to the member.

Projects (optional, data/projects.yaml)

- id: "homebot"
  title: "Household Manipulation"
  description: "Robots that tidy up, fetch things and put them away in real homes."
  website: "https://example.org/projects/homebot"
  image: "images/projects/homebot.jpg"
  status: "active"

id and title are required; status is active (the default) or completed. image is a URL or a site path, the same kind of value as a person's photo, and is null when absent. It is carried as plain text: deciding which URLs are safe to render is the renderer's job.

Checking inputs in an editor

Each input file has a JSON Schema in schema/input/v2/: lab.schema.json, people.schema.json, projects.schema.json and collaborators.schema.json. The wheel installs them under sslabdata/schema/input/v2/. An editor can use them to check and complete the files as you write; --validate stays the check, and also reports what a schema cannot see, such as a repeated id or a missing file. With the YAML language server (the VS Code YAML extension, among others), name the schema in a comment at the top of the file:

# yaml-language-server: $schema=https://raw.githubusercontent.com/siddhss5/sslabdata/input-schema-v2/schema/input/v2/people.schema.json
- id: "aadams"
  name: "Alice Adams"
  role: "professor"

The URL is the schema's $id, served from the input-schema-v2 tag, which is never moved (SPEC.md §6). A file that names v1 still validates, because v1 allows keys it does not list, but only v2 checks and completes bio. Use lab.schema.json, projects.schema.json or collaborators.schema.json in the same way for the other files.

How author matching works

sslabdata matches the structured parts of each BibTeX author name (given, von, family, suffix) to lab members: first on the full name against each person's name and any alias written in full, then, only when the name is itself abbreviated, on a declared alias. Nothing is guessed. A name that fits more than one person gets no person_id and is reported under RESOLVE-AMBIGUOUS-NAME; a near miss is never linked and is reported under RESOLVE-SUGGESTION with the ids it might be. Both are warnings, so --validate lists them and still exits 0. To resolve one, add the spelling to that person's aliases. SPEC.md gives the normalisation and the order the match decides in.

A name that matches nobody keeps person_id: null and its authorship references a collaborators entry instead, by collaborator_key. sslabdata --config lab.yaml --unresolved lists those names so you can add aliases — or, if you have configured no people_file, tells you resolution was never attempted.

collaborators is a grouping over unresolved authorships, not a list of humans, and its key is a lookup key, not an identity. The grouping can be wrong in both directions, so sslabdata reports a key that spans more than one spelling and an initials-only key that could be any of several fuller ones, and collaborators_file lets you join spellings yourself. What the grouping does and does not promise is in SPEC.md §5.

Reading the document

The output is one YAML or JSON file. Strings in it that are meant for display are plain Unicode text: not HTML, not Markdown, not escaped. They are untrusted, and a title really may contain <, &, " or *, so escape them when you render them. Math is the one markup exception and stays delimited by $…$. bibtex, identifiers and URLs are not display text.

sslabdata enforces that rule only where it converts LaTeX from BibTeX. Strings you supply directly in YAML, and everything under lab, are copied through as written and never checked, so keeping them plain is on you. SPEC.md §2 draws the line precisely.

Validate a document against the schema with any JSON Schema tool. sslabdata does not do this for you, and does not depend on a validator — jsonschema is a test-only dependency, so install it first. The installed package carries the current schema:

pip install jsonschema
python -c "
import json, yaml, jsonschema
from importlib.resources import files
schema = json.loads(files('sslabdata.schema').joinpath('v6/output.schema.json').read_text())
jsonschema.Draft202012Validator(schema).validate(yaml.safe_load(open('lab.yml')))
print('valid')
"

Python API

The CLI is the reference compiler. The Python API is a convenience wrapper over the same pipeline:

from sslabdata import LabDataConfig, assemble, export_to_yaml

config = LabDataConfig.from_yaml("lab.yaml")
data = assemble(config)

export_to_yaml(data, "lab.yml")

for work in data.works:
    authors = ", ".join(a.name for a in work.authors)
    print(f"{work.title} ({authors})")

Public: the names exported from sslabdata/__init__.py. Everything else — sslabdata.parsers, sslabdata.loaders, sslabdata.resolver — is private and may change without a version bump.

The demo renderer

sslabdata-site renders the Example Lab document as a website (what it looks like). It is an optional downstream consumer, not part of sslabdata and not part of what sslabdata promises; it installs sslabdata from a pinned tag or commit and keeps its own copy of the demo. sslabdata ignores a site: section in lab.yaml, so a renderer can keep its own settings there.

Before a release (maintainers)

These steps are run from a clone: tools/ is not in the published package. Before publishing a sslabdata release, build sslabdata-site against the candidate: run its Release gate workflow with the candidate's git ref as sslabdata_ref. It builds without deploying, and keeps the renderer's toolchain out of this repository's CI.

Also run the smoke check over real, messy bibliographies. It compiles each .bib file under a directory on its own. It must report no crashes and no uncoded lines. Review any LaTeX remnants it lists. A local TeX Live installation has a directory of such files. Nothing is fetched, and nothing from it is committed. It is not in CI because it needs TeX:

uv run python tools/smoke.py /usr/local/texlive/2025/texmf-dist/bibtex/bib

Releasing (maintainers)

Releases are published by .github/workflows/release.yml through PyPI Trusted Publishing; no token is stored anywhere. The version is written in pyproject.toml and in sslabdata/__init__.py, and every tag must equal it.

  1. Rehearse on TestPyPI. Set both versions to a release candidate, such as 3.0.0rc1, merge that, then tag the merge commit and push the tag: git tag v3.0.0rc1 && git push origin v3.0.0rc1. The workflow builds and checks the distributions, runs the full test suite, publishes those exact files to TestPyPI, and installs sslabdata==3.0.0rc1 back from it. A failed rehearsal is repeated as rc2, since an index accepts each version once.

  2. Release. Set both versions to 3.0.0, date the CHANGELOG heading, merge, and push the tag v3.0.0. The PyPI job waits in the release environment for a reviewer's approval, then publishes the files the run checked and creates the GitHub Release with them. A release that ships a new schema version also pushes that version's tag, schema-vN or input-schema-vN, at the same commit, since the schema's $id points there.

  3. Approve from the Actions page, or from the command line:

    RUN=$(gh run list -R siddhss5/sslabdata --workflow release.yml --event push \
          --limit 1 --json databaseId -q '.[0].databaseId')
    gh api repos/siddhss5/sslabdata/actions/runs/$RUN/pending_deployments \
      -q '.[] | "\(.environment.name) \(.environment.id)"'   # what waits
    gh api -X POST repos/siddhss5/sslabdata/actions/runs/$RUN/pending_deployments \
      -F 'environment_ids[]=<id>' -f state=approved -f comment='Release'
    

A manual run of the workflow only builds and checks, unless its publish_testpypi box is ticked.

Dependencies

  • pybtex — BibTeX parsing
  • pylatexenc — LaTeX to Unicode text
  • pyyaml — YAML I/O

No network calls. All processing is local and offline.

License

MIT License. Copyright (c) 2024 Personal Robotics Laboratory, University of Washington.

Metadata

Release files for sslabdata 5.0.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sslabdata 5.0.0
File Size Uploaded Provenance Actions
sslabdata-5.0.0.tar.gz 121.7 kB With provenance Details for sslabdata-5.0.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for sslabdata 5.0.0
File Interpreter ABI Platform Actions
sslabdata-5.0.0-py3-none-any.whl Python 3 none any Details for sslabdata-5.0.0-py3-none-any.whl

Total release size: 204.2 kB

Release files / sslabdata-5.0.0.tar.gz

Download URL sslabdata-5.0.0.tar.gz
Size 121.7 kB
Tags Source
SHA-256 checksum
How to use checksums
a6baa041cc2d0f644d1b78180e74236d2c0b0083690eb035dd19325152d6bb58
BLAKE2b-256 checksum
How to use checksums
745bf215a41e1e4630cd9cf548f75d160811e27c667e430c2c17f8b7a02e1472
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 1, 2026.

Transparency log

Release files / sslabdata-5.0.0-py3-none-any.whl

Download URL sslabdata-5.0.0-py3-none-any.whl
Size 82.5 kB
Tags Python 3
SHA-256 checksum
How to use checksums
5c704c5ff66cc5e9f7fd3c296bc967a0af930f1f125acdb5cd3c43b8c70c4bb8
BLAKE2b-256 checksum
How to use checksums
1af5b6342f34bb177a0c47a79f001eb4a9eb0c53595ff76979a77ef3e2f08d60
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.14

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 1, 2026.

Transparency log

Security for sslabdata 5.0.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for sslabdata-5.0.0.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Oct 1, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page