grobid-tei

grobid-tei is a reusable Python client for an independently managed GROBID service and a parser for its TEI XML output. It supports Python 3.10 and newer. It does not start Docker or manage the GROBID service lifecycle.

Installation

pip install grobid-tei

For a local checkout, install the package and its test tools with uv:

uv pip install .
uv sync --extra test

Pre-commit checks

Install pre-commit and enable the repository hooks:

uv tool install pre-commit
pre-commit install
pre-commit run --all-files

The hooks check common file issues, format with Black, lint with Ruff, ensure Python 3.10-compatible syntax, and run the test suite. The copied TEI XML fixture is excluded from whitespace-fixing hooks so its source representation is preserved.

Convert a PDF to TEI

Start or otherwise provide a GROBID service yourself, then configure its full base URL explicitly:

from pathlib import Path

from grobid_tei import GrobidClient, TEIDocument

client = GrobidClient(service_url="http://localhost:8070", timeout=180)
if not client.is_available():
    raise RuntimeError("GROBID is not available")

tei_xml = client.process_fulltext(Path("paper.pdf"))
Path("paper.tei.xml").write_bytes(tei_xml)

document = TEIDocument.from_bytes(tei_xml)
print(document.metadata())
print(document.metadata(abstract=True))
print(document.abstract())
print(document.references(include_citation_counts=True))

process_fulltext returns XML bytes. To enable GROBID consolidation explicitly, pass consolidate_header=True and/or consolidate_citations=True.

Parse an existing TEI file

No running GROBID service is needed to parse an existing file:

from pathlib import Path

from grobid_tei import TEIDocument

document = TEIDocument.from_file(Path("paper.tei.xml"))
print(document.grobid_version())
print(document.keywords())
print(document.author_details())

for paragraph in document.iter_paragraphs():
    print(paragraph)

Public API

• GrobidClient(service_url=..., timeout=180.0) provides is_available(), version(), optional check_compatibility(), and process_fulltext(...). The compatibility check rejects known-incompatible versions and warns, but proceeds, for versions not yet tested with this package. It is not run automatically during processing. IncompatibleGrobidVersionError identifies a known-incompatible server version.
• TEIDocument.from_bytes(...) and TEIDocument.from_file(...) create a parsed document. It provides to_bytes(), grobid_version(), metadata(), abstract(), keywords(), author_details(), references(), citations_by_section(), iter_paragraphs(), and annotate_reference_ids(...).
• TEIReference is a mapping of generic lowercase bibliographic fields (title, author, year, doi, journal, and similar), with the GROBID identifier in .tei_id and an optional citation count in .in_text_citation_count. The identifiers and count are not included in its bibliographic field mapping.
• GrobidServiceError describes service request failures; TEIParsingError describes unsafe or invalid TEI XML. Both inherit from GrobidTEIError.

The parser uses defusedxml for safe XML parsing. annotate_reference_ids() applies only a caller-supplied mapping from TEI IDs to caller IDs; it performs no reference matching or workflow decisions.

License

This project is licensed under the MIT License.