pubmed-research-classifier 0.1.0

Classify PubMed articles as research or non-research using a trained MLP + ModernBERT.

pubmed-research-classifier

Classify PubMed articles as research or non-research using a trained MLP on top of EMBO/ModernBERT-neg-sampling-PubMed embeddings.

Model weights, StandardScaler, and publication-type vocabulary are bundled in the package — no external model downloads are needed for the embedding-mode API.

Installation

# Embedding mode only (no sentence-transformers required)
pip install pubmed-research-classifier

# Text mode (package embeds internally)
pip install "pubmed-research-classifier[embed]"

Quick start

Text mode

from pubmed_research_classifier import classify

result = classify({
    "title": "Structural basis of CRISPR-Cas9 activity",
    "abstract": "We report crystal structures of Cas9 ...",
    "pub_types": ["Journal Article"],
    "n_authors": 8,
    "n_refs": 42,
})
# {"label": "research", "p_nr": 0.018}

Embedding mode

Pre-compute embeddings with EMBO/ModernBERT-neg-sampling-PubMed using normalize_embeddings=True, then pass them directly:

from pubmed_research_classifier import classify
import numpy as np

result = classify({
    "title_emb":     title_embedding,    # np.ndarray, shape (768,)
    "abstract_emb":  abstract_embedding, # np.ndarray, shape (768,); zeros if absent
    "has_abstract":  True,
    "length_title":  52,
    "length_abstract": 1240,
    "pub_types":     ["Journal Article"],
    "n_authors":     8,
    "n_refs":        42,
})

Batch — millions of records

results = classify(records, batch_size=128)
# Returns a list in the same order as the input.

Input fields

Field	Type	Mode	Notes
title	str	text
abstract	str or None	text	empty/None → treated as absent
title_emb	array (768,)	embed	L2-normalised
abstract_emb	array (768,)	embed	L2-normalised; zeros if absent
has_abstract	bool	embed
length_title	int	embed	auto-derived from title in text mode
length_abstract	int	embed	auto-derived from abstract in text mode
pub_types	list[str] or str	both	PubMed PT tags; comma-sep string accepted
n_authors	int	both
n_refs	int	both
has_funding	bool	both	optional; inferred from "Research Support" PTs if omitted

Output

{"label": "research",     "p_nr": 0.018}
{"label": "non-research", "p_nr": 0.921}

p_nr is P(non-research) from the model. Default threshold: 0.75 (configurable via classify(..., threshold=0.75)).

Obtaining has_funding from PubMed XML

has_funding is True when the article's PubMed XML record contains at least one <Grant> element inside a <GrantList>. It is not the same as the "Research Support, …" publication type tags (those are a separate, coarser signal also used by the model via pub_types).

If you fetch articles via the NCBI E-utilities API (efetch, XML format), you can extract it like this:

import xml.etree.ElementTree as ET

def has_funding_from_xml(article_xml: str) -> bool:
    """Return True if the PubMed XML contains at least one <Grant> entry."""
    root = ET.fromstring(article_xml)
    return len(root.findall(".//Grant")) > 0

Or, if you are working with a parsed xml.etree.ElementTree.Element object (e.g. the <PubmedArticle> node returned by your ETL pipeline):

has_funding = len(article_element.findall(".//Grant")) > 0

If you do not have access to the raw XML and only have the metadata fields, omit has_funding entirely — the package will fall back to checking whether any of the pub_types start with "Research Support", which is a reasonable proxy and is already captured separately in the model's publication-type features.

Publishing a new version to PyPI

The built artifacts live in pubmed-research-classifier/dist/.

Workflow for every new release

1. Update the model weights — copy new mlp_best.pt, scaler.joblib, and/or mlp_config.json into src/pubmed_research_classifier/_data/ and overwrite the old files.

2. Bump the version in two places:

   # pyproject.toml
   version = "0.2.0"
   

   # src/pubmed_research_classifier/__init__.py
   __version__ = "0.2.0"
   

3. Rebuild the wheel:

   cd pubmed-research-classifier
   pip install build          # first time only
   python -m build
   # produces dist/pubmed_research_classifier-0.2.0-py3-none-any.whl
   #          and dist/pubmed_research_classifier-0.2.0.tar.gz
   

4. Upload to PyPI:

   pip install twine           # first time only
   twine upload dist/pubmed_research_classifier-0.2.0*
   # Username: __token__
   # Password: <your PyPI API token>
   

   PyPI API tokens are managed at https://pypi.org/manage/account/token/. Use a project-scoped token (not account-wide) for safety.

5. Verify the release:

   pip install "pubmed-research-classifier==0.2.0" --force-reinstall
   python -c "from pubmed_research_classifier import classify; print('ok')"
   

First-time PyPI setup

If the package does not yet exist on PyPI, the first upload creates it automatically. You will need a PyPI account and a project-scoped (or account-scoped) API token. Test releases can go to https://test.pypi.org first:

twine upload --repository testpypi dist/pubmed_research_classifier-0.1.0*
pip install --index-url https://test.pypi.org/simple/ pubmed-research-classifier