graphrag-tagger
A lightweight toolkit for extracting topics from PDFs and visualizing their connections using graphs.
Overview
graphrag-tagger automates topic extraction from PDF documents and builds graphs to visualize relationships between text segments. It offers a modular pipeline for processing text, applying topic modeling, refining results with an LLM, and constructing a graph-based representation of topic similarities.
Key Features
✅ PDF Processing – Extracts text from PDFs efficiently.
✅ Text Segmentation – Splits extracted text into manageable chunks.
✅ Topic Modeling – Supports two methods:
- Scikit-learn: Classic Latent Dirichlet Allocation (LDA) for topic extraction.
- ktrain: A deep-learning-based approach with vocabulary filtering.
✅ LLM-Powered Refinement – Uses a language model to clean and classify topics.
✅ Graph Construction – Builds topic similarity graphs using network analysis.
Core Dependencies
- PyMuPDF – Extracts text from PDF files.
- scikit-learn & ktrain – Performs topic modeling.
- LLM Client – Enhances and refines extracted topics.
- networkx – Constructs and analyzes graphs.
Installation
Ensure you have Python installed, then build and install the package locally:
python -m build
pip install .
Usage
Extract Topics from PDFs
Run the topic extraction pipeline on a folder of PDFs:
python -m graphrag_tagger.tagger \
--pdf_folder /path/to/pdfs \
--output_folder /path/to/output \
--chunk_size 512 \
--chunk_overlap 25 \
--n_features 512 \
--min_df 2 \
--max_df 0.95 \
--llm_model ollama:phi4 \
--model_choice sk
Build a Topic Similarity Graph
Generate a graph from the extracted topics:
python -m graphrag_tagger.build_graph \
--input_folder /path/to/output \
--output_folder /path/to/graph \
--threshold_percentile 97.5
How It Works
1️⃣ PDF Processing – Extracts raw text from documents.
2️⃣ Text Segmentation – Divides the text into structured chunks.
3️⃣ Topic Modeling – Uses either LDA or ktrain-based modeling to extract key topics.
4️⃣ LLM-Based Refinement – Cleans and classifies topics for better accuracy.
5️⃣ Graph Construction – Builds a network where:
- Nodes represent text chunks.
- Edges represent topic similarities.
- The graph reveals clusters and connections between document sections.
Contributing
Contributions are welcome! Feel free to submit issues or pull requests.
Metadata
Release files for graphrag-tagger 0.1.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| graphrag_tagger-0.1.1.tar.gz | 16.9 kB | Without provenance | Details for graphrag_tagger-0.1.1.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| graphrag_tagger-0.1.1-py3-none-any.whl | Python 3 | none | any | Details for graphrag_tagger-0.1.1-py3-none-any.whl |
Total release size: 37.0 kB
Release files / graphrag_tagger-0.1.1.tar.gz
| Download URL | graphrag_tagger-0.1.1.tar.gz |
|---|---|
| Size | 16.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
7de8162272a8583880fc58bf582be8ce4ea9d8bfe0e5d0e0ef78c5a559c30540
|
|
BLAKE2b-256 checksum How to use checksums |
176d0c1c9433d77a5001a178fff268743cfb4b4a6d6f29558b203d71270f0716
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.13
|
Release files / graphrag_tagger-0.1.1-py3-none-any.whl
| Download URL | graphrag_tagger-0.1.1-py3-none-any.whl |
|---|---|
| Size | 20.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
a63541ab263c321cb0f2dcf6452aaa41f3f3e6158933ffaa0cbef895766a2241
|
|
BLAKE2b-256 checksum How to use checksums |
271b3d70115d05dd7ad1054285c369acac9fbf6065f20cb8270fb3d4400b006a
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.13
|
Security for graphrag-tagger 0.1.1
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.