TextSpitter

Transforming documents into insights, effortlessly and efficiently.

Built with the tools and technologies:

Table of Contents

• Table of Contents
• Overview
• Features
• Project Structure
• Getting Started
  • Prerequisites
  • Installation
  • Usage
  • Testing
• Roadmap
• Contributing
• License

Overview

TextSpitter is a lightweight Python library that extracts text from documents and source-code files with a single call. It normalises diverse input types — file paths, BytesIO streams, SpooledTemporaryFile objects, and raw bytes — into plain strings, making it ideal for pipelines that feed text into LLMs, search engines, or data-processing workflows.

Why TextSpitter?

• 📄 Multi-format extraction — PDF (PyMuPDF + PyPDF fallback), DOCX, TXT, CSV, and 50 + programming-language file types.
• 🔌 Stream-first API — accepts file paths, BytesIO, SpooledTemporaryFile, or raw bytes; no temp files required.
• 🛠️ Optional structured logging — install textspitter[logging] to add loguru; falls back to stdlib logging transparently.
• 🖥️ CLI included — uv tool install textspitter gives you a textspitter command for quick one-off extractions.
• 🚀 Automated CI/CD — GitHub Actions run the test matrix (Python 3.12–3.14) and publish docs to GitHub Pages on every push.

Features

Project Structure

TextSpitter/
├── .github/
│   └── workflows/
│       ├── docs.yml             # pdoc → GitHub Pages
│       ├── python-publish.yml   # PyPI release
│       └── tests.yml            # pytest matrix (3.12 – 3.14)
├── TextSpitter/
│   ├── __init__.py              # TextSpitter() + WordLoader public API
│   ├── cli.py                   # argparse CLI entry point
│   ├── core.py                  # FileExtractor class
│   ├── logger.py                # Optional loguru / stdlib fallback
│   ├── main.py                  # WordLoader dispatcher
│   ├── py.typed                 # PEP 561 marker
│   └── guide/                   # pdoc documentation pages (subpackage)
├── tests/
│   ├── conftest.py              # shared fixtures (log_capture)
│   ├── test_cli.py
│   ├── test_file_extractor.py
│   ├── test_txt.py
│   └── ...
├── CHANGELOG.md
├── CONTRIBUTING.md
├── pyproject.toml
└── uv.lock

Getting Started

Prerequisites

• Python ≥ 3.12
• uv (recommended) or pip

Installation

From PyPI:

pip install textspitter

# With optional loguru logging
pip install "textspitter[logging]"

Using uv:

uv add textspitter

# With optional loguru logging
uv add "textspitter[logging]"

As a standalone CLI tool:

uv tool install textspitter

From source:

git clone https://github.com/fsecada01/TextSpitter.git
cd TextSpitter
uv sync --all-extras --dev

Usage

As a library (one-liner):

from TextSpitter import TextSpitter

# From a file path
text = TextSpitter(filename="report.pdf")
print(text)

# From a BytesIO stream
from io import BytesIO
text = TextSpitter(file_obj=BytesIO(pdf_bytes), filename="report.pdf")

# From raw bytes
text = TextSpitter(file_obj=docx_bytes, filename="contract.docx")

Using the WordLoader class directly:

from TextSpitter.main import WordLoader

loader = WordLoader(filename="data.csv")
text = loader.file_load()

As a CLI tool:

# Extract a single file to stdout
textspitter report.pdf

# Extract multiple files and write to a combined output file
textspitter file1.pdf file2.docx notes.txt -o combined.txt

Testing

uv run pytest tests/

# With coverage
uv run pytest tests/ --cov=TextSpitter --cov-report=term-missing

Roadmap

• Stream-based API (BytesIO, SpooledTemporaryFile, raw bytes)
• CLI entry point (uv tool install textspitter)
• Optional loguru logging with stdlib fallback
• Programming-language file support (50 + extensions)
• CI matrix (Python 3.12 – 3.14) + GitHub Pages docs
• Async extraction API
• CSV → structured output (list of dicts)
• PPTX support

Contributing

• 💬 Join the Discussions: Share insights, give feedback, or ask questions.
• 🐛 Report Issues: Submit bugs or log feature requests.
• 💡 Submit Pull Requests: Review open PRs or submit your own.

License

TextSpitter is released under the MIT License.