PDF Mapper Module

The core PDF field extraction, mapping, embedding, and filling engine.

🚀 Quick Start

1. Configure the Module

IMPORTANT: You must configure before running!

# Copy configuration templates
cp .env.example .env
cp config.ini.example config.ini

# Edit .env - Add your API keys
nano .env

# Edit config.ini - Set your storage paths
nano config.ini

See SETUP_GUIDE.md for detailed configuration instructions.

2. Install Dependencies

# Core dependencies
pip install -r requirements.txt

# For API server
pip install -r requirements-api.txt

3. Run API Server

Set API_KEY first — the server refuses to serve requests without it (see Environment Variables below).

export API_KEY=change-me-to-a-long-random-secret
python api_server.py

Server will be available at: http://localhost:8000

Interactive docs at: http://localhost:8000/docs

📁 File Structure

modules/mapper/
├── .env.example            ← Copy to .env (add your API keys)
├── config.ini.example      ← Copy to config.ini (configure paths)
├── config.ini              ← Active configuration (DO NOT COMMIT)
├── .env                    ← Active environment (DO NOT COMMIT)
├── SETUP_GUIDE.md          ← Detailed setup instructions
├── API_SERVER.md           ← API server documentation
├── api_server.py           ← FastAPI server (run this!)
├── requirements.txt        ← Python dependencies
├── requirements-api.txt    ← API server dependencies
├── setup.py                ← Package setup
└── src/                    ← Core source code
    ├── orchestrator.py     ← Main orchestration logic
    ├── extractors/         ← PDF field extraction
    ├── mappers/            ← Field mapping (LLM)
    ├── embedders/          ← Metadata embedding
    ├── fillers/            ← PDF filling
    ├── chunkers/           ← Document chunking
    ├── groupers/           ← Field grouping
    ├── headers/            ← Header detection
    ├── validators/         ← Field validation
    └── core/               ← Configuration & logging

🎯 What This Module Does

1. Extract - Extracts form fields from PDF files
2. Map - Maps extracted fields to your data schema using LLM
3. Embed - Embeds mapping metadata into PDF for reuse
4. Fill - Fills embedded PDFs with actual data

Operations

🔧 Configuration Overview

Required Configuration

In .env:

# Choose one
CLOUD_PROVIDER=local          # For local development
# CLOUD_PROVIDER=aws          # For AWS deployment
# CLOUD_PROVIDER=azure        # For Azure deployment

# Add your LLM API key
OPENAI_API_KEY=sk-your-key-here

In config.ini:

[general]
source_type = local

[mapping]
llm_model = gpt-4o
use_second_mapper = false

[local]
cache_registry_path = /path/to/cache/hash_registry.json
output_base_path = /path/to/output

See SETUP_GUIDE.md for complete details.

🌐 Running as API Server

# Start server
python api_server.py

# In another terminal, test it
curl http://localhost:8000/health

Authentication is required. Set API_KEY to a strong secret before starting the server — every endpoint except /, /health, /docs, and /redoc requires it as the X-API-Key header, or the server responds with a config error. (For local-only experimentation you can instead set MAPPER_ALLOW_INSECURE_NO_AUTH=true, but never do this in a deployment reachable by anyone else.) Cross-origin access defaults to open (*) for convenience during local development — set MAPPER_CORS_ALLOWED_ORIGINS (comma-separated) to lock this down before deploying.

Available endpoints:

• GET / - API info
• GET /health - Health check
• POST /mapper/extract - Extract fields
• POST /mapper/map - Map fields
• POST /mapper/embed - Embed metadata
• POST /mapper/fill - Fill PDF
• POST /mapper/make-embed - Extract+Map+Embed
• POST /mapper/fill-pdf - Fill embedded PDF
• POST /mapper/check-embed-file - Check if PDF has embeddings
• POST /mapper/run-all - Complete pipeline

See API_SERVER.md for API documentation.

📦 Using as Python Module

from src.orchestrator import run_extraction, run_mapping, run_embedding, run_filling

# Extract fields
extracted = run_extraction(pdf_path, user_id, pdf_doc_id)

# Map fields
mapped = run_mapping(user_id, pdf_doc_id)

# Embed metadata
embedded = run_embedding(pdf_path, user_id, pdf_doc_id)

# Fill PDF
filled = run_filling(embedded_pdf_path, user_id, pdf_doc_id, input_data)

🧪 Testing

# Run all tests
pytest

# Run specific test
pytest tests/test_extract.py

# With coverage
pytest --cov=src

🐳 Deployment Options

Local Development

python api_server.py

Docker

docker build -t pdf-mapper .
docker run -p 8000:8000 --env-file .env pdf-mapper

AWS Lambda

See deployment/aws/ for Lambda deployment scripts.

Azure Functions

See deployment/azure/ for Azure deployment scripts.

GCP Cloud Functions

See deployment/gcp/ for GCP deployment scripts.

🔗 Integration with SDK

Once the API server is running, install the SDK:

cd ../../sdks/python
pip install -e .

# Use CLI
pdf-autofiller --api-url http://localhost:8000 extract input.pdf

# Or Python
from pdf_autofiller import PDFMapperClient
client = PDFMapperClient("http://localhost:8000")
result = client.extract("input.pdf", 1, 100)

📚 Documentation

• SETUP_GUIDE.md - Configuration setup
• API_SERVER.md - API documentation
• INSTALLATION_GUIDE.md - Installation details
• ../../docs/ - Complete project documentation

🔍 Troubleshooting

Module 'boto3' not found

# In config.ini, set:
[general]
rag_api_url = 
# Leave empty to disable RAG

API key errors

# Make sure .env has your key:
OPENAI_API_KEY=sk-your-actual-key-here

Import errors

# Install dependencies:
pip install -r requirements.txt

Server won't start

# Install API dependencies:
pip install -r requirements-api.txt

⚙️ Environment Variables

Key environment variables (set in .env):

📝 License

See LICENSE file in project root.

🤝 Contributing

See CONTRIBUTING.md for guidelines.

Quick Command Reference

# Setup
cp .env.example .env
cp config.ini.example config.ini
pip install -r requirements.txt requirements-api.txt

# Run server
python api_server.py

# Test
curl http://localhost:8000/health
pytest

# Install SDK (for client usage)
cd ../../sdks/python && pip install -e .