pse 0.0.1

Proxy Structuring Engine: Ensures AI-generated output adheres to predefined schemas without compromising creativity or context.

Proxy Structuring Engine (PSE)

Bringing Order to Chaos: Efficient schema-guided sampling for LLMs

Overview • Features • Benchmarks

Overview

The Proxy Structuring Engine (PSE) works in tandem with LLMs to ensure generated output conforms to a defined JSON schema.

The PSE enables error free custom tool calling, complex multi-step reasoning, and unlocks new creative possibilities for AI applications.

PSE achieves this through a novel schema-guided sampling approach, leveraging a Directed Acyclic Word Graph (DAWG) and finite state machines.

Installation

pip install pse

Features

• JSON Schema Sampling: Enforces schema constraints while maintaining creativity in model outputs.
• Enhanced Tool Calling: Enables precise tool integration by guaranteeing valid JSON output, streamlining workflows and automation.
• Universal Compatibility: Works with any LLM that provides logits or log probabilities, both locally and via API.
• Enhanced Creativity: Balances structure with quality, generating actionable and creative outputs that meet your schema requirements.
• Performance Optimized: Incorporates several optimizations for speed and efficiency, including:
  • DAWG (Directed Acyclic Word Graph): Efficiently validates tokens against the schema.
  • Lazy Evaluation with Logits: Processes tokens only as needed.
• Expanded Schema Support: Supports JSON Schema with plans to expand to other formats (SQL, Cypher, Python, U-DIFF).
• Direct HuggingFace Integration: Seamlessly integrates with HuggingFace transformers.
• Comprehensive Unit Testing: Ensures code reliability with 90% test coverage.
• Detailed Documentation and Type Hinting: Improves readability and developer experience.
• Hooks for Custom Logic: start_hook and end_hook callbacks enable custom logic injection.
• Robust Error Handling: Facilitates debugging and integration.

Benchmarks

The Proxy Structuring Engine consistently outperforms traditional sampling methods in both speed and accuracy:

// add benchmarks here //

Acknowledgements

The PSE builds upon the groundwork laid by LLM Structured Output and utilizes lexpy for efficient lexicon analysis.

---

Made with care ❤️

Twitter • Website • Contact