Yoink

Yoink is a command-line tool and FastAPI backend designed to bundle, sanitize, and visualize codebases for Large Language Model (LLM) contexts. It converts raw project directories into structured Markdown documents, optimizing token consumption and preventing compliance and security leaks.

Features

• Token Shredder: Safely strips comments, docstrings, and excess whitespace to reduce prompt size.
• Secret Shield: Programmatically redacts credentials, private keys, emails, and IP addresses.
• Compliance Stripper & Masking: Redacts proprietary terms and internal domains using deterministic, token-efficient phonetic codenames (e.g. Delta7) to keep code syntax compilable. Features an interactive terminal UI (TUI) to configure and live-test rules in a sandbox.
• Dependency Trees: Automatically generates AST-based ASCII dependency trees and Mermaid flowcharts.

Installation

Install from PyPI:

pip install yoinky

Or install locally in editable mode:

git clone https://github.com/Asifdotexe/yoink.git
cd yoink
pip install -e .

Quick Start

1. Pack a Codebase

Pack the current directory:

yoink

2. Configure Censor Lists (CLI / TUI)

Manage word and domain censorship with deterministic pseudonym generation (e.g., Delta7) and configuration persistence via .yoinkconfig.json:

• Initialize rules (like poetry init):

  yoink censor init
  

• Inspect active codename mappings (e.g., Theseus -> Delta7):

  yoink censor show
  

• Launch interactive dashboard & sandbox:

  yoink censor tui
  

Documentation

For detailed instructions and references, please refer to the dedicated guides:

• CLI Usage Guide: Complete reference of CLI commands, flags (--raw, --max-size, --no-visualize, --censor, --censor-tui, etc.), and usage examples.
• REST API Guide: Detailed guide to API endpoints, schema structures, and Render blueprint web service deployment.
• Token Reduction Learnings: Technical reference on how BPE tokenizers handle whitespace, license header stripping, and performance tuning.

Running Tests

Verify your local installation:

pytest

License

This project is licensed under the GNU AGPLv3 License.