---

✨ Features

• LLM-ready statistics: every pack ends with the total line count and an estimated token count (accurate cl100k_base counting when tiktoken is installed, ~4 chars/token heuristic otherwise), printed in the output header and in the CLI summary.
• Three output formats: classic plain text (--- START OF FILE: ... ---), Markdown (--format markdown) with language-aware fenced code blocks, and JSON (--format json) with per-file metadata (path, lines, tokens, content) for scripts and API processing.
• --include filter: pack ONLY files matching a pattern (pyai --include 'src/**', --include '*.py'); directories that cannot contain matching files are pruned from the tree, and it composes with --exclude.
• LLM budget control (--budget): warn when the pack exceeds an LLM context limit (e.g. --budget 128k) and name the heaviest files to drop first; per-file line/token statistics are always available.
• --dry-run preview: see exactly what would be packed (files sorted by token weight, totals, budget verdict) without writing anything.
• --strip-docs: optionally strip comments and docstrings from Python files (AST-based, syntax-error files pass through untouched) to shrink the pack.
• File-type summary: every pack header ends with a compact per-extension breakdown (e.g. File types: .py x12 (3,456 lines), .md x2 (120 lines)); disable with --no-type-summary.
• Smart Filtering: automatically ignores VCS folders (.git, .github), virtual environments (.venv, venv), IDE settings (.vscode, .idea), build artifacts (dist, build, *.egg-info), caches (__pycache__, .pytest_cache), binary files (images, archives, databases, executables — plus a NUL-byte content heuristic) and hidden files (except explicitly allowed configs like .gitignore, .env.example, .editorconfig, .dockerignore).
• .gitignore / .pyaiignore support: honored automatically when the optional pathspec dependency is installed (pip install py-for-ai[gitignore]).
• Custom exclusions: additional glob patterns via --exclude (repeatable) and a per-file size cap via --max-file-size.
• Output control: --quiet/-q for CI-friendly silent runs (errors still go to stderr), --verbose/-v for extra details, --no-tree to drop the directory tree section, and --no-token-count to skip token estimation on large projects.
• Symlink-Safe: symlinks pointing outside the project root are never followed or packed; directory symlinks are never traversed, so cycles and aliased duplicates are impossible; file-symlink aliases of already-packed files are packed only once. Suspicious links (outside-root, dangling, aliased) stay visible in the directory tree with an explanatory note.
• Deep-Project-Proof: iterative (stack-based) directory traversal — no RecursionError on very deeply nested projects.
• Encoding-Aware: reads UTF-8, UTF-8-BOM, UTF-16/32 (via BOM), legacy Cyrillic cp1251 and other 8-bit encodings automatically; line endings are normalized to LF (CRLF/CR → LF), so packs are byte-consistent across platforms; true binary files (NUL-byte heuristic over the whole file) are skipped with a clear warning and marked in the tree.
• ASCII Directory Tree: a clean, sorted representation of your project layout, consistent with the packed content (skipped files are annotated).
• Clipboard Integration: automatically copies the packed content; gracefully falls back with a warning in headless/SSH environments.

---

⚠️ Known limitations (by design)

• UTF-16/32 without a BOM contains NUL bytes and is skipped as binary — such files cannot be reliably told apart from real binary data.
• The final latin-1 fallback never fails, so an exotic legacy encoding may be transcoded with mojibake instead of being skipped.
• Directories named env, venv, build, dist are ignored at any depth (common Python/.gitignore convention). A legitimate data folder with one of these names will not be packed.
• In the classic text format, a source file that itself contains --- START OF FILE: ... --- lines can confuse naive parsers of the pack. The Markdown format is immune thanks to adaptive fences.
• Token estimation with the ~4 chars/token heuristic is approximate; install py-for-ai[tokens] for exact cl100k_base counting (both computed over the final document).

📂 Project Layout

py_ai_project/
├── pyproject.toml
├── README.md
├── CHANGELOG.md
├── .github/workflows/tests.yml   # CI: pytest + ruff on 3 OS × 6 Python versions
├── verify_all.py                 # cross-platform pre-release verification
├── tests/                        # pytest suite
└── src/py_ai/
    ├── __init__.py               # version (read from installed metadata)
    ├── __main__.py               # python -m py_ai
    ├── cli.py                    # argument parsing & user interaction
    ├── core.py                   # traversal orchestration + tree builder
    ├── filters.py                # ignore rules, --exclude, .gitignore support
    ├── readers.py                # encoding/binary-aware file reading
    ├── stripping.py              # --strip-docs (AST comment/docstring removal)
    ├── tokens.py                 # line & token statistics
    ├── formatting.py             # text / markdown output assembly
    └── py.typed                  # PEP 561 type marker

---

🚀 Quick Start

Installation

Install the utility directly from PyPI (note: the distribution name is py-for-ai):

pip install py-for-ai

Optional extras:

pip install "py-for-ai[tokens]"       # accurate token counting via tiktoken
pip install "py-for-ai[gitignore]"    # respect .gitignore / .pyaiignore files
pip install "py-for-ai[all]"          # everything above

(Or install it locally for development):

git clone https://github.com/Maksum867/py-ai.git
cd py-ai
pip install -e ".[dev]"
pytest

Full pre-release verification (one command)

Run everything — tests, lint, CLI smoke checks, git hygiene, build, wheel smoke — from the repo root:

python verify_all.py                 # full check (tests, lint, CLI smoke, build, wheel smoke)
python verify_all.py --skip-build    # quick check (no build)

The script exits 0 when every check passes and 1 when anything fails.

Usage

Run the utility from any terminal session — both pyai and the py-ai alias are available:

# 1. Pack current directory and copy to clipboard:
pyai

# 2. Pack a specific directory:
pyai /path/to/your/project

# 3. Save the results to a custom file:
pyai . -o custom_output.txt

# 4. Run without clipboard copy (ideal for remote servers or CI/CD):
pyai --no-clipboard

# 5. Markdown output with syntax-aware code fences:
pyai --format markdown -o context.md

# 6. Skip large files and extra paths:
pyai --max-file-size 200KB --exclude '*.log' --exclude 'docs/*'

# 7. Same via the alias or as a module; show the installed version:
py-ai --no-clipboard
python -m py_ai --version

# 8. CI-friendly: no informational output, no clipboard, no tree:
pyai --quiet --no-clipboard --no-tree -o context.txt

# 9. Skip token estimation (faster on big repos) or show extra details:
pyai --no-token-count
pyai --verbose

# 10. Pack ONLY the source tree (include filter), preview first:
pyai --dry-run --include 'src/**'
pyai --include 'src/**' --include '*.md' -o context.txt

# 11. Check the pack against an LLM context budget:
pyai --dry-run --budget 128k
pyai --budget 128k -o context.txt

# 12. Machine-readable output for scripts / per-file API calls:
pyai --format json -o context.json

# 13. Shrink the pack by stripping Python comments and docstrings:
pyai --strip-docs -o lean_context.txt

---

📄 Output Format Examples

Plain text (default)

================================================================================
PROJECT CONTEXT PACK: my_project
Generated on: 2026-07-31 12:00:00
Total files packed: 2
Total lines: 33
Estimated tokens: ~410 (heuristic (~4 chars/token))
================================================================================

================================================================================
DIRECTORY TREE
================================================================================
my_project/
├── src/
│   └── main.py
└── pyproject.toml

================================================================================
FILES CONTENT
================================================================================

--- START OF FILE: pyproject.toml ---
[project]
name = "my_project"
version = "0.1.0"
--- END OF FILE: pyproject.toml ---

--- START OF FILE: src/main.py ---
def main():
    print("Hello, AI!")
--- END OF FILE: src/main.py ---

Markdown (--format markdown)

# Project Context Pack
- Project: my_project
- Generated on: 2026-07-31 12:00:00
- Total files packed: 1
- Total lines: 15
- Estimated tokens: ~120 (cl100k_base (tiktoken))

## Directory Tree

```text
my_project/
└── src/
    └── main.py
```

## Files Content

### `src/main.py`
```python
def main():
    print("Hello, AI!")
```

Files that exist in the project but were not packed (binaries, oversized files, unsafe symlinks) remain visible in the tree with an explanatory note, e.g.:

├── data/
│   ├── dump.bin  [skipped: binary file]
│   ├── huge.log  [skipped: exceeds size limit (200.0 KB)]
│   └── notes.txt -> /etc/hostname  [symlink outside project root — not followed, not packed]

---

⚙️ Filtering Behavior (what exactly is excluded)

• Directories/files by name: .git, .github, .gitlab, .svn, .hg, node_modules, __pycache__, .venv, venv, env, .env, .pytest_cache, .mypy_cache, .ruff_cache, .tox, .nox, build, dist, .idea, .vscode, .settings, .DS_Store, Thumbs.db, desktop.ini (matched case-insensitively), any path component ending in .egg-info.
• Binary extensions: compiled artifacts, archives, images, audio/video, fonts, databases, office documents, ML artifacts (.pkl, .npy, .onnx, …) and executables (.exe, .msi, .bin, .dll, .so, …). Files of any other type that contain NUL bytes are treated as binary too and skipped with a warning.
• Hidden files (starting with .) are ignored, except: .gitignore, .gitattributes, .gitmodules, .env.example, .env.template, .pylintrc, .flake8, .coveragerc, .dockerignore, .editorconfig, .pre-commit-config.yaml, .python-version, .readthedocs.yaml, .readthedocs.yml, .codecov.yml.
• .gitignore / .pyaiignore rules (when py-for-ai[gitignore] is installed; disable with --no-gitignore).
• User patterns from --exclude (matched against the relative POSIX path and the file name). Git-style directory patterns work too: --exclude 'build/' excludes the build/ directory and everything under it. Note: patterns use Python's fnmatch, where * also matches across / (so docs/* excludes docs/deep/file.py as well).
• Oversized files when --max-file-size is given.
• Symlinks resolving outside the project root and directory symlinks are never followed or packed (shown in the tree with a note).
• The output file itself is never included in its own pack.

---

🪙 Token Estimation

• With pip install py-for-ai[tokens], tokens are counted precisely with tiktoken using the cl100k_base encoding (used by GPT-3.5/4 families and a reasonable approximation for other models).
• Without it, a widely used heuristic of ~4 characters per token is applied, and the CLI tells you which method was used.

---

🛠️ Requirements & Dependencies

• Python 3.8 or higher (verified in CI on 3.8–3.14, Linux/Windows/macOS).
• pyperclip (for clipboard operations; requires xclip/xsel on X11 or wl-clipboard on Wayland for Linux desktops — otherwise a warning is shown and the output file is still produced).

---

🤝 Contributing

git clone https://github.com/Maksum867/py-ai.git
cd py-ai
pip install -e ".[dev]"
pytest

Please make sure the whole pytest suite passes and add tests for new features.