purgo-ingest 0.1.4

CLI tool to analyze and create text dumps of codebases for LLMs

Gitingest

Turn any Git repository into a prompt-friendly text ingest for LLMs.

You can also replace hub with ingest in any GitHub URL to access the corresponding digest.

gitingest.com · Chrome Extension · Firefox Add-on

🚀 Features

• Easy code context: Get a text digest from a Git repository URL or a directory
• Smart Formatting: Optimized output format for LLM prompts
• Statistics about:
  • File and directory structure
  • Size of the extract
  • Token count
• CLI tool: Run it as a shell command
• Python package: Import it in your code

📚 Requirements

• Python 3.7+

📦 Installation

Purgo Ingest is available on PyPI. You can install it using pip:

pip install purgo_ingest

However, it might be a good idea to use pipx to install it. You can install pipx using your preferred package manager.

brew install pipx
apt install pipx
scoop install pipx
...

If you are using pipx for the first time, run:

pipx ensurepath

# install purgo_ingest
pipx install purgo_ingest

🧩 Browser Extension Usage

The extension is open source at lcandy2/gitingest-extension.

Issues and feature requests are welcome to the repo.

💡 Command line usage

The purgo_ingest command line tool allows you to analyze codebases and create a text dump of their contents.

# Basic usage
purgo_ingest /path/to/directory

# From URL
purgo_ingest https://github.com/cyclotruc/gitingest

# See more options
purgo_ingest --help

This will write the digest in a text file (default digest.txt) in your current working directory.

🐍 Python package usage

# Synchronous usage
from purgo_ingest import ingest

summary, tree, content = ingest("path/to/directory")

# or from URL
summary, tree, content = ingest("https://github.com/cyclotruc/gitingest")

By default, this won't write a file but can be enabled with the output argument.

# Asynchronous usage
from purgo_ingest import ingest_async
import asyncio

result = asyncio.run(ingest_async("path/to/directory"))

Structured Output

If you need to ingest a repository and want to access each file separately rather than combined in a single prompt, use the structured output entry points:

# Synchronous structured usage
from purgo_ingest import ingest_structured

# Returns a dictionary with metadata and individual file contents
result = ingest_structured("path/to/directory")

# Access metadata
repo_name = result["metadata"]["repository"]

# Access individual files
for file in result["files"]:
    file_path = file["path"]
    file_content = file["content"]
    file_size = file["size"]
    # Process each file separately

# Asynchronous structured usage
from purgo_ingest import ingest_structured_async
import asyncio

# Returns a dictionary with metadata and individual file contents
result = asyncio.run(ingest_structured_async("path/to/directory"))

The structured output is also available in the CLI:

# Generate structured JSON output
purgo_ingest /path/to/directory --structured

# Output to a specific file
purgo_ingest /path/to/directory --structured -o output.json

Jupyter notebook usage

from purgo_ingest import ingest_async

# Use await directly in Jupyter
summary, tree, content = await ingest_async("path/to/directory")

This is because Jupyter notebooks are asynchronous by default.

🐳 Self-host

1. Build the image:

   docker build -t gitingest .
   

2. Run the container:

   docker run -d --name gitingest -p 8000:8000 gitingest
   

The application will be available at http://localhost:8000.

If you are hosting it on a domain, you can specify the allowed hostnames via env variable ALLOWED_HOSTS.

# Default: "gitingest.com, *.gitingest.com, localhost, 127.0.0.1".
ALLOWED_HOSTS="example.com, localhost, 127.0.0.1"

🤝 Contributing

Non-technical ways to contribute

• Create an Issue: If you find a bug or have an idea for a new feature, please create an issue on GitHub. This will help us track and prioritize your request.
• Spread the Word: If you like Gitingest, please share it with your friends, colleagues, and on social media. This will help us grow the community and make Gitingest even better.
• Use Gitingest: The best feedback comes from real-world usage! If you encounter any issues or have ideas for improvement, please let us know by creating an issue on GitHub or by reaching out to us on Discord.

Technical ways to contribute

Gitingest aims to be friendly for first time contributors, with a simple Python and HTML codebase. If you need any help while working with the code, reach out to us on Discord. For detailed instructions on how to make a pull request, see CONTRIBUTING.md.

🛠️ Stack

• Tailwind CSS - Frontend
• FastAPI - Backend framework
• Jinja2 - HTML templating
• tiktoken - Token estimation
• posthog - Amazing analytics

Looking for a JavaScript/FileSystemNode package?

Check out the NPM alternative 📦 Repomix: https://github.com/yamadashy/repomix

🚀 Project Growth