A tool to efficiently extract and compress Git repository contents for LLMs.
Project description
Siphon
Efficiently extract, compress, and cache Git repository contexts for seamless integration with Large Language Models (LLMs).
Table of Contents
Features
- Efficient Extraction: Extracts and compresses repository contents while respecting
.gitignore
rules. - Customizable Filtering: Include or exclude files and directories with ease.
- Multiple Output Formats: Supports text, tarball, and markdown formats optimized for LLM contexts.
- Caching and Chunking: Pre-cache large repositories for faster querying.
- Token Count Estimations: Get token counts for specific LLMs like GPT-3 and Claude.
- Clipboard and Stdout Support: Streamline workflows with seamless copying options.
- Modularity: Extend functionality with community-driven extensions.
- Interactive Mode: Granular file selection through an interactive interface.
Installation
Install Siphon using pip
:
pip install siphon-cli
Usage
Navigate to your Git repository and run:
si -o context.txt
This command extracts the repository content into context.txt
.
Examples
-
Include Specific File Types:
si -i "*.py" -o python_files.txt
-
Exclude Directories:
si -e "tests/*" -o code_without_tests.txt
-
Interactive Mode:
si --interactive -o selected_files.txt
-
Copy Output to Clipboard:
si --clipboard
Arguments
path
: Path to the Git repository (default: current directory).-i
,--include
: Include file patterns (e.g.,.py
,src/
).-e
,--exclude
: Exclude file patterns (e.g.,tests/
,*.md
).-o
,--output
: Output file name (default:output.txt
).-f
,--format
: Output format (text
,tar
,markdown
).-c
,--cache
: Enable caching (future feature placeholder).--tokenizer
: Tokenizer for token count estimation (gpt3
,claude
).--interactive
: Interactive mode for file selection.--clipboard
: Copy output to clipboard.--stdout
: Print output to stdout.
Contributing
We welcome contributions from the community! To contribute:
-
Fork the repository.
-
Create a new branch:
git checkout -b feature/your-feature-name
-
Commit your changes:
git commit -am 'Add a new feature'
-
Push to the branch:
git push origin feature/your-feature-name
-
Open a Pull Request.
Please read our Contributing Guidelines for more details.
License
This project is licensed under the MIT License.
Contact
- Email: morganj2k@gmail.com
- GitHub: atxtechbro
- Project Link: https://github.com/atxtechbro/siphon
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file siphon_cli-1.3.1.tar.gz
.
File metadata
- Download URL: siphon_cli-1.3.1.tar.gz
- Upload date:
- Size: 5.1 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 38f851a6e36137b97b97071fe78bde03b99dcb317775101c3ae77417517ebfad |
|
MD5 | fd4db150c0359872b4cff119ba984a20 |
|
BLAKE2b-256 | e7856ea5649f57b4f98c9fc662c11c9c89d3a610d21a6edbb7b9db067f322a1f |
File details
Details for the file siphon_cli-1.3.1-py3-none-any.whl
.
File metadata
- Download URL: siphon_cli-1.3.1-py3-none-any.whl
- Upload date:
- Size: 5.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/5.1.1 CPython/3.12.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | faba8397ce893dbbf8f212e3f3afc8c8d55844bd4a52b5ba728d259eb100fa9f |
|
MD5 | 56593d5421355a42a7112e89917be609 |
|
BLAKE2b-256 | 8a0982d6bfd480fed26baa6843b89fbcf6a18cb86cfaee913c8c0d500e89cf4f |