sewsource 0.1.3

A CLI tool to sew github documentations into a compiled source for LLMs

🧵 SewSource

📖 Overview

SewSource is a powerful command-line interface (CLI) tool that helps you aggregate documentation files from GitHub repositories. It clones repositories, combines specified documentation files, and creates a unified source perfect for interactive Learning Language Model (LLM) discussions using tools like NotebookLM.

• 🚀 Installation
• 🛠️ Usage
  • Basic Command
  • Advanced Usage with Options
• 📋 Command Options
• 💡 Examples
  • 1. Basic Repository Aggregation
  • 2. Specific Documentation Directories
  • 3. Excluding Certain Content
• ⚠️ Common Issues and Solutions
• Next TODOs
• 🤝 Contributing
• 📄 License

🚀 Installation

pip install sewsource

🛠️ Usage

Basic Command

sewsource https://github.com/username/repository

Advanced Usage with Options

sewsource https://github.com/username/repository \
    --output-dir "./docs_combined" \
    --include-dirs "docs,wiki" \
    --exclude-dirs "tests" \
    --blacklist "README.md,CHANGELOG.md" \
    --extensions ".md,.rst,.txt"

📋 Command Options

Option	Short	Description	Default
--output-dir	-o	Output directory for combined files	~/.sewsource
--include-dirs	-i	Directories to include	All directories
--exclude-dirs	-x	Directories to exclude	None
--blacklist	-b	Files to exclude	None
--extensions	-e	File extensions to include	.md, .mdx
--version	-	Show version information	-
--help	-	Show help message	-

💡 Examples

1. Basic Repository Aggregation

sewsource https://github.com/tensorflow/tensorflow

2. Specific Documentation Directories

sewsource https://github.com/pytorch/pytorch \
    --include-dirs "docs" -i "tutorials" \
    --extensions ".md" -e ".rst"

3. Excluding Certain Content

sewsource https://github.com/kubernetes/kubernetes \
    --exclude-dirs "vendor" -x "test" \
    --blacklist "CONTRIBUTING.md"

⚠️ Common Issues and Solutions

• Large Repositories

  # For large repos, use specific directories
  sewsource https://github.com/large/repo --include-dirs "docs"
  

Next TODOs

• Add support for comma separated values as arguments for multiple folders/files
• Add is_all option to concatenate all sources into a single final source file

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

---