Clip Files 📋📁

Introducing clip-files - a simple yet useful command-line utility to gather files with a specific extension, format them with their full path, and copy their contents to your clipboard with a token count for GPT-4 usage! Makes it simple to paste in your entire project into e.g., ChatGPT or Claude.ai, now that massive token limits are available.

:star2: Features

• 📋 Collect files with a specific extension and copy their formatted contents to your clipboard.
• 📁 Surrounds each file with <file path="...">...</file> for easy parsing by LLMs.
• 🧮 Calculate and display the total number of tokens used, compatible with GPT-4 tokenization.
• 🔄 Easily handle and process multiple files in various directories.

:books: Usage

To use clip-files, simply provide the directory to search and the file extension you want to collect:

clip-files --help

Shows the help message:

usage: clip-files [-h] [-i INITIAL_FILE] [-f FILES [FILES ...]] [-m MAXDEPTH]
                  [folder] [extension ...]

Collect files with specific extensions or specific files, format them for clipboard, and count tokens.
There are two main ways to use clip-files:
1. Collecting all files with specific extensions in a folder:
   `clip-files FOLDER EXTENSION [EXTENSIONS ...]`
   Examples:
   - `clip-files . .py`  # all Python files in current directory
   - `clip-files . .py .md .txt`  # all Python, Markdown, and text files in current directory
   - `clip-files src .txt`  # all text files in src directory
   - `clip-files docs .md --initial-file instructions.txt`  # with custom instructions
2. Collecting specific files (can be of different types):
   `clip-files --files FILE [FILE ...]`
   Examples:
   - `clip-files --files src/*.py tests/*.py`  # using shell wildcards
   - `clip-files --files src/main.py docs/README.md`  # different file types
   - `clip-files -f src/*.py -i instructions.txt`  # with custom instructions

Options:
  -f, --files        Specify individual files to include
  -i, --initial-file A file containing initial instructions

Note: When using wildcards (e.g., *.py), your shell will expand them before passing to clip-files.

positional arguments:
  folder                The folder to search for files.
  extension             The file extensions to look for (e.g., .py, .txt). If
                        not provided, includes all non-hidden, non-binary
                        files.

options:
  -h, --help            show this help message and exit
  -i, --initial-file INITIAL_FILE
                        A file containing initial instructions to prepend to
                        the clipboard content. Default is an empty string.
  -f, --files FILES [FILES ...]
                        Specific file paths to include (e.g., --files
                        path/to/file1.py path/to/file2.md). If not provided,
                        all files with the specified extensions are included.
  -m, --maxdepth MAXDEPTH
                        Maximum directory depth to traverse (default: no
                        limit)

clip-files will traverse the specified folder, gather files with the desired extension, format them with their paths, and copy the results to your clipboard along with the token count.

:bulb: Examples

[!TIP] Pro-tip! Just use uv to run clip-files without installing it. For example, uv run clip-files --files *.py will run clip-files in an temporary environment.

There are two main ways to use clip-files:

1. Collecting all files with a specific extension in a folder:

# Collect all Python files in the current directory
clip-files . .py

# Collect all text files in the documents folder
clip-files documents .txt

# Collect all Python files, including custom initial instructions
clip-files src .py --initial-file instructions.txt

2. Collecting specific files (can be of different types):

# Collect specific files
clip-files --files src/main.py tests/test_app.py

# Using shell wildcards to select files
clip-files --files src/*.py tests/*.py

# Mix different file types with wildcards
clip-files --files src/*.py docs/*.md config/*.json

# With custom initial instructions
clip-files --initial-file instructions.txt --files src/*.py

Note: When using wildcards (e.g., *.py), your shell will expand them before passing to clip-files.

:hammer_and_wrench: Installation

To install clip-files, use pip:

pip install clip-files

Alternatively, clone the repository:

git clone https://github.com/basnijholt/clip-files.git
cd clip-files
pip install .

Or download the script directly:

wget https://raw.githubusercontent.com/basnijholt/clip-files/main/clip_files.py

and run it using:

python clip_files.py --help

:file_folder: Supported File Types

clip-files supports any file type as long as you provide the correct file extension. Common examples include:

• .py for Python files
• .txt for text files
• .md for Markdown files

:1234: Token Counting

clip-files integrates with the tiktoken library to calculate tokens as they would be counted in GPT-4. This helps in understanding the cost and feasibility of processing the collected text with GPT-4 models.

The script will display the total token count after copying the formatted content to the clipboard.

:heart: Support and Contributions

We welcome feedback and contributions! If you encounter any issues or have suggestions for improvements, please file an issue on our GitHub repository. Contributions via pull requests are also appreciated.

Happy collecting and clipping! 📋📁🎉