LLM agent with direct Vim access using neovim/pynvim
Project description
VimGPT
Experimental LLM agent/toolkit with direct Vim access, using neovim/pynvim. Intended for use either as a sub-agent for efficient file editing, or as its own agent runtime.
Here's a lil demo video. Built with:
- LiteLLM for LLM client interop
- PromptLayer for observability
- Helicone for caching
Why?
There are a few common options for using LLM agents to edit files:
- Rewrite the entire file: reliable, but very expensive (in time and tokens), and for small changes/big files it is incredibly wasteful and difficult to interpret where the change was made. Also, what if you want to make a 1-line change to a million-line file that won't fit in the context window?
- Make a patch: using a patch format such as UDF can make edits much faster and more clear, but is highly error-prone as the current generation of LLMs often make mistakes with line numbers, leading spaces, capitalization, etc.
- Roll your own editor: this can work pretty well, but 1) the models don't have training data on your custom format, and 2) now you're maintaining a custom format.
The Solution: Instead of asking the agent to call a tool with the new contents, give the agent a "Editor" tool which drops into a sub-agent with appropriate context. The simulated editor shows the state of the editor in plaintext, takes individual keystrokes, and reflects updates. And why invent your own editor when LLMs already know about Vim?
Drawbacks / open issues
:warning: WARNING: VimGPT is still experimental and not yet ready for production.
- VimGPT's performance varies considerably depending on your chosen agent runtime - your mileage may vary.
- VimGPT tends to be quite fast, but does consume a lot of requests/tokens because each command is its own request. There is an option for allowing multiple commands per request, but this tends to degrade agent performance.
- So far, only
gpt-4
seems capable of using this properly - more research tbd. - even GPT-4 can sometimes get confused and submit files that are clearly wrong sometimes, and sometimes can enter endless loops of using the wrong commands - unclear if just a prompt eng issue or whether the models are not quite good enough at Vim.
- some misc bugs, e.g. trouble with translating escape characters
Installation
You'll need to install neovim, if you're on MacOS you can simply brew install neovim
. Then install from PyPI:
pip install vimgpt # or poetry add vimgpt
Usage
As a CLI / Demo
CLI usage (intended to demo/test):
$ vimgpt --help
usage: vimgpt [-h] [--path PATH] [--output [OUTPUT]] [--socket SOCKET] [--loglevel {DEBUG,INFO,WARNING}] [--max-calls MAX_CALLS] [--delay-seconds DELAY_SECONDS] command
VimGPT CLI
positional arguments:
command Task for VimGPT to perform on the file, in natural language:
'Rename Bob to Bill in paragraph 2`,
'make arg `user` optional on line 34',
'rewrite this in iambic pentameter', etc.
options:
-h, --help show this help message and exit
--path PATH, -p PATH File for VimGPT to open.
NOTE: for safety, VimGPT will NOT make changes directly
to the file unless the --output flag is provided.
--output [OUTPUT], -o [OUTPUT]
Specify output file. If flag is not provided,
VimGPT will NOT make changes directly to the file.
If flag is provided without value, uses the same path as the input file.
If flag is provided with no value and there is no input file specified,
will output to 'vimgpt_output.txt'.
--socket SOCKET, -s SOCKET
Path to nvim socket of running nvim process.
If left empty, VimGPT will run in headless mode.
Suggested value: '/tmp/nvimsocket'.
--loglevel {DEBUG,INFO,WARNING}
Set the logging level (default: WARNING)
--max-calls MAX_CALLS
Maximum number of LLM calls. Default is 1000.
--delay-seconds DELAY_SECONDS
Delay in seconds. If not provided, defaults to None.
To run with headless vim:
vimgpt "Edit the contents of the README file to recommend Vim as the best text editor." --path tests/samples/README.md --loglevel INFO
To attach to a running Neovim instance so you can view what's happening in real-time as the agent does:
NVIM_LISTEN_ADDRESS=/tmp/nvimsocket nvim
# in separate terminal
vimgpt "Edit the contents of the README file to recommend Vim as the best text editor." --path tests/samples/README.md --loglevel INFO --socket '/tmp/nvimsocket'
As a library
VimGPT is intended to be used as a subagent/tool so it can be composed within larger agents, so you can just call vim_gpt_agent()
directly with a specific goal.
from vimgpt import vimgpt_agent
original_file_content: str = '???'
new_file_content: str = vimgpt_agent(
command='Edit to recommend vim over emacs.',
content=original_file_content,
file_path='does_not_matter_just_useful_for_prompt.md',
# socket=None, # or '/path/to/running/nvim'
# max_calls=1000, # can use prevent cycles
# delay_seconds=2, # if you want to follow in real-time more easily
)
Using with LangChain/other agents
Look at vimgpt/core.py
for inspiration. For LangChain examples, check out ./vimgpt/langchain_agents/
for alternative runtimes and examples of how to use VimGPT as a tool.
Development
This project uses poetry.
# If you don't already have 3.11 installed
pyenv install 3.11:latest
# typical poetry install methods didn't work for me
brew install poetry
# this will create a .venv in the project directory.
# it should automatically check the right version is installed.
poetry install
# view info about your new venv
poetry env list
Publish to PyPI
To update the package on PyPI (https://pypi.org/project/vimgpt):
poetry build
poetry config pypi-token.pypi YOUR_PYPI_TOKEN
poetry publish
Linting, Formatting, Typechecking, Testing
Use ruff + black + mypy + pytest. If you're using VSCode, you might want to follow these instructions to make sure the mypy extension settings are set to use your active interpreter (the poetry venv).
black .
ruff check . --fix
mypy vimgpt
pytest -vv -s --verbose -o log_cli=true
Roadmap
- Improvements
- make a full-fledged agent using builtin LSP/treesitter
- Async/await
- Simplify prompt engineering so can work with weaker models than gpt-4 (gpt-3.5, claude-2)
- Experiments
- Use relative line numbers (
set relativenumber
) - cursor representation (suggested by GPT-4) likely causes some problems/could be optimized. Tried using an inserted
_
or|
but this didn't really work.
- Use relative line numbers (
- Open issues
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.