local-llm-cli 0.1.3

Converse with GPT4 LLM locally

local_llm_cli

local_llm_cli is a Python package that allows you to converse with a GPT4All Language Model (LLM) locally. This can be useful for testing, developing, and debugging.

Currently, this library supports interacting with the GPT4All model. However, support for other models and additional functionalities are planned for future updates.

Installation

To install local_llm_cli, you can use pip:

pip install local_llm_cli

You'll also need to ensure that you have the necessary model files available locally.

Usage

The converse sublibrary provides a function to load a GPT4All LLM and converse with it.

Here's a simple usage example:

from local_llm_cli.converse.chat import load_and_interact

# define the model path
model_path = 'path/to/your/model'

# call the function to start conversing with the LLM
load_and_interact(model_path)

In this example, the model_path should be the path to the GPT4All model files on your local system.

The load_and_interact function also accepts optional arguments to specify the model context (model_n_ctx) and batch size (model_n_batch). If these arguments are not provided, they default to 1024 and 8, respectively.

Here's an example with custom context and batch size:

load_and_interact(model_path, model_n_ctx=2048, model_n_batch=16)

You can stop the conversation at any time by typing exit.

License

This project is licensed under the MIT License. See the LICENSE file for more details.

This package was crafted with ❤️ by Harsh Avinash in approximately 22 minutes. Enjoy conversing with your local LLM!