gguf-connector 0.3.4

GGUF connector(s) with GUI

GGUF Connector

GGUF (GPT-Generated Unified Format) is a successor of GGML (GPT-Generated Model Language), it was released on August 21, 2023; by the way, GPT stands for Generative Pre-trained Transformer.

This package is a simple graphical user interface (GUI) application that uses the ctransformers or llama.cpp to interact with a chat model for generating responses.

Install the connector via pip (once only):

pip install gguf-connector

Update the connector (if previous version installed) by:

pip install gguf-connector --upgrade

With this version, you can interact straight with the GGUF file(s) available in the same directory by a simple command.

GUI

Select model(s) with ctransformers connector:

gguf-c

Select model(s) with llama.cpp connector:

gguf-cpp

CLI

Select model(s) with ctransformers connector:

gguf-cx

Select model(s) with llama.cpp connector:

gguf-cpx

You can even choose which connector (either ctransformers or llama.cpp) to be used at the very beginning.

Include/import the connector selection menu to the code by:

from gguf_connector import __main__

For standalone version please refer to the repository in the reference list (below).

Reference

model selector

Resources

ctransformers llama.cpp