simple-vllm 0.0.1

VLLM & Unsloth Wrapper for simple local inference.

EasyVLLM - A very simple unsloth + vllm wrapper to allow for easy generation locally.

"Does what is says on the tin." - By a good friend.

Getting Started:

1. Installation:

pip3 install easy_vllm

2. Running:

from simple_vllm import get_vllm_generator

# Create the client.
client = get_vllm_generator(
    model_name="Qwen/Qwen2.5-3B-Instruct",
    max_seq_length = 4096,
    gpu_memory_utilization=0.5,
    load_in_4bit = True,
)

# Create a messages object in OpenAI format.
messages = [
    {
        "role" : "user",
        "content" : "Hello, could you tell me how to become a better human?",
    },
]

# Get the response as list of str
responses = client.generate(
    messages, 
    num_generations=1,
    max_completion_tokens=100,
    temperature=0.0,
    )

# print response.
print(responses[0]) #this will print the model output.

(C) - Nikolai Rozanov, 2025 - Present