Skip to main content

SimpleVLLM - A very simple unsloth + vllm wrapper to allow for easy generation locally.

"Does what is says on the tin." - good friend.

Getting Started:

  1. Installation:
pip3 install simple_vllm

Note: this will take time, as you are install vllm and unsloth.

  1. Running:
from simple_vllm import get_vllm_generator

# Create the client.
llm = get_vllm_generator(model_name="Qwen/Qwen2.5-3B-Instruct")

# Get the response as list of str
responses = llm("Hello, could you tell me how to become a better human")

print(responses[0])

More Advanced examples and parameters:

from simple_vllm import get_vllm_generator

# Create the client.
llm = get_vllm_generator(
    model_name="Qwen/Qwen2.5-3B-Instruct",
    max_seq_length = 4096,
    gpu_memory_utilization=0.5,
    load_in_4bit = True,
)

# Create a messages object in OpenAI format.
messages = [
    {
        "role" : "user",
        "content" : "Hello, could you tell me how to become a better human?",
    },
]

# Get the response as list of str
responses = llm(
    messages, 
    num_generations=1,
    max_completion_tokens=100,
    temperature=0.0,
    # other params are also possible have a look inside: simple_vllm/llm.py
    )

# print response.
print(responses[0])

# generate again just using a string input this time, without having to reload the model.
responses = llm("tell me something about life.", max_completion_tokens=10)

print(responses[0])

(C) - Nikolai Rozanov, 2025 - Present

Metadata

Release files for simple-vllm 0.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for simple-vllm 0.0.4
File Size Uploaded Provenance Actions
simple_vllm-0.0.4.tar.gz 41.6 kB Without provenance Details for simple_vllm-0.0.4.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for simple-vllm 0.0.4
File Interpreter ABI Platform Actions
simple_vllm-0.0.4-py3-none-any.whl Python 3 none any Details for simple_vllm-0.0.4-py3-none-any.whl

Total release size: 70.5 kB

Release files / simple_vllm-0.0.4.tar.gz

Download URL simple_vllm-0.0.4.tar.gz
Size 41.6 kB
Tags Source
SHA-256 checksum
How to use checksums
f4d08446befee769d3ca208ce5949dc1282328565da40994164c68b05c5b4554
BLAKE2b-256 checksum
How to use checksums
8f9e67085998fceba0738846117a0ae1764f5ae1ac6856f0e0455d247384e8b3
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.11.11

Release files / simple_vllm-0.0.4-py3-none-any.whl

Download URL simple_vllm-0.0.4-py3-none-any.whl
Size 28.9 kB
Tags Python 3
SHA-256 checksum
How to use checksums
51b259062522df699a61e06c7fe0ae69ddc15de8a28bc1b1ba4179056af6e772
BLAKE2b-256 checksum
How to use checksums
c75ccdfd80f68416e4e8adbf3e03809fd4b02aa0012e28d527a6a2f7b669e934
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/6.1.0 CPython/3.11.11

Security for simple-vllm 0.0.4

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page