vllm-plugin-meralion2 0.3.0

MERaLiON2 vLLM Plugin

Licence

MERaLiON-Public-Licence-v3

Set up Environment

This plugin family has three release lines:

• v0.1.x: compatibility lane for vLLM version 0.6.5 ~ 0.7.3 (V0 engine), and 0.8.5 ~ 0.8.5.post1 (V1 engine).
• v0.2.x: compatibility lane for vLLM >=0.8.5,<=0.10.0.
• v0.3.x: compatibility lane for vLLM >=0.12.0,<0.17.0. Targets the V1 engine and handles all internal API changes across the 0.12–0.16 minor series.

See scripts/compatibility/ for the automated version-matrix runner and detailed vLLM + transformers compatibility results.

Install by your vLLM version:

# For vLLM 0.6.5~0.7.3, 0.8.5.
pip install "vllm-plugin-meralion2<0.2"

# For vLLM 0.8.5 ~ 0.10.0
pip install "vllm-plugin-meralion2>=0.2,<0.3"

# For vLLM 0.12.0 ~ 0.16.x
pip install "vllm-plugin-meralion2>=0.3,<0.4"

Attention backend: MERaLiON-2 uses Gemma2's attention logit softcapping which requires the FlashInfer backend. The serve example handles this automatically. See openai_serve_example.sh for details.

Offline Inference

Refer to offline_example.py for offline inference example.

OpenAI-compatible Serving

Refer to openai_serve_example.sh for OpenAI-compatible serving example.

To call the server, you can refer to openai_client_example.py.

Alternatively, you can try calling the server with curl, refer to openai_client_curl.sh.

Full release history

See CHANGELOG.md.

vLLM + transformers compatibility (v0.3.x)

Tested with transformers==4.57.6 on H100 (TP=1). Each cell covers install, unit tests, and full-dataset ASR evaluation.

To reproduce or extend this matrix, see the compatibility matrix runner.

Security and dependency scanning

The repository uses separate workflows so each scan has a clear purpose:

• Security (Bandit SAST) (.github/workflows/security.yml): static security linting of project Python source (bandit -r src).
• CodeQL (.github/workflows/codeql.yml): semantic code scanning for Python + GitHub Actions security issues.
• Dependency Audit (pip-audit) (.github/workflows/dependency-audit.yml): installed dependency vulnerability scanning.
• Dependency Review (PR) (.github/workflows/dependency-review.yml): checks dependency changes in pull requests and fails on moderate+ severity vulnerabilities.