vllm-openaivLLM
High-throughput LLM serving with PagedAttention, continuous batching and FP8/AWQ quantisation.
On MZGrid: Powers every LLM template, including llama-3.1-8b, llama-3.3-70b, qwen2.5-32b and the generic vllm-openai template.
Get started