Development

vLLM

Run and serve large language models with request batching, streaming, distributed inference and an OpenAI-compatible API server.

Open vLLM vllm.ai

Workflow use

This profile describes the product. No current evidence-qualified workflow assignment is available in the visible catalogue.

Explore reviewed workflows

Overview

vLLM is infrastructure for LLM inference and serving. Use offline batches or expose a model through its compatible API server. It includes continuous request batching, streaming, distributed execution and structured-output generation. Installation depends on the hardware backend; the detailed quickstart documents a Linux baseline and Python 3.11–3.14.1 3 2

What you can do

Expose a model through an OpenAI-compatible API server.2
Batch incoming inference requests continuously.2
Stream model outputs to the client.2
Use tensor, pipeline, data, expert or context parallelism for distributed inference.2
Generate structured outputs from supported model configurations.2
Perform offline batched inference.3

Get started

Use the quickstart for your hardware backend; its NVIDIA example installs vLLM with uv and starts a selected model with vllm serve.3

Ways to use it

Ideas based on documented features.

Serve an LLM to an application using the compatible API protocol.2
Process a set of prompts as an offline batch.3

Before you start

Check the backend-specific installation guide. The detailed quickstart lists Linux and Python 3.11–3.14, rather than assuming every platform uses the same package.3
The quickstart server hosts one model at a time; plan serving instances accordingly.3