Native Windows vLLM for AMD ROCm

A focused Windows runtime and build setup for running vLLM on an AMD Radeon RX 9070 XT with native ROCm/HIP. The project uses the OpenAI-compatible vLLM server and does not require WSL, Docker, Linux, or a virtual machine.

The repository contains only the project-specific setup code, Windows compatibility patches, dependency pins, and smoke tests. vLLM and the Windows ROCm plugin are downloaded at fixed commits during setup instead of being copied into this repository.

Supported configuration

Component	Version or value
Operating system	Windows 11 x64
GPU	AMD Radeon RX 9070 XT
GPU architecture	gfx1201
ROCm/HIP	7.2.1 for Windows
PyTorch	2.9.1+rocm7.2.1
Python	3.12 x64
vLLM source	commit b1388b1fbf5aaef47937fabe98931211684666a6
Windows ROCm plugin	commit 7c16fdd7244217e2e45e2c5118b49dd942f8487f
GPU count	One

The scripts intentionally stop when the detected GPU is not gfx1201 or when more than one GPU is visible. This keeps the tested configuration explicit and avoids architecture overrides.

Prerequisites

Install the following before running the setup script:

• Git
• Python 3.12 x64
• AMD graphics driver with native HIP support
• Visual Studio Build Tools 2022 with the C++ workload and Windows SDK if the optional native extension will be built
• An internet connection for the pinned source checkouts and Python packages

The standard vLLM/Triton path does not require Visual Studio. It is needed only for -BuildNativeC.

Installation

Open PowerShell in the repository root and run:

.\setup.ps1

The setup process:

1. Creates .venv.
2. Installs the self-contained Windows ROCm and PyTorch wheels.
3. Downloads the pinned vLLM and Windows ROCm plugin checkouts into the ignored vendor directory.
4. Applies the project patches.
5. Installs vLLM and the plugin in editable mode.
6. Verifies the GPU and prepares all runtime caches below runtime.

To reuse an existing native ROCm PyTorch installation instead of installing the bundled wheels:

.\setup.ps1 -ExistingTorchSitePackages 'C:\path\to\Lib\site-packages'

Verify the installation

Run the GPU and Triton smoke tests:

.\test_gpu.ps1

Run a small end-to-end vLLM generation test:

.\test_first_token.ps1 -Model 'facebook/opt-125m'

The first-token test downloads the selected model into runtime\models when it is not already cached.

Start the OpenAI-compatible server

.\run_server.ps1 `
    -Model 'facebook/opt-125m' `
    -Port 8000 `
    -MaxModelLen 512 `
    -GpuMemoryUtilization 0.80

The server listens on 127.0.0.1:8000. Check that it is available:

Invoke-RestMethod http://127.0.0.1:8000/v1/models

Send a completion request:

$body = @{
    model = 'facebook/opt-125m'
    prompt = 'Explain paged attention in one sentence.'
    max_tokens = 32
} | ConvertTo-Json

Invoke-RestMethod `
    -Uri 'http://127.0.0.1:8000/v1/completions' `
    -Method Post `
    -ContentType 'application/json' `
    -Body $body

The server is configured for one GPU, eager execution, and the Triton attention backend used by the tested Windows ROCm stack.

Optional native C extension

The baseline uses the Windows ROCm plugin, Torch fallbacks, and Triton kernels. A native vLLM C extension can be built when MSVC and the Windows SDK are installed:

$env:VLLM_VCVARS64 = 'C:\Program Files\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvars64.bat'
.\build_vllm.ps1 -BuildNativeC

The build prepares the ROCm headers and import libraries automatically. The compiled extension and all intermediate files are written below runtime and are excluded from version control.

Repository layout

README.md
LICENSE
setup.ps1
prepare_runtime.ps1
build_vllm.ps1
run_server.ps1
test_gpu.ps1
test_first_token.ps1
requirements-rocm712-windows.txt
requirements-vllm-rdna4-windows.txt
patches/                       Windows compatibility patches
scripts/                       Runtime preparation and smoke-test helpers
vendor/                        Pinned checkouts downloaded by setup.ps1
.venv/                         Local Python environment, generated
runtime/                       Model cache, Triton cache, logs, and build files, generated

The generated environment, model files, source checkouts, caches, logs, and native binaries are ignored by .gitignore; the repository stays limited to setup code and reproducible configuration.

Windows-specific patches

The patches provide the small compatibility layer required by this tested native Windows configuration:

• Windows ROCm bootstrap and Torch fallbacks for optional vLLM C operations.
• Loopback TCP and Windows-safe socket and process handling.
• Windows selector event-loop support for pyzmq.
• Native HIP extension linking fixes for the Windows PyTorch wheel.
• POSIX separator normalization for Hugging Face repository paths.

They are applied automatically and are safe to reapply through setup.ps1.

Limitations

• The tested target is one AMD Radeon RX 9070 XT with gfx1201.
• Multi-GPU execution, tensor parallelism, RCCL, and distributed Windows execution are outside this setup.
• ROCm wheels, GPU drivers, Python, and native extensions are platform-specific.
• Model files are not included in the repository; they are downloaded to the local runtime cache.
• The current baseline uses --enforce-eager; graph-based execution is not part of the supported path.

License and attribution

The project-specific code is released under the Apache License 2.0. vLLM and the Windows ROCm plugin remain subject to their respective upstream licenses and terms. See LICENSE and the pinned upstream repositories for details.