[Windows 11] AMD Adrenalin driver (32.0.31007.5012) causes R9700 VRAM to offload to system RAM on idle - crash with 32GB RAM

[realsheox-max]

Problem

AMD Radeon AI PRO R9700 on Windows offloads entire VRAM content
to system RAM after ~15 seconds idle, causing system crash
(32GB RAM fills up instantly).

Reproduces with both Vulkan and ROCm backends.
Also reproduces in LM Studio (Vulkan backend).

Root Cause

AMD Adrenalin Software (driver 32.0.31007.5012, dated 12.05.2026)
installs aggressive DPM power management that evicts VRAM on idle.
The basic Windows driver without Adrenalin does NOT have this behavior.

System

• OS: Windows 11 (Build 26200.8457)
• GPU: AMD Radeon AI PRO R9700 (32GB VRAM)
• CPU: Intel i7-11700KF
• RAM: 32GB DDR4
• Affected driver: 32.0.31007.5012 (Adrenalin, 12.05.2026)
• Working driver: 32.0.22042.14002 (09.03.2026, no Adrenalin)

Fix

1. Uninstall AMD Adrenalin, rollback to driver 32.0.22042.14002
2. Use --no-mmap without --mlock

Working command

.\llama-server.exe --device Vulkan1
--model "model.Q4_K_M.gguf"
--ctx-size 65536
--threads 8
--parallel 1
--n-gpu-layers -1
--host 127.0.0.1
--port 8080
--cache-type-k q4_0
--cache-type-v q4_0
--cache-ram 0
--no-mmap

What does NOT fix this

• DisableGpuMemoryTrim registry key
• KMD_EnableContextBasedPowerManagement=0
• PCI Express Link State Power Management=Off
• WSL2 with ROCm (same issue)

Additional Notes

The R9700 is marketed as a dedicated AI workstation card.
The target use case is exactly keeping LLMs permanently loaded
in VRAM for inference workloads.

Ironically, AMD ships the same consumer Adrenalin driver with
aggressive laptop-style power management on this workstation card.
When the LLM is idle between requests, the GPU already draws only
~8W — there is absolutely no reason to evict VRAM contents to
system RAM at that point.

This DPM behavior makes sense on laptops/integrated GPUs to save
battery. On a dedicated 32GB AI accelerator it actively destroys
the primary use case.

AMD should either:

1. Ship a compute/workstation driver profile without aggressive DPM
2. Add an option in Adrenalin to disable VRAM eviction on idle
3. Document this behavior clearly for AI/LLM workloads

Related Issues

• Dual AMD AI Pro R9700 cards failing #22871 - Similar symptoms on Linux/Docker
• [Issue]: Shared memory is forced with llama.cpp llama-server --mlock ROCm/legacy-rocm-build#4903 - --mlock forces shared memory on Windows AMD
• llama-server keeps loading and unloading the model from RAM: how to keep it in VRAM? #12800 - AMD DPM offloads VRAM to GTT on Linux (fix: amdgpu.runpm=0)

[Jrjy3]

Just got this card today, and I'm facing the same bug. This is the behavior I see in task manager.

Note that I also have a 5080 in this system, but it's disabled and only running on the R9700.

[TOS10000]

Same issue with that version of Adrenalin including up to current version 26.6.4 (32.0.31021.5001) with a 7900xtx. 'Problem' and 'Fix' sections are absolutely correct.

[badbat75]

Second independent confirmation here, on a different system and a newer Adrenalin build, so this is not a one-off regression tied to a single driver release.
System:

GPU: single Radeon AI PRO R9700 (32GB), gfx1201
OS: Windows 11 25H2, build 26200.8737
Affected driver: Adrenalin 32.0.31021.5001 (newer than the 32.0.31007.5012 in the original report)
Known-good here: driver 32.0.22042.14002 (30 April 2026), with AMD Software 26.3.1 installed. Correction to my earlier wording: the trigger is not the presence of Adrenalin, it is the driver build. 26.3.1 is clean even with the full Adrenalin package installed, as independently confirmed in ROCm/TheRock#7221, where the regression window is 26.3.1 good, 26.5.1 onward broken.

Same behavior: with the model fully resident in VRAM, after roughly 15s idle the entire allocation is evicted to system RAM, and the next request pays a full PCIe re-upload cost.
One technical clarification worth adding, because it changes which mitigations make sense: this is a real WDDM eviction to the allocation backing store, not a GTT/shared-memory spillover. In Task Manager, Dedicated GPU memory drops while the process committed/working set rises, and the "Shared GPU memory" counter stays flat. So it is a full de-residency with a complete host copy of the model, which is why low-RAM systems fill up and crash. This also means anything aimed at GTT sizing has no effect, consistent with the fixes reported as not working above.
Two known reporters now, two different Adrenalin builds, two different systems, both on 25H2. The OS variable is controlled: the behavior appears and disappears by changing only the driver.

[YUXUANCHENG]

Can confirm too. Rolling back to an earlier version driver is the only fix.

[SirMathhman]

I agree. Need to download driver at link: https://www.amd.com/en/support/downloads/previous-drivers.html/graphics/radeon-ai-pro/radeon-ai-pro-r9000-series/amd-radeon-ai-pro-r9700.html specifically on version: 2026-03-19

[realsheox-max]

Still on 32.0.23033.1002 / 09.03.2026 Driver only no Adrenaline ... tried it with newer Driver ... no chance still the same issue haha... imagine release a Workstation GPU and no working drivers since months ^^
But now with the old driver and vulkan backend i can run my r9700 + 4070 in hybrid Gpu without any problems =)

[dsh761]

Just brought the R9700 today, tried in Window LMStudio with the latest driver download from Adrenaline. I got the same issue. VRAM was offloaded completely to 0 and my 32GB ram was exploding and caused the system crash. Thank you all for clearly described the issue here and provided a solution of using a rolled backed driver.

[peppekerstens]

Confirming this on a Windows 11 + Radeon AI PRO R9700 (32GB) setup, running Unsloth Studio's bundled llama-server (Vulkan backend, -ngl 99, Vulkan1 device) for chat inference.

Driver: 32.0.31035.1003 (AMD Software: Adrenalin Edition 26.7.1, the current latest WHQL release as of 2026-08-18) — bug is still present. The latest driver does not fix this.

Three full-system bugchecks within ~2 hours of real usage today, all flagging the same driver:

• 0x00000116 (VIDEO_TDR_FAILURE) — after ~1hr of sustained chat inference
• 0x0000007E (SYSTEM_THREAD_EXCEPTION_NOT_HANDLED, access violation 0xC0000005) — ~5 min after a driver/llama.cpp binary update+restart
• 0x00000116 again — 33 seconds after a long (7.7 min) generation request finished and the engine went idle

All three System event log entries name amdkmdag.sys as the likely-related driver. The idle-adjacent timing (crash arriving shortly after load ends, not during) matches the VRAM-offload-on-idle behavior described in this thread rather than a mid-computation hang.

Checked Windows' TDR watchdog settings — both TdrDelay/TdrLevel were unset (Windows defaults, 2s timeout). Increased TdrDelay to 10 as a partial mitigation; hasn't been long enough to confirm whether it helps, but doesn't address the underlying idle-VRAM-eviction mechanism this thread describes.

[lunasnow1231]

New data point for this thread, from the same 2x R9700 / Windows 11 box as my earlier report (driver matrix: 31007/31019/31035 evict, 23027/22042 don't).

The 31xxx eviction is gated on display power state, not idle time. With a dummy plug + live monitor, 31035 kept ~53GB resident through 9+ hours of idle — looked fixed. The moment Windows turned the displays off (30-min timer), the driver evicted the dummy-plugged card's entire 24.8GB in ~6s (~4GB/s, caught on a 1-second fsync'd sampler), started on the second card, hung mid-eviction, failed TDR recovery twice (LiveKernelEvent 141) and bugchecked with 0x116 VIDEO_TDR_FAILURE / amdkmdag.sys. Three nights in a row, then reproduced on demand with a SC_MONITORPOWER broadcast: screen blank to BSOD in ~27 seconds.

So: dummy plugs only mask the idle path while screens are on; they do not prevent the display-off eviction, and there seems to be no registry switch for it (consistent with the attempts reported above).

On 32.0.22042.14002 the same artificial display-off test shows zero eviction (VRAM immovable 7+ min after blank). That rollback remains the only real fix I'm aware of.

(Posting as a data point; I likely won't be able to follow up.)

[zsmooter]

As a workaround to this, I've just been keeping a tiny workload constantly on the GPU, like so:

a = torch.tensor(0, device=torch.device("cuda"))
while True:
    a.clone()
    time.sleep(0.01)

[realsheox-max]

Solved (for now) by switching driver branch: AMD Software: PRO Edition 26.Q3 fixes the idle VRAM eviction on the R9700.

Short version: the eviction is tied to the Adrenalin driver branch. Moving from Adrenalin to the dedicated PRO Edition driver stops it. On my machine the model now stays fully resident in VRAM even after long idle periods, on both Vulkan and ROCm.

What was happening

With Adrenalin builds from 26.5.1 onward, a fully VRAM-resident model got evicted to system RAM after ~15s idle (WDDM eviction to the allocation backing store). On a 32GB-RAM box this fills RAM and the next request pays a full PCIe re-upload. Adrenalin 26.3.1 (32.0.22042.14002) was the last clean build.

The fix

AMD Software: PRO Edition 26.Q3 (released 18 Aug 2026) added support for the R9700 SKU. Switching to it resolves the eviction.

Steps:

1. Download AMD Software: PRO Edition 26.Q3 from the R9700 driver page (don't install yet).
2. Run the AMD Cleanup Utility, let it reboot into safe mode, and remove all existing AMD drivers.
3. Install the PRO Edition 26.Q3 driver.
4. Reboot.

Test result

• Model: 35B MoE, 160k–180k context
• Vulkan: stays resident on idle, no RAM growth. ✅
• ROCm (gfx1201, HIP device 0): stays resident on idle. ✅
• Even with the High-Performance profile active. ✅

Notes / caveats

• This is a branch switch, not an Adrenalin fix. Adrenalin 26.5.1+ still evicts. If you need Adrenalin for some reason, you're still stuck on 26.3.1.
• The PRO driver only gained R9700 support on 18 Aug 2026, so before that this workaround simply wasn't available — which is why we were all stuck on the old Adrenalin build.
• On my setup Vulkan is still noticeably faster than ROCm for token generation (~135 t/s vs ~84 t/s on the same 35B MoE), so I'm staying on Vulkan — but both are now free of the eviction bug.
• Reminder for ROCm launch: the R9700 is HIP device 0 (verify with llama-server --list-devices), and set ROCR_VISIBLE_DEVICES=0 to hide the iGPU so llama.cpp doesn't put the KV cache on it.

Hopefully this saves others the months of driver roulette. Would still be good to see AMD fix the Adrenalin branch itself, since plenty of R9700 users run it.

[realsheox-max]

start_llama-dualgpuv2.zip
I added my Hybrid Llama Starting bat ... working fine for me now

[badbat75]

Initially it didn't work for me, now it seems doing. So let me do further checks before confirming... this is very strange.

[badbat75]

it stopped working after some reboots. Something weird happened. I uninstalled everything with AMD Cleanup Utility e reinstalled again. Still VRAM eviction. Reverted to stock windows driver that works.

[the629]

Same issue here with 2x r9700s. Went through all troubleshooting steps over the last month. Sleep settings, purchased a dummy plug for the second GPU, etc.

I have rolled back and I am monitoring currently.

[realsheox-max]

yep same for me again ... i could puke ...

[realsheox-max]

Update on my earlier "solved by PRO Edition 26.Q3" post — I need to correct the outcome.

To be clear about the history: the first PRO Edition driver I installed genuinely worked. The card ran fully loaded — ~90% of the 32GB VRAM in use — with no eviction, exactly as it should. This was never a light-load fluke.

It only broke again after a later driver update. So the PRO branch was fixed at one point, and a subsequent update reintroduced the regression. That's the key point: this isn't the card's limit or my setup — AMD had a working driver and a newer build broke it again.

Current state: model loads fully into VRAM, then after ~10s idle the driver evicts the whole allocation to system RAM, and the next request crashes — same eviction as on Adrenalin. My current (broken) driver is 32.0.31036.15, driver date 2026-08-12.

Worth stressing: on my box the R9700 is a pure compute card with no display attached (video output runs off a separate GPU), so this has nothing to do with monitor/display-off power states — it's straight idle eviction on a headless compute card, triggering in ~10s.

Reliable fix remains rolling back to 32.0.22042.14002 (AMD Software 26.3.1) — the build the card ran fine on under full load.

I've reopened my case with AMD support and pushed for escalation to the driver team, with the full writeup (WDDM eviction, regression window, the registry tweaks that don't work, and this thread as reference), and asked for a fix timeline on both Adrenalin and PRO.

Will post back if I get anything substantive.

[badbat75]

AMD has root-caused this on the tracker, ROCm/TheRock#7221. From nkulshre-amd, 2026-09-10:

The root cause of the issue is that the KMD driver failed to correctly provide the OS with details about the memory segments that should remain active when the device is in D3.
The fix is mainlined into v26.9.2 which can be expected around end of Sep 2026.

That fits the ETW traces I posted there (ROCm/TheRock#7221 (comment)): after ~10 s with no GPU work, Windows suspends the headless card to D3, and since the driver does not declare which memory segments survive D3, dxgkrnl purges the whole VRAM first. It also explains why none of the registry tweaks discussed here helped, and why no build before 26.9.2 is expected to fix it: I tried PRO 26.9.1 (32.0.31041.3013) this week and it still evicts.

Until 26.9.2 ships, rolling back to 32.0.22042.14002 remains the workaround most of us are using. I'll report back here once I've tested 26.9.2 on my R9700.

[badbat75]

Follow-up to my comment of 2026-09-11, as promised: AMD driver 26.9.2 fixes the idle VRAM eviction on my headless R9700.

• Driver 32.0.32015.2008 (2026-09-28), clean reboot after install.
• ~30 GiB model loaded in llama-server (ROCm, split with a 4070 Super), then left completely idle: after ~14 minutes the process still holds 30,437 MiB of dedicated VRAM on the R9700. On 26.5.1 through PRO 26.9.1 it dropped to 0 about 10 s after the last request.

So the rollback to 32.0.22042.14002 is no longer needed: update to 26.9.2 instead. AMD's root cause and the tracker are in ROCm/TheRock#7221.

[SV-ZeroOne]

I ran into this bug with my dual R9700 setup, here is a workaround for people with two R9700 or more until AMD decides to fix their drivers :/

Dual R9700 on current Adrenalin — a working interim fix, no rollback needed

Rollback to 32.0.22042.14002 is this thread's standard advice. If you can't roll back, or you're
running two R9700s, here's what works on current drivers. Tested on Adrenalin 26.8.1
(32.0.31041.1004), llama.cpp Vulkan b10997, Qwen3.8-27B.

The dual-card gotcha nobody's mentioned yet

A keepalive on one card does not protect the other. On a two-card box, the card you aren't
actively using goes to CM_PROB_FAILED_ADD (code 31) within ~5 minutes, drops out of Vulkan
enumeration, and logs nothing at all — no WHEA, no bugcheck, no Kernel-PnP entry. Recovery is
a Device Manager disable/re-enable, or a reboot if that doesn't take.

A spare card also makes the bug fire sooner, because it guarantees an idle GPU always exists.

Fix: use a layer split. Every inference then runs a full forward pass through all layers, so
even a 1-token keepalive poke touches every card. Neither goes idle.

llama-server.exe --device Vulkan1,Vulkan2 --split-mode layer -ngl 999 -c 32768 ...

Check --list-devices first and pin explicitly — indices move, and an iGPU will happily advertise
~48 GiB of UMA and swallow the whole model into system RAM without complaining.

The keepalive

Eviction fires at ~8.7 s idle on my system (#7221 measured ~9.5 s). Poke every 5 s for margin:

while true; do
  curl -s -m 120 http://127.0.0.1:8080/completion \
    -H 'Content-Type: application/json' \
    -d '{"prompt":"x","n_predict":1,"cache_prompt":false}' >/dev/null 2>&1
  sleep 5
done &

PowerShell equivalent:

Start-Job { while ($true) {
  try { Invoke-RestMethod -Uri 'http://127.0.0.1:8080/completion' -Method Post -TimeoutSec 120 `
      -ContentType 'application/json' `
      -Body '{"prompt":"x","n_predict":1,"cache_prompt":false}' | Out-Null } catch {}
  Start-Sleep -Seconds 5 } }

/health will not work. It doesn't touch the GPU queue, so it never resets the inactivity
timer. The poke has to be real inference. This is the easiest thing to get wrong.

Cost

~41 ms per poke — a 0.8 % duty cycle. Measured over a 50-minute run: 28.41 GB held resident
across 596 consecutive pokes, no gap over 10 s, and no measurable throughput impact (a real
223-token request mid-run returned 43.39 tok/s decode, matching the normal baseline).

Dual-card layer split: 34.20 GB resident (19.03 + 15.17), both cards healthy through 341 s
idle, zero WHEA. ~45–50 tok/s decode up to ~30k context.

With -np 1 it's self-regulating — if the server is busy the poke just queues behind real work,
and a busy GPU isn't an idle one.

Caveats

• This doesn't fix the bug, it outruns it. Stop poking and you're back to a ~9 s fuse.
• No help for apps you can't modify to self-poke.
• Untested: whether an uneven layer split leaves the lighter card idle long enough to fail.
• If you do roll back instead: note at least two packages ship as "26.3.1". The R9700-specific one
  (32.0.23033.1002) fixes the eviction but then crashes llama-server on its first request
  inside amdvlk64.dll — llama-bench and llama-completion are fine, so it's easy to
  benchmark it, think it worked, and only find out when you start serving. Quote build numbers,
  not marketing versions.

[ilroby73]

Same issue here on 2 different setup: W7800 48gb and R7900 xtx 24gb

[realsheox-max]

Hmm i downloaded the newest pro driver yesterday and for now it is working again .... for now XD

[ilroby73]

Thanks for the info!

[FasterDEV]

@realsheox-max thanks for the update. Could you share the exact driver build number and where you downloaded it from?

As of today (2026-09-27) the R9700 download page still only lists PRO Edition 26.Q3

[realsheox-max]

hey sorry .. no time for pc XD

[realsheox-max]

still running qwen coder in Vram without Memory crash ...
Solo r9700 and Hybrid with 4070 no probs

[konopka]

I have a bit different experience

with
amd-software-pro-edition-26.q3-win11-b.exe
32.0.31036.15
12.8.2026

immediate crash (maybe different kind of bug?) after starting prompt

0.00.001.933 I srv  llama_server: initializing ...
0.00.693.281 I cmn  common_param: common_params_print_info: verbosity = 3 (adjust with the `-lv N` CLI arg)
0.00.693.368 I srv  init_listene: The UI is disabled
0.00.693.369 I srv  init_listene: Use --ui/--no-ui (or deprecated --webui/--no-webui) to enable/disable
0.00.693.454 W srv  llama_server: security: no API key is set and CORS allows all origins (see https://github.com/ggml-org/llama.cpp/pull/25655)
0.00.701.526 I srv    load_model: loading model '.\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-UD-Q4_K_M.gguf'
0.11.929.363 I cmn          init: llama threadpool init, n_threads = 12
0.14.830.660 I common_speculative_init_result: creating MTP draft context against the target model '.\unsloth\Qwen3.8-27B-GGUF\Qwen3.8-27B-UD-Q4_K_M.gguf'
0.15.019.741 W load_hparams: Qwen-VL models require at minimum 1024 image tokens to function correctly on grounding tasks
0.15.019.747 W load_hparams: if you encounter problems with accuracy, try adding --image-min-tokens 1024
0.15.019.747 W load_hparams: more info: https://github.com/ggml-org/llama.cpp/issues/16842

0.18.077.719 I srv    load_model: loaded multimodal model, '.\unsloth\Qwen3.8-27B-GGUF\mmproj-BF16.gguf'
0.18.077.742 I srv    load_model: initializing, n_slots = 4, n_ctx_slot = 262144, kv_unified = 'true'
0.18.983.407 W srv          init: chat template supports preserving reasoning, it is enabled by default (may use more tokens, disable via --no-reasoning-preserve)
0.18.983.423 I srv  llama_server: model loaded
0.18.983.425 I srv  llama_server: listening on http://0.0.0.0:8080
0.18.983.425 W srv  llama_server: notice: server default port will be changed to :9931 in a future release (ref: https://github.com/ggml-org/llama.cpp/pull/26508)
0.52.759.743 I slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
0.52.759.860 I slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
4.30.957.327 W ggml_vulkan: Failed to allocate pinned memory (vk::Device::allocateMemory: ErrorDeviceLost)
4.31.014.149 E srv  update_slots: decode() failed: vk::CommandBuffer::end: ErrorDeviceLost
4.31.014.162 E srv    send_error: task id = 0, error: decode() failed: vk::CommandBuffer::end: ErrorDeviceLost
4.31.014.174 I slot      release: id  3 | task 0 | stop processing: n_tokens = 48, truncated = 0
4.31.014.181 W srv          stop: cancel task, id_task = 0
4.31.014.192 I slot get_availabl: id  2 | task -1 | selected slot by LRU, t_last = -1
4.31.017.457 I slot launch_slot_: id  2 | task 2 | processing task, is_child = 0

So I went for latest adrenalin driver
whql-amd-software-adrenalin-edition-26.9.2-win11-c.exe
32.0.32015.2008
28.9.2026

seems to be working so far for me ...