Skip to content

vulkan: decline GATED_DELTA_NET raw gates instead of computing them wrong - #239

Merged
bri-prism merged 1 commit into
prismfrom
fix/vulkan-gdn-raw-gates-unsupported
Sep 22, 2026
Merged

bri-prism merged 1 commit into
prismfrom
fix/vulkan-gdn-raw-gates-unsupported

Conversation

@bri-prism

Copy link
Copy Markdown
Collaborator

Fixes #237.

Problem

ggml_gated_delta_net_set_raw_gates() delivers beta and g pre-activation, so the op has to apply

beta = 1/(1 + exp(-beta))                    // sigmoid
g    = a[h] * softplus(g + dt_bias[h])       // dt_bias = src[7], a = src[8]

as the CPU implementation does (ggml/src/ggml-cpu/ops.cpp, the if (raw_gates) branch).

gated_delta_net.comp has neither binding and does none of that math — it applies exp(g) unconditionally. But supports_op only checks src[6], the head size and the src types; it never looks at op_params[1]. So the Vulkan backend claims these ops and returns results uncorrelated with the reference, silently — no abort, no warning, no fallback.

Measured on an Intel Arc B390 before this change, test-backend-ops test -o GATED_DELTA_NET -b Vulkan0:

case (head_count=4, head_size=128) ERR (tolerance 1e-7)
n_seq_tokens=1, n_seqs=1, v_repeat=1 0.897 - 0.996
n_seq_tokens=1, n_seqs=2, v_repeat=2 0.958 - 0.992
n_seq_tokens=64, n_seqs=1, v_repeat=1 1.000

15/15 failures over five consecutive runs. All 45 raw_gates=0 cases pass, and no raw_gates=1 case has ever passed — so the split is exactly on this flag. The SYCL backend on the same machine declines all four raw_gates=1 cases as not supported, which is the behaviour this change adopts.

Change

Nine lines in supports_op: decline when ggml_get_op_params_i32(op, 1) != 0, with a comment recording why.

This makes the case fall back to the CPU, which computes it correctly. It does not implement raw gates on Vulkan — that needs two new shader bindings plus the sigmoid/softplus math and care around the kda path, which is a much larger change. The trade here is speed for correctness on those shapes, which seems clearly right when the current behaviour is silent corruption.

Result

test-backend-ops test -o GATED_DELTA_NET -b Vulkan0, three consecutive runs: 36/36 passed, Backend Vulkan0: OK each time. The four raw_gates=1 cases now report not supported [Vulkan0].

Full unfiltered test-backend-ops test -b Vulkan0 on the same device:

OK failing cases not supported verdict
prism 422590f5d 17187 3 4667 FAIL
this branch 17188 0 4670 OK

The three failures become not supported; the Vulkan backend passes the suite on this device for the first time.

Note for #187

PR #187 implements the rows-indexed state path, which makes supports_op accept a fourth raw_gates=1 case (K=2, rows_mode=1) that stock prism declines for an unrelated reason. That case then executes and hits this same bug, which looks like a regression in #187 but is not one — it is this bug surfacing on one more shape. With this change applied that case is declined too, and #187's GDN results are clean.

…rong

ggml_gated_delta_net_set_raw_gates() delivers beta and g pre-activation,
so the op must apply beta = sigmoid(beta) and
g = a * softplus(g + dt_bias), with dt_bias in src[7] and a in src[8].

gated_delta_net.comp has neither of those bindings nor that math - it
applies exp(g) unconditionally - but supports_op never checked the flag,
so Vulkan claimed these ops and returned results uncorrelated with the
reference (ERR ~1.0 against a 1e-7 tolerance), silently and with no
fallback.

Decline raw gates in supports_op so they fall back to the CPU, which
implements them. This mirrors what the SYCL backend already does.

On Arc B390, test-backend-ops -b Vulkan0 goes from 3 failing
GATED_DELTA_NET cases to a clean run: the full unfiltered suite reports
FAIL before and OK after.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

vulkan: GATED_DELTA_NET silently wrong for raw_gates=1 (ERR ~1.0, 3 cases, 15/15 runs) — supports_op claims a path the shader doesn't implement

1 participant