vulkan: bf16 SSM state pools + gated_delta_net rows mode (+22% decode on 890M)

[MrFadiAi]

Split-out part 2 of #187, rebased on current prism, per review feedback.

Scope: bf16 SSM state + GDN rows-mode only (PTQ1_0 matvec already landed; httplib edits dropped; MUL+FWHT withdrawn).

• gated_delta_net.comp: USE_STATE_ROWS variant reads the initial state directly from the cache row given by an int32 index buffer (binding 7); STATE_BF16 variant reads a bf16 state pool with bit-exact uint16<<16 conversion
• 8 new pipelines: rows f32 / rows bf16state × {clustered, subgroup, shmem} × kda
• scale_bf16 pipeline + GGML_OP_SCALE BF16 dispatch/support
• dispatch + supports_op accept src[6] index tensor with F32 or BF16 state
• llama-model.cpp: opt-in env LLAMA_SSM_BF16_STATE / LLAMA_SSM_BF16_CONV allocate hybrid recurrent pools as BF16 (default behavior unchanged)
• ggml.c: ggml_gated_delta_net_rows accepts BF16 state

Measured, Radeon 890M (RDNA 3.5), Bonsai-2-27B Q2_0-fork + MTP n-max 1:

• single-stream: 10.4 → 12.7 t/s (+22%)
• batch np16 aggregate: 18.46 → 20.30 t/s
• Quality: 5/5 short-form gates + three 900-token factual generations (bf16 state error does not compound into answers on this model)

Independently valuable on bandwidth-starved iGPUs; no interaction with the landed PTQ1_0 paths.

[bri-prism]

Thanks for splitting this out. As pushed the branch doesn't build for me (shader-gen only emits the clustered bf16state variant, a stray brace in supports_op GATED_DELTA_NET, and the leftover src[6] early-return). With those fixed it builds and the f32 rows-mode tests pass, but LLAMA_SSM_BF16_STATE=1 aborts at load because supports_op SCALE still only admits F32, and the scale_bf16 shader is compiled as float16_t rather than bf16. Could you push a branch that builds and loads with the opt-in, add test-backend-ops cases for the bf16 state and bf16 SCALE paths, and include a KLD or PPL ratio for bf16 vs f32 state? Happy to retest on Intel and Apple after that.

[bri-prism]

Follow-up after getting the BF16 path to load (needed a scale.comp bf16 branch, uint16 shader types, and SCALE BF16 in supports_op). BF16 state alone is byte-identical to F32 in decode with KLD 1.7e-4 on wikitext, so that part is sound. LLAMA_SSM_BF16_CONV=1 produces degenerate output in real decode (prompt repetition) even though chunked perplexity looks fine, since the conv state is never read back there. On an Intel Xe3 iGPU the decode gain is 2 to 3% with rows mode, not 22%, so the 890M number may depend on the MTP head. Could you drop or fix BF16_CONV and add decode-level tests for the BF16 state path?

[bri-prism]

Three more things from reading the diff, not covered above:

1. The ggml.c assert now allows a BF16 state for ggml_gated_delta_net_rows on every backend, and the env vars are read in llama-model.cpp, so they apply everywhere. But the CPU kernel reads src[5] as float *, and Metal's supports_op doesn't check the state type. With the env var set and speculation on, CPU, Metal, and any CPU fallback (for example CUDA, which rejects src[6]) would read BF16 data as F32 without any error. Could you reject BF16 state in those backends' supports_op, or keep the assert F32-only outside Vulkan?

2. In ggml_vk_op_get_pipeline, the bf16state pipeline is chosen whenever src[5] is BF16, even when src[6] is null, and in that case the dispatch binds only 7 buffers. supports_op also skips the src[5] type check for BF16 without requiring src[6]. Please gate both on src[6] != nullptr.

3. Minor: the rows pipelines reuse the non-rows names from gdn_names, which makes them hard to tell apart in perf logs, and the scale_bf16 line is over-indented.

Approved at a head that does not build and whose BF16 opt-in aborts at load; see review comments above.