All pull requests

Pull requests

• cuda: prefetch the next PTQ1_0 mat-vec's weights into L2 from the last CTAs

  CUDAggml
  #314·sb32445 opened Oct 4, 2026·

  ·

• common : take the top-k straight from the logits when top-k starts the chain

  #313·sb32445 opened Oct 4, 2026·

  ·

• server : reuse the buffers of evicted prompt checkpoints

  server
  #312·sb32445 opened Oct 4, 2026·

  ·

• kv-cache : limit seq_rm to the used cell range

  #311·sb32445 opened Oct 4, 2026·

  ·

• cuda: gate (SwiGLU) fused PTQ1_0 mat-vec for 2-4 columns

  CUDAggml
  #310·sb32445 opened Oct 4, 2026·

  ·

• cuda: use the transposing concat kernel on all GPUs, not only on GB10

  CUDAggml
  #309·sb32445 opened Oct 4, 2026·

  ·

• cuda: smaller KV tile for the 8-column MMA flash attention config (head size 256)

  CUDAggml
  #308·sb32445 opened Oct 4, 2026·

  ·

• cuda: use the MMA flash attention kernel for GQA above 4 with quantized K/V on Ada

  CUDAggml
  #307·sb32445 opened Oct 4, 2026·

  ·

• cuda: PQ2_0 mat-vec kernel for 3-8 columns on Ada

  CUDAggml
  #306·sb32445 opened Oct 4, 2026·

  ·

• speculative: add opt-in prompt lookup for DFlash2

  #304·bri-prism opened Oct 3, 2026Collaborator·

  ·

• fix(cuda/msvc): add CUDA 13.4 CUB fixes, Win32 CreateFileW, and graceful tensor name fallback

  Apple MetalconversionCUDAdevopsdocumentationexamplesggmltestingvendor
  #300·vrwallace opened Sep 30, 2026·

  ·

• qwen35: allow Vulkan for rows-mode GDN state + Vulkan 890M recipe doc

  documentationmodel
  #299·MrFadiAi opened Sep 30, 2026·

  ·

• vulkan: bf16 SSM state pools + gated_delta_net rows mode (+22% decode on 890M)

  ggmlVulkan
  #298·MrFadiAi opened Sep 30, 2026·

  ·
  3

• metal: opt-in Metal 4.1 direct Q2 operand experiment

  Apple Metaldocumentationggml
  #297·bri-prism opened Sep 30, 2026Collaborator·

  ·

• tests : GET_ROWS -> RESHAPE -> GATED_DELTA_NET case, eager + CUDA graph capture/replay

  testing
  #296·professorpalmer opened Sep 29, 2026·

  ·

• cuda: bound the one-column MUL_MAT_ID mat-vec store by the expert row count

  CUDAggmltesting
  #295·professorpalmer opened Sep 29, 2026·

  ·

• feat(metal): add multi-column batch verification kernel for PQ2_0 (M=2..8)

  Apple Metalggml
  #293·nogeonwoo opened Sep 28, 2026·

  ·
  4

• vulkan: support GPUs without hardware DP4A in mul_mat_vec_ptq1_0 and eliminate dequant divergence

  ggmlVulkan
  #291·Tetto-chan opened Sep 28, 2026·

  ·
  2

• arm: NEON + i8mm vec_dot for PTQ1_0

  ggml
  #290·karusrus opened Sep 28, 2026·

  ·

• qwen35: opt-in GDN verify tape that replays kept rows instead of per-row snapshots

  modeltesting
  #289·bri-prism opened Sep 28, 2026Collaborator·

  ·

• cuda+server: q8_0 K/V at the full 262k window on 12 GB (tiered VMM KV cache), MTP drafting at every depth

  CUDAggmlmodelservertesting
  #285·professorpalmer opened Sep 27, 2026·

  ·
  2

• vulkan: add dedicated PQ2_0 mat-vec and BC-250 batch tuning

  ggmltestingVulkan
  #281·renovys opened Sep 26, 2026·

  ·
  3

• metal : static-K32 tensor mul_mm for full-tile Q1_0 products (GGML_METAL_Q1_MM_K32=1)

  Apple Metalggmltesting
  #280·jasontitus opened Sep 26, 2026·

  ·

• metal : opt-in PQ2_0 mat-vec for two columns (GGML_METAL_PQ2_MULTICOL=1)

  Apple Metalggmltesting
  #279·jasontitus opened Sep 26, 2026·

  ·

• metal : PTQ1_0 multi-column mat-vec for 5-8 columns

  Apple Metalggmltesting
  #277·jasontitus opened Sep 26, 2026·

  ·