Skip to content

Add fused reduction support - #4369

Open
sakupan102 wants to merge 1 commit into
ml-explore:mainfrom
sakupan102:fuse-reduction
Open

Add fused reduction support#4369
sakupan102 wants to merge 1 commit into
ml-explore:mainfrom
sakupan102:fuse-reduction

Conversation

@sakupan102

@sakupan102 sakupan102 commented Aug 21, 2026

Copy link
Copy Markdown

Closes #4355.

  • Introduced the CompiledReduce primitive, which fuses unary element-wise operations
    with a following all-reduce operation.
  • Added JIT generation and execution of fused reduction kernels with unary prefixes
    for Metal and CUDA.

Metal results

Function and input Before (us) After (us) Change Speed-up
max(abs(x)), 32 x 32 5.610 5.225 -6.9% 1.07x
max(abs(x)), 128 x 128 14.780 9.544 -35.4% 1.55x
max(abs(x)), 1024 x 1024 111.615 35.655 -68.1% 3.13x
sum(exp(abs(x))), 1024 x 1024 111.318 35.064 -68.5% 3.17x

TODO

  • Support column and row reductions. The current implementation supports only
    all reductions.
  • Support fusing into ArgReduce.

  • ☑️ I understand it is strictly prohibited to use AI to write PR description
  • AI usage disclosure: I used Codex for writing the code.

@sakupan102

Copy link
Copy Markdown
Author

@nastya236 Could you take a look at this?

@nastya236 nastya236 added enhancement await verification This pull request is non-trivial and requires a human expert to verify its correctness. await discussion This pull request makes some major changes that requires the team to have a discussion. labels Aug 21, 2026
Co-authored-by: Anastasiia Filippova <a_filippova@apple.com>
@sakupan102

Copy link
Copy Markdown
Author

@nastya236
Since this is quite a large change, would it be better to split this PR into multiple smaller ones (e.g., separate PRs for Metal, CPU, and CUDA support)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

await discussion This pull request makes some major changes that requires the team to have a discussion. await verification This pull request is non-trivial and requires a human expert to verify its correctness. enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support fusing element-wise operations into reductions

2 participants