Add new application: petsc-gmres

[william-dawson]

A PETSc KSP (GMRES + GAMG) benchmark, verified on Rikyu, Fugaku, and R-CCS Cloud's DGX Spark. [code:petsc-gmres]

The performance tables previously in this description have been removed rather than updated. They were measured on the audikw_1 matrix under a setup-inclusive FOM, and on a build that deadlocked on any 2+ node GPU run (fixed in this branch), so both the absolute numbers and the multi-node conclusions drawn from them no longer describe what this branch builds. Re-measured results will be posted once they are complete.

[yoshifuminakamura]

Thanks for the detailed validation notes. One current blocker is DCO: the check is failing on this draft PR. Could you please add Signed-off-by lines to the commits that are missing them, or otherwise update the branch so the DCO check passes?

[william-dawson]

GPU scaling re-measured under the iterate-only FOM

Per Suzuki-san's recommendation, the FOM now measures only the GMRES iterate (ksp_iter_time_s), with KSPSetUp called explicitly before the timed KSPSolve so GAMG setup is excluded. Re-measured on Rikyu (GB200 NVL4, 2026-08-17):

GPUs	solve_time_s (old, setup-inclusive)	ksp_iter_time_s (new, iterate-only)	relative L2
1	1.55	0.129	0.0443346
2	5.81	0.211	0.0342875
4	6.04	0.183	0.03154

Setup was 91-97% of the old FOM — even more dominant than the 70% -log_view reported. Under the iterate-only FOM, 1 GPU still wins (0.129s), but the gap narrowed from ~4x to ~1.5x. Notably, 4 GPUs now beats 2 GPUs (0.183 vs 0.211), reversing the old near-tie — the iterate does parallelize, just not enough at 943k rows (~236k rows/GPU at 4 GPUs) to overtake 1 GPU. A larger matrix is the honest next test of whether multi-GPU can win under the iterate-only FOM.

Correctness held throughout (L2 in the expected 0.03-0.04 band, bit-identical to the old runs at each GPU count).

[william-dawson]

stokes2 matrix — full three-machine results (iterate-only FOM)

Switched the benchmark matrix from audikw_1 (943k rows, SPD structural mechanics) to stokes2 (4.26M rows, Stokes flow saddle-point system, generated via Gmsh/FreeFEM by Suzuki-san). Already in PETSc binary format (FreeFEM ObjectView = MatView), so no mtx2petsc conversion needed. KSPSolve is ~10× KSPSetUp (confirmed via -log_view), so the iterate dominates — the iterate-only FOM is the right metric.

Rikyu (GB200 NVL4, GPU, 1 rank/GPU)

GPUs	ksp_iter_time_s	iterations	relative L2	speedup
1	1.497	232	0.2516	1.00x
2	0.932	200	0.2566	1.61x
4	0.683	193	0.2278	2.19x

Multi-GPU now wins — at ~1M rows/GPU there is enough work to amortize communication. Compare with audikw_1 under the same FOM: 1 GPU 0.129s, 4 GPU 0.183s (1.41× slower). The larger matrix flips the conclusion.

Fugaku (A64FX, multi-node CPU, 48 ranks/node, f-pt queue)

nodes	ranks	ksp_iter_time_s	relative L2	speedup
1	48	44.881	0.2663	1.00x
2	96	28.532	0.2582	1.57x
4	192	16.777	0.2317	2.67x
8	384	15.959	0.2641	2.81x

Clean scaling through 4 nodes (2.67× at 4× ranks), then nearly flat 4→8 (2.81× at 8× ranks) — communication wall hits ~192 ranks for this problem size on A64FX.

DGX Spark (GB10 Blackwell, 1 GPU)

machine	GPU	ksp_iter_time_s	relative L2
Rikyu	B200	1.497	0.2516
DGX Spark	GB10	21.430	0.2516

Correctness bit-identical (L2 0.251638 on both). 14.3× slowdown on GB10 vs B200 — larger than audikw_1's 5.2×, since the larger matrix stresses memory bandwidth more and GB10 has a smaller bandwidth budget.

Summary

All three machines validated via test_submit.sh with the stokes2.dat data staged at the per-system paths in run.sh. The f-pt queue was used for the Fugaku multi-node runs (wider node range than small); the committed list.csv/system.csv stay on small.