Add HIP MFMA dense GiMMiK kernel

[tomjen12]

Summary

Add an initial HIP dense GEMM path using CDNA F64 MFMA instructions.

The kernel densifies and bakes the constant A matrix into MFMA fragment order, stages B through LDS using an m-split/k-blocked path, and is emitted as an additional GiMMiK autotune candidate.

This PR is intended to stack on top of #19.

Results

On seven dense PyFR cases on MI300X, the new MFMA dense path reaches ~78% of rocBLAS geomean. It is added as an additional candidate rather than replacing the existing #19 HIP GiMMiK kernels.

Further tuning will continue in follow-up work, focusing on MFMA utilization, dependency stalls, and register pressure.

Test plan

• Ran MI300X dense PyFR matmul benchmarks for the seven cases shown above.
• Ran HIP GiMMiK correctness checks against NumPy reference results.

[tomjen12]

Hi @FreddieWitherden, could you help review the PR first, we will keep optimizing the kernel.

[FreddieWitherden]

${'beta*' if beta != 1 else ''}

[tomjen12]

Updated as suggested. I simplified the preload-C beta handling in both bstream.mako and the matching bstream-msplit.mako path.

[tomjen12]

Update: MFMA Dense GEMM vs rocBLAS

This update adds the mfma-tile-gemm template-based path for HIP GiMMiK and benchmarks it on seven PyFR cases that currently fail the GiMMiK suitability check (nuq > 128 and nnz / arr.size > 0.15) and therefore fall back to rocBLAS.

The goal is to avoid the long rocBLAS autotuning phase for these cases while keeping execution performance close to rocBLAS.

GPU	Cases	Precision	MFMA perf vs rocBLAS (min / geomean)	rocBLAS candidates/case	MFMA candidates/case	rocBLAS create total	MFMA create total	Create speedup (geomean)
MI300X	7 dense PyFR cases	double	93% / 101%	383-421	4	10402.51 ms	5174.58 ms	2.22x
MI325X	7 dense PyFR cases	double	96% / 102%	383-421	4	7407.14 ms	2002.32 ms	3.86x
MI355X	7 dense PyFR cases	double	100% / 104%	36-59	4	1836.60 ms	1707.15 ms	1.03x

Observations

On MI300X and MI325X, rocBLAS evaluates hundreds of solution candidates per case, while the MFMA path evaluates only 4. This reduces kernel creation/autotune time while keeping MFMA execution performance close to rocBLAS.

The benefit is clearest on MI325X, where MFMA reaches 102% geomean performance vs rocBLAS and improves kernel creation time by 3.86x geomean. MI355X shows a smaller creation-time gap because rocBLAS exposes fewer candidates for these shapes, only 36-59 per case.

[FreddieWitherden]

The benefit is clearest on MI325X, where MFMA reaches 102% geomean performance vs rocBLAS and improves kernel creation time by 3.86x geomean. MI355X shows a smaller creation-time gap because rocBLAS exposes fewer candidates for these shapes, only 36-59 per case.

I expect this number might increase with future rocBLAS releases as they tend to add more kernels over time, so the improvement in start-up is probably bigger than we expect.

With regards to performance do we have any % vs roofline for the dense cases to give us an idea about what is potentially left on the table?

[FreddieWitherden]

Also, do the kernels work for MI250X? Be good if we could decouple what will compile on other GPUs vs what has been tuned for. Then we can decide how best to gate things.

[tomjen12]

Additional validation

Commit 57c8230 adds indexed candidate rendering in base.py, which is used by the PyFR integration in PyFR/PyFR#570 to render GiMMiK candidates independently during HIP GiMMiK autotuning.

I also ran a 1000-case synthetic-shape benchmark on MI300X to check whether the new HIP MFMA dense candidates generalize across broader M/K/N coverage. The first 20 cases are PyFR internal p2-p5 tet shapes; the remaining cases are synthetic dense shapes.

Metric	Value
Cases	1000
Worst-case MFMA / rocBLAS BW	95%
Geomean MFMA / rocBLAS BW	102%

Across the selected 1000 shapes, the HIP MFMA GiMMiK candidates remain competitive with rocBLAS: the worst case is 95% of rocBLAS bandwidth, and the geomean is 102%.

Attached CSV files:

• synthetic_1000_summary.csv: aggregate bandwidth summary.
• synthetic_1000_details.csv: per-case bandwidth results for the selected 1000 shapes.

synthetic_1000_summary.csv
synthetic_1000_details.csv

[tomjen12]

The benefit is clearest on MI325X, where MFMA reaches 102% geomean performance vs rocBLAS and improves kernel creation time by 3.86x geomean. MI355X shows a smaller creation-time gap because rocBLAS exposes fewer candidates for these shapes, only 36-59 per case.

I expect this number might increase with future rocBLAS releases as they tend to add more kernels over time, so the improvement in start-up is probably bigger than we expect.

With regards to performance do we have any % vs roofline for the dense cases to give us an idea about what is potentially left on the table?

I have not done a roofline analysis for these dense cases. For this stage I have focused on making the MFMA GiMMiK path competitive with rocBLAS across a broader shape set, and on reducing the rocBLAS autotuning/start-up cost.

For the selected 1000 MI300X synthetic cases, MFMA GiMMiK is at 95% worst-case and 102% geomean bandwidth relative to rocBLAS, while the PyFR-side parallel compile path makes tuning 9.47x faster than rocBLAS over the same cases.
PyFR/PyFR#570 (comment)

I think a roofline study would be useful as a next-stage performance analysis if we want to quantify the remaining kernel-level headroom.

[tomjen12]

Also, do the kernels work for MI250X? Be good if we could decouple what will compile on other GPUs vs what has been tuned for. Then we can decide how best to gate things.

So far I have only run this on MI300X/MI325X/MI355X. I can run it on MI250X as well and do some tuning there to see whether we should enable these candidates on MI250X by default.

[tomjen12]

Update: I rolled this PR back to the previous commit and split the indexed candidate rendering changes into #21.

This keeps #20 focused on the HIP MFMA dense GiMMiK kernel support, while #21 covers the candidate-rendering infrastructure used by the PyFR parallel compile integration.

[FreddieWitherden]

Are you okay to rebase? This is the next item I'm looking to review and get merged.

[tomjen12]

Are you okay to rebase? This is the next item I'm looking to review and get merged.

Yes, I am going to rebase this branch.

[FreddieWitherden]

One thing we'll want to think about is how to fallback. Specifically, on tuned arch's we should not need to rocBLAS, but on say older CDNA and RDNA we may want to still pass through it (unless we're extremely confident in our new MFMA path).

[tomjen12]

Rebased onto the latest master.

I also tested MI250X/gfx90a. The MFMA candidates compile and run correctly, but performance is weaker on the 7 dense PyFR cases I tested:

GPU	Cases	Precision	MFMA perf vs rocBLAS (min / geomean)
MI250X	7 dense PyFR cases	double	81% / 96%
MI300X	7 dense PyFR cases	double	93% / 101%
MI325X	7 dense PyFR cases	double	96% / 102%
MI355X	7 dense PyFR cases	double	100% / 104%

Based on this, I added a GCN arch gate so the MFMA tile candidates are only emitted for gfx942 and gfx950 for now. This keeps gfx90a on the existing fallback path while still enabling the tuned MI300/MI325/MI355 targets.

[FreddieWitherden]

Thanks, this is looking good.For the cases where we lose do we know what tiling approach the winning rocBLAS kernel uses? Might be an easy strategy we can adopt. I think we can get it from the profiler once the auto tuning has finished.Regards, Freddie.On 22 Jul 2026, at 00:42, tom.jen ***@***.***> wrote:tomjen12 left a comment (PyFR/GiMMiK#20) Rebased onto the latest master. I also tested MI250X/gfx90a. The MFMA candidates compile and run correctly, but performance is weaker on the 7 dense PyFR cases I tested: GPU Cases Precision MFMA perf vs rocBLAS (min / geomean) MI250X 7 dense PyFR cases double 81% / 96% MI300X 7 dense PyFR cases double 93% / 101% MI325X 7 dense PyFR cases double 96% / 102% MI355X 7 dense PyFR cases double 100% / 104% Based on this, I added a GCN arch gate so the MFMA tile candidates are only emitted for gfx942 and gfx950 for now. This keeps gfx90a on the existing fallback path while still enabling the tuned MI300/MI325/MI355 targets. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: ***@***.***>