Two deviations from PETSc's defaults, both costly
Underworld sets pc_mg_type = "additive" in every GAMG bundle
(petsc_generic_snes_solvers.pyx lines 3260, 4173, 5189, and 6073 for the
Stokes velocity block). PETSc's default for GAMG is multiplicative.
Additive multigrid is a strictly weaker cycle — the levels do not see each
other's corrections.
And the velocity operator reaches GAMG with block size 1 on a
two-component field. GAMG then aggregates each scalar degree of freedom
independently rather than aggregating nodes, which is the standard way to
cripple algebraic multigrid on a vector problem.
Measured
SolKz, unit box, constant viscosity, Taylor-Hood P2-P1, 11 727 unknowns,
refinement 2 — nothing hard is happening here:
| GAMG configuration |
cycles per velocity solve |
flops |
wall |
| as Underworld configures it |
243–347 |
1.155e10 |
1.38 s |
pc_mg_type=multiplicative |
74–118 |
6.892e9 |
0.90 s |
| block size 2 |
84–120 |
4.599e9 |
0.70 s |
| both corrected |
34–47 |
3.121e9 |
0.54 s |
Seven times the cycles and 3.7 times the arithmetic, for two settings.
At larger sizes the uncorrected configuration is worse still: at 186 879
unknowns it needs up to 4 136 cycles per velocity solve, which overruns any
reasonable ksp_max_it and then silently reports a truncated solve.
Why it matters beyond performance
GAMG is what every solver falls back to when the mesh carries no refinement
hierarchy, which is the common case. So this is the default path for most
users, and it is slow for reasons that have nothing to do with the method.
It also makes any geometric-versus-algebraic comparison meaningless: a
measurement against this configuration is measuring our own settings.
Suggested fix
Remove pc_mg_type = "additive" and let PETSc choose. Deviating from a PETSc
default wants a recorded reason, and there is no comment giving one here.
Set the block size on the operator for vector fields so GAMG aggregates nodes.
The matrix is built by PETSc from the DM, so the natural place is wherever the
section is set up rather than after the fact.
Worth checking the other three call sites for the same reasoning, and whether
pc_gamg_agg_nsmooths=2 and pc_gamg_repartition=true have recorded
justifications or are also inherited without one.
Underworld development team with AI support from Claude Code
Two deviations from PETSc's defaults, both costly
Underworld sets
pc_mg_type = "additive"in every GAMG bundle(
petsc_generic_snes_solvers.pyxlines 3260, 4173, 5189, and 6073 for theStokes velocity block). PETSc's default for GAMG is
multiplicative.Additive multigrid is a strictly weaker cycle — the levels do not see each
other's corrections.
And the velocity operator reaches GAMG with block size 1 on a
two-component field. GAMG then aggregates each scalar degree of freedom
independently rather than aggregating nodes, which is the standard way to
cripple algebraic multigrid on a vector problem.
Measured
SolKz, unit box, constant viscosity, Taylor-Hood P2-P1, 11 727 unknowns,
refinement 2 — nothing hard is happening here:
pc_mg_type=multiplicativeSeven times the cycles and 3.7 times the arithmetic, for two settings.
At larger sizes the uncorrected configuration is worse still: at 186 879
unknowns it needs up to 4 136 cycles per velocity solve, which overruns any
reasonable
ksp_max_itand then silently reports a truncated solve.Why it matters beyond performance
GAMG is what every solver falls back to when the mesh carries no refinement
hierarchy, which is the common case. So this is the default path for most
users, and it is slow for reasons that have nothing to do with the method.
It also makes any geometric-versus-algebraic comparison meaningless: a
measurement against this configuration is measuring our own settings.
Suggested fix
Remove
pc_mg_type = "additive"and let PETSc choose. Deviating from a PETScdefault wants a recorded reason, and there is no comment giving one here.
Set the block size on the operator for vector fields so GAMG aggregates nodes.
The matrix is built by PETSc from the DM, so the natural place is wherever the
section is set up rather than after the fact.
Worth checking the other three call sites for the same reasoning, and whether
pc_gamg_agg_nsmooths=2andpc_gamg_repartition=truehave recordedjustifications or are also inherited without one.
Underworld development team with AI support from Claude Code