Add massiv vs accelarate blog code - #11
Conversation
javiert01
left a comment
There was a problem hiding this comment.
Hi team! I am leaving a review with things that may be worth addressing. I used AI assistance but feel free to address/reject the suggestions if they don't make sense (since I don't have enough context). I leave an approval but worth taking a look. Let me know if you have comments! @alexc957 @juliojm13
| bench "Massiv (CPU Parallel)" $ | ||
| nf (M.computeAs M.P . M.map (+1)) massivArr, | ||
| bench "Accelerate (LLVM Native)" $ | ||
| whnf (Native.run . A.map (+1) . A.use) accelArr |
There was a problem hiding this comment.
For lines 75, 92,113:
- Accelerate pays per-iteration AST construction that Massiv doesn't
Native.run . A.map (+1) . A.use rebuilds the expression graph and does a cache lookup on every Criterion iteration. Accelerate's own guidance is to hoist that out with runN/run1:
let mapAccel = Native.runN (A.map (+1)) -- compiled once, outside the timed loop
...
bench "Accelerate (LLVM Native)" $ nf mapAccel accelArrAs written, the Accelerate numbers include front-end overhead that has no Massiv counterpart. For a post whose whole point is a head-to-head comparison, this is the finding I'd fix first.
There was a problem hiding this comment.
A.map (+1) was just a simple example, the blog post is based on the stencil operation
| let mArr = M.makeArray M.Par sz (\(i M.:. j) -> fromIntegral (i + j)) :: M.Array M.P M.Ix2 Double | ||
|
|
||
| -- Accelerate Setup | ||
| let aArr = A.fromList (A.Z A.:. side A.:. side) [0..(fromIntegral (side*side - 1))] :: A.Array A.DIM2 Double | ||
|
|
||
| defaultMain [ | ||
| bgroup "3x3 Mean Blur" [ | ||
| bench "Massiv (Stencil)" $ | ||
| nf (M.computeAs M.P . M.mapStencil (M.Fill 0.0) massivMeanStencil) mArr, | ||
|
|
||
| bench "Accelerate (LLVM Stencil)" $ | ||
| nf (Native.run . A.stencil accelMeanStencil A.clamp . A.use) aArr |
There was a problem hiding this comment.
Two mismatches in the same benchmark:
Input data differs: Massiv gets fromIntegral (i + j), Accelerate gets [0 .. side*side-1].
Border handling differs: M.Fill 0.0 (zero-pad) vs A.clamp (edge-replicate). These are different functions, not two implementations of one.
Pick one semantics and match it — M.Edge on the Massiv side is the equivalent of A.clamp, or use A.function (const 0) to match Fill 0.0. Derive both arrays from a single generator so the data is identical too.
There was a problem hiding this comment.
We care more about how each library uses the arrays not the implementations.
| runAccelOnly = do | ||
| let side = 3162 | ||
| let aArr = A.fromList (A.Z A.:. side A.:. side) [0..(fromIntegral (side*side - 1))] :: A.Array A.DIM2 Double | ||
| defaultMain [ bench "Accelerate (Isolated)" $ whnf (Native.run . A.stencil accelMeanStencil A.clamp . A.use) aArr ] |
There was a problem hiding this comment.
[Main.hs:113][Main.hs:128] uses whnf for the same computation.
The stencil-accel mode exists so you can profile it in isolation and compare against the paired run — but it measures a different amount of work, so the two aren't comparable. Same whnf/nf split appears at [Main.hs:72-75] and [Main.hs:89-92] between the two libraries. Standardize on nf everywhere.
There was a problem hiding this comment.
Accelerate doesn't compile the arrays with Haskell it compiles them with target architecture (CPU/GPU), so nf doesn't apply here but whnf does because it evaluates their constructor.
| let massivArr = M.makeArray M.Par (M.Sz1 size) id :: M.Array M.P M.Ix1 Int | ||
| let accelArr = A.fromList (A.Z A.:. size) [0..size-1] :: A.Vector Int |
There was a problem hiding this comment.
Setup work leaks into the first timed iteration
[Main.hs:67-68][Main.hs:81-85],[Main.hs:102-105]
These are lazy let bindings, so building the 10M-element arrays (including replicate size 2 as a lazy list) happens inside the benchmark loop on first evaluation. Use Criterion's env, which forces setup to NF before timing starts:
main = defaultMain [ env setupStencil $ \ ~(mArr, aArr) -> bgroup "3x3 Mean Blur" [...] ]| source-repository-package | ||
| type: git | ||
| location: https://github.com/AccelerateHS/accelerate.git | ||
| tag: master |
There was a problem hiding this comment.
cabal.project pins tag: master on two git deps
[cabal.project:6][cabal.project:11]
A reader following the blog post in six months gets different Accelerate source than you benchmarked, and quite possibly a build failure (especially combined with the broad allow-newer: *:base). Pin explicit commit SHAs and consider committing a cabal.project.freeze. This is the single biggest barrier to anyone reproducing the results.
adds massv vs accelarate code for Haskell benchmarking.