perf(windows): present the canvas through a DXGI flip-model swap chain - #330
Open
jhodges10 wants to merge 12 commits into
Open
perf(windows): present the canvas through a DXGI flip-model swap chain#330jhodges10 wants to merge 12 commits into
jhodges10 wants to merge 12 commits into
Conversation
|
@jhodges10 is attempting to deploy a commit to the Vercel Labs Team on Vercel. A member of the Team first needs to authorize it. |
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
jhodges10
marked this pull request as draft
August 12, 2026 15:57
jhodges10
force-pushed
the
perf/windows-resize-continuity
branch
from
August 12, 2026 17:22
d7ddfce to
b68fc7c
Compare
Phase 0 of the flip-model migration: no behavior change, just the instruments needed to decide whether the migration is worth doing and to prove it worked afterwards. `ensureTargets` drops the whole per-surface Direct2D bitmap cache on any dimension change, because D2D bitmaps belong to the render target that created them. Every app that runs a hardware canvas on Windows today draws zero bitmaps, so the flush has never cost anything observable and the claim that it is expensive was unmeasured. Three pieces: - `NATIVE_SDK_GPU_PROFILE` names a log path and the renderer emits one `present` and one `paint` line per event. Unset (every shipped run) each probe is one predicted branch and nothing is written. `present` and `paint` become thin wrappers so the accumulators reset and emit on exactly one path each -- between them those two have a dozen refusal returns, and per-return bookkeeping would rot on the first one added. - `tools/gpu-image-fixture/` holds the runtime's entire registered-image ceiling (16 x 512x512 = 16 MiB) on a hardware canvas, which is the worst case reachable through that path. Two tests pin it to `canvas_limits` so the fixture cannot drift below the worst case unnoticed, registered as `test-tool-gpu-image-fixture`. - `tools/windows-truth/gpu-resize-profile.ps1` drives a synthetic resize sweep (and optionally a real modal border drag), then reduces the log. Unlike perf-input.ps1 the sweep needs no interactive scheduled-task hop, because SetWindowPos is not desktop input. Measured at the ceiling on a 240 Hz desktop: image re-upload costs 1.50 ms per resize step, flat across surface size, against a 0.5 ms kill criterion. The same runs also showed `paint()` running 2.0-2.5x per accepted present, so the redundant window blit is ~1.5 ms per step and is paid by every app, not only texture-holding ones. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 1 of the flip-model migration. The device stack exists and is reachable; nothing presents through it yet, so this phase is observably a no-op -- which is the point, because it isolates device creation, adapter selection, and the WARP fallback from the presentation rewrite that follows. - `d2d_factory_` becomes `ID2D1Factory1`. That is the interface that owns `CreateDevice`, and it inherits every geometry and stroke entry point `d2dFactory()` already hands out, so the existing call sites are unchanged. - D3D11 device with BGRA_SUPPORT (mandatory for D2D interop) and SINGLETHREADED (matches the D2D factory and the host's one-UI-thread model). Hardware first, WARP on failure; feature levels 11_1 down to 9_1, with the documented E_INVALIDARG retry for drivers that reject the whole list rather than negotiating down. - The DXGI factory comes from the device's own adapter via GetParent, not from CreateDXGIFactory: a swap chain built by a foreign factory cannot present this device. - One device, one context, owned by the renderer and shared by every surface. D2D contexts are cheap to retarget and expensive to multiply. Created eagerly in `initialize()` so a machine that cannot produce a device at all fails the renderer up front and the runtime takes its software pixel path from the first frame, instead of discovering the problem mid-drag. `NATIVE_SDK_GPU_FORCE_WARP` forces the fallback on a machine that would never otherwise take it. Both paths verified rendering correctly: hardware selects "NVIDIA GeForce RTX 4090" at feature level 11_1, WARP selects "Microsoft Basic Render Driver" at the same level. Also in this commit, because the fixture is what proves the phases: - Fixture cells are now a fixed logical size. They were sized in device pixels against a 125%-scaled display, so the fourth column fell off the surface, dropped out of the display list, and stopped being uploaded -- the fixture was quietly measuring 12 textures, not 16. Fixed cells also keep the display list identical at every window size, so the sweep varies only the backing surface. - The reducer reports `BlitPerStep`, total blit microseconds attributed by seq to resize presents. WM_PAINT coalescing trades paint COUNT against paint SIZE run to run -- the same workload measures anywhere from 0.8 to 2.4 paints per present -- so neither of those halves is quotable on its own. Their product is. Phase 1 baseline at the texture ceiling, unchanged from Phase 0 as intended: image upload 1.513 ms/step, present 2.310 ms, window blit 0.919 ms/step, 3.229 ms total. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 2. The display list now renders into a device-owned `ID2D1Bitmap1` through the shared `ID2D1DeviceContext`, and `paint()` copies that surface into a flip-model swap chain's back buffer and Presents, instead of blitting through an `ID2D1HwndRenderTarget` and the DWM redirection surface. `FLIP_SEQUENTIAL`, not `FLIP_DISCARD`: the incremental path repaints only damaged regions, so undamaged pixels must survive. `SCALING_NONE` so Windows never stretches stale content while a drag outruns the buffer resize. `MakeWindowAssociation(NO_ALT_ENTER)` because the host owns that key. Three things this phase forced that the plan scheduled elsewhere: - **The paint-rect clip had to go.** Under the blt model `paint()` could clip the copy to the update region, because the redirection surface persisted and untouched pixels were last frame's. A flip-model back buffer does not persist: with two buffers the one being drawn holds the frame from TWO presents ago, so a clipped copy leaves a stale alternating image outside the damage. The copy is now full-surface, which is also the precondition Present1 states for dirty rects. - **Phase 5 came along.** `readColorAt` used `ID2D1GdiInteropRenderTarget::GetDC` + `GetPixel`, which is why the backing surface carried GDI_COMPATIBLE. That flag lives on the `CreateCompatibleRenderTarget` call this phase deletes, so the read is now a 1x1 CPU_READ staging bitmap: CopyFromBitmap, Map, read BGRA. Verified on gpu-dashboard (hidden_inset_tall, the only caller): 152 readbacks, every one S_OK, zero fallbacks to the command estimate. - **CopyFromBitmap is not available for the surface copy.** It needs identical D2D pixel formats, and the backing surface is PREMULTIPLIED while a flip-model HWND swap chain's D2D view must be ALPHA_MODE_IGNORE -- E_INVALIDARG on that pair, which first showed up as 158 refused presents. Aligning the backing surface to IGNORE would fix the copy but changes the blend semantics every layer and opacity group renders under, so the copy stays a 1:1 nearest-neighbour DrawBitmap. **This phase is a measured regression, and that is expected.** At the texture ceiling, per resize step: blit 0.919 -> 1.340 ms (copy 1.055, Present 0.285), total 3.229 -> 3.664 ms. gpu-dashboard: 2.402 -> 2.582 ms. The blt model could present a clipped region; the flip model cannot, so option (a) -- keeping WM_PAINT as the presentation trigger -- pays a full-surface copy per paint and buys nothing back on its own. Phase 0 predicted little improvement here; it is worse than that. The win is in Phase 3 (the image-cache flush, 1.68 ms/step) and in presenting once per rendered frame rather than once per WM_PAINT. Both need this phase underneath them. Profiler additions: `present_us` splits Present out of the paint span (it is only 0.14-0.29 ms, so the cost is the copy, not a queue stall), `paint-fail` records the HRESULT and both surface sizes, and `readback` makes the caption sample's success visible instead of silently degrading. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 3, and the reason for the migration. `ensureTargets` no longer calls `releaseImageBitmaps()` on a dimension change. Under the blt model it had to: image bitmaps belonged to the backing render target, that branch recreated the target, so every live texture was re-uploaded from CPU memory on the next frame. They are created from the `ID2D1Device` now, so they outlive a surface resize and `ensureImageBitmap` keeps hitting its cache mid-drag. The two surfaces that genuinely are size-shaped -- the backing bitmap, whose pixel size and DPI are fixed at creation, and the blur snapshot -- still go. Replacement and eviction are untouched: `applyImageActions` still drops a bitmap whose id reconciles to absent, and `ensureImageBitmap` still compares the resource serial, so re-registering an id under new pixels still re-uploads exactly once. At the runtime's 16 MiB registry ceiling, per resize step: textures re-uploaded 16 -> 0 image upload 1.513 -> 0.000 ms display-list render 2.051 -> 0.402 ms (it was mostly uploads) packet present 2.310 -> 0.650 ms Against the pre-migration Phase 1 baseline, total per-step cost across the fixture matrix: 16 x 512 (16 MiB) 3.229 -> 2.201 ms -32% 8 x 512 (8 MiB) 2.663 -> 2.086 ms -22% 16 x 256 (4 MiB) 2.407 -> 2.016 ms -16% 16 x 128 (1 MiB) 1.997 -> 1.977 ms -1% 1 x 8 (~0) 1.742 -> 2.029 ms +16% gpu-dashboard 2.402 -> 2.759 ms +15% Zero refusals in every run, and the textures stay visually correct through a full sweep. The shape of that table is the honest result: the migration pays for itself where there are textures to keep, and the two texture-light rows are still carrying Phase 2's full-surface copy, which bought nothing on its own. Window blit is now 66-79% of a resize step everywhere, so it is the whole remaining target -- Phase 4. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 4. Two changes that only make sense together.
**Damage-accumulated partial copy.** Phase 2 had to copy the whole
backing surface into the back buffer on every paint, because a
flip-model buffer does not persist the way the DWM redirection surface
did: with `BufferCount = 2` the buffer about to be drawn is the one
presented TWO frames ago. So the region that must be refreshed is not
this paint's damage, it is
damage(this paint) UNION damage(previous paint)
-- everything that changed since the pixels currently in this buffer
were correct. Copy that and the buffer equals the current frame again.
`swap_history_valid_` guards the cases where no history exists at all
(fresh swap chain, ResizeBuffers, device loss) and forces a full copy.
**Dirty-rect Present1.** With the buffer now provably equal to the
current frame, `Present1` can be told what changed against the
PREVIOUSLY PRESENTED frame -- this paint's damage alone, which is a
different set from the copied union. That is the precondition the API
states, and it is why the two changes could not land separately.
Beyond `kSwapDirtyRectCap` combined rectangles the per-rect clipped
copies stop beating one full pass, and the paint falls back.
One trap worth recording: the fallback was originally also gated on the
backing and swap surfaces having identical pixel sizes. They almost
never do. The backing surface is sized from the packet
(`ceil(logical * scale)`) and the swap chain from `GetClientRect`, and at
125% scaling those come out 1349x895 against 1348x894. That single pixel
disabled the partial path on essentially every frame. Size agreement was
never the right question -- a partial copy draws the same scaled image
the full copy would, just clipped -- so only the sampling mode consults
it now.
Measured on gpu-dashboard, mid-sweep, localized damage:
full copy 527 us
partial copy 52 us 10x
Measured on gpu-dashboard held still, pointer moving: **no difference**
(0.114 ms partial vs 0.103 ms full). The new `dmg_px` field says why --
damage coverage is 162% of the surface, so that app repaints everything
on every hover and there is nothing for dirty rects to save. The
mechanism is correct; the workload has no locality. That distinction is
exactly what `DamageCoveragePct` exists to make visible, because a
correct optimization and a broken one otherwise look identical.
Also adds `NATIVE_SDK_GPU_FULL_PRESENT=1` to pin every paint to the full
path, so the partial path can be A/B'd against itself on one build and
one workload, and `-HoldMs` to the harness, which holds the window still
and jiggles the pointer -- resize steps are all full-surface repaints and
say nothing about partial-update cost.
Resize-step cost is unchanged from Phase 3 (2.19 ms at the texture
ceiling, zero refusals), as expected: every resize step invalidates
everything.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Phase 6. Before the migration, a lost Direct2D target invalidated one surface's backing bitmap and nothing else. Now a removed D3D device invalidates everything every surface owns -- backing bitmap, swap chain, readback bitmap, blur snapshot, and the texture cache that Phase 3 made worth keeping. The renderer carries a `device_generation_`, bumped every time the stack is built. Surfaces stamp the generation they built against and call `syncDevice()` before any frame work; a mismatch means their resources belong to a dead device and can only be released. That is how a loss reaches sibling surfaces without the renderer keeping a registry of them, and it means recovery is driven by the next frame rather than by a callback into unknown state. `GpuRendererImpl::deviceLost()` names the HRESULTs that mean the device is gone -- DEVICE_REMOVED, DEVICE_RESET, DEVICE_HUNG, DRIVER_INTERNAL_ERROR -- and deliberately separates them from D2DERR_RECREATE_TARGET, which means only the target went and the device is still good. `handleDeviceLoss()` is the single exit for all of them: rebuild the shared stack, clear this surface, return false so the host sets `gpu_force_full_repaint_pending` and the runtime resends a full packet. That existing recovery hook needed no changes. `recoverDeviceStack()` records `GetDeviceRemovedReason()` before releasing, so the log says why. A machine that can no longer produce any device fails honestly and leaves the runtime on its software path. `NATIVE_SDK_GPU_SIMULATE_DEVICE_LOSS=<n>` takes the recovery path on the nth paint. Disabling an adapter mid-drag is the more honest test but not a repeatable one; this drives the same code on demand. Verified: the fixture loses its device mid-sweep at the texture ceiling, logs `device-lost reason=0x00000000 recovered=1 generation=2`, rebuilds, and finishes the sweep with 0 refusals, 0 re-uploads, and correct pixels. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Windows renderer is pinned by source-text assertions in root.zig, and the migration moved the text out from under three of them. Two were mechanical: `backing_target_->` became `ctx()->` when the render path moved onto the shared device context. The third is worth naming. The GDI pin asserted that `D2D1_COMPATIBLE_RENDER_TARGET_OPTIONS_GDI_COMPATIBLE` appears in the renderer, and it still passed after Phase 2 deleted the flag -- because the comment explaining why the flag is gone contains the constant. A pin that matches prose is not a pin. It is replaced by assertions on what actually guarantees the behaviour that mattered: the CPU_READ staging bitmap and its Map, i.e. that the caption sample still reads a real pixel rather than the retained-command estimate. New test for the two rules that are subtle enough to be re-broken by someone doing something reasonable: - FLIP_SEQUENTIAL, never FLIP_DISCARD. The incremental path copies undamaged content forward, so a buffer DXGI may discard corrupts patch frames intermittently, on real hardware only. - A partial copy refreshes this paint's damage AND the previous paint's, while Present1's dirty rects carry this paint's alone. Those are different sets for a non-obvious reason (BufferCount = 2 means the buffer being drawn was presented two frames ago), which is exactly the kind of thing that gets "simplified" later. The absence assertion matches `DXGI_SWAP_EFFECT_FLIP_DISCARD`, not `FLIP_DISCARD`: the first draft of this test failed on its own comment, which is the mirror image of the bug it replaces. desktop-platform-tests: 209 pass, 1 skip, 0 fail. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Found by pointing the profiler at a real app instead of a fixture. `seq` counts presents PER SURFACE. Every vehicle used so far has exactly one gpu_surface, so seq happened to be a unique key and the reducer grouped paints to presents by it. A video editor has eighteen. Sequence numbers collide across them immediately, so paints were attributed to whichever surface happened to share a number -- the reducer reported 214 paints per present and a 22 ms per-step blit, both meaningless. Every line now carries `surface=<n>`, and the reducer keys on (surface, seq). The same run then reads 18 surfaces, ~2150 paints, and a mean blit of 261 us. Also adds `-SettleMs` to the harness. It waited 1.5 s before sweeping, which is fine for a fixture and far too short for an app that restores a 4K project on launch: the first two editor runs swept an empty window and recorded zero registered textures and zero flushes -- indistinguishable in the log from an app that has none. At 15 s the same runs show the flushes that are actually there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ResizeBuffers was running on every step of a resize drag. It is the most
expensive single thing in a resize frame: it frees and reallocates both
back buffers, tears down and rebuilds the D2D view of buffer 0,
re-associates the chain with the window, and leaves the buffers holding
undefined pixels so the next copy owes the whole surface.
Round the ALLOCATION up to a 128 px grid instead. DXGI_SCALING_NONE was
already the swap chain's scaling mode, and it aligns the buffer's
top-left with the window's and CLIPS rather than stretches -- so a buffer
larger than the window shows exactly the window-sized top-left crop,
which is where this renderer draws. Reallocate only when the window
outgrows the allocation, or shrinks past half of it; a shrinking drag
must not reallocate at every grid line it crosses on the way down.
Measured on tools/gpu-image-fixture at the registered-image ceiling,
three runs each way:
buffer allocations 82 -> 13 (distinct client sizes vs. reallocations)
blit per resize step 1.79 ms -> 0.42 ms
median blit 1.06 ms -> 0.15 ms
p90 blit 1.30 ms -> 0.31 ms
Present per step 0.41 ms -> 0.14 ms
The p90 is the interesting one: the tail this removes is the same one the
flip-model migration introduced, and it was the strongest argument
against shipping that work.
Deliberately NOT via IDXGISwapChain2::SetSourceSize, which is the
interface DXGI documents for exactly this ("an effective resize without
calling the more-expensive ResizeBuffers"). Paired with SCALING_NONE it
renders wrong: every surface whose window is much shorter than its
rounded-up buffer -- a 38 px panel header rounded up to 128 px -- came
out as a small fragment at the top-left on a field of background colour.
In the editor that was the app menu bar, every panel header strip, the
status bar and the dock dividers, all blank, while the taller panels
looked fine. It also measured as worth nothing next to the plain crop:
0.428 ms against 0.426 ms per resize step. The pin in root.zig matches
the call rather than the type name, so the paragraph explaining its
absence cannot satisfy its own test.
NATIVE_SDK_GPU_SWAP_GRANULARITY=<n> overrides the grid (1 disables it).
That switch is what separated over-allocating from what was done with the
over-allocation, and it stays as the escape hatch if a driver ever
disagrees about SCALING_NONE.
Two supporting changes fall out. Damage and dirty rects are now clamped
to the PRESENTED extent rather than the buffer's pixel size -- an
over-allocated buffer is larger than the window, and a dirty rect outside
the presented region is not a valid one. And SetBackgroundColor gets the
app's own clear colour, so the band DXGI fills while a growing drag
outruns the next present is the colour that band is about to be painted.
Verified against alchemist-editor's screenshot goldens: all 7 aux windows
byte-identical, and still byte-identical with the grid forced to 1024 px,
which is an 8x over-allocation on every one of them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two gaps the editor measurements walked into. An app that stops on a startup dialog measures the dialog. -AppArgs lets a before/after pair launch alchemist-native with --restore-session so the same project comes back every time with nothing to click. And the sweep/drag split is a single global sequence number, while sequence numbers are per surface — fine for the one-surface fixture, meaningless for an app with twenty, where the boundary lands somewhere arbitrary inside the sweep. -SkipSweep runs the drag alone so the whole logged population is the drag and no boundary is needed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
jhodges10
force-pushed
the
perf/windows-resize-continuity
branch
from
August 12, 2026 18:16
b68fc7c to
554c62b
Compare
jhodges10
marked this pull request as ready for review
August 13, 2026 06:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Warning
Known regression, unfixed — do not merge yet. On an app with an extended/custom titlebar (
titlebar = "hidden_inset_tall"), the window's caption buttons stop being drawn. Reproduced onexamples/gpu-dashboardat natural startup size with no resize involved;examples/voice-memo(standard caption) is pixel-identical between builds, so it is scoped to the extended-frame path. Working diagnosis: the flip-model swap chain covers non-client pixels the blt path left to DWM.Twelve commits, one idea each, moving the Windows canvas from a
CreateCompatibleRenderTargetblt path to a D3D11/D2D1.1 device presenting through a DXGI flip-model swap chain. Renderer-side only. Based onmain.The host-scheduling fixes that used to ride in this PR now live in #323 — they are what makes a resize deliver frames at all, they carry no renderer risk, and they should not be blocked by the regression above.
Why
The texture cache did not survive a resize.
ensureTargetsrecreated the backing render target on any pixel-size, logical-size, or scale change, and Direct2D bitmaps belong to the target that created them — so every resize step dropped the whole per-surfaceimage_bitmaps_map and the next display-list walk re-uploaded every texture it drew. At the runtime's registered-image ceiling that is 1.44 ms of pure re-upload per resize step, and it scales with texture bytes, not window area.Every hardware canvas paid for a window blit. The blt model copies the render target into the window's redirection surface on each present; the flip model hands the buffer to DWM.
What is in it
3c2851bbtools/gpu-image-fixture— the only vehicle that can reach the texture ceiling, because every showcase app on Windows draws zero bitmaps.34350eca845e90a6108a695a61df4d6f9893ab52f3525337e2dfa2664a2c0a1eSCALING_NONEcrop them, so a drag stops callingResizeBuffersevery step.b74ee5cd,29af3930,554c62bad3d11/dxgiwhere the renderer now needs them.readColorAthad to move with the target: it read pixels throughID2D1GdiInteropRenderTarget::GetDC, which requires the backing surface to beGDI_COMPATIBLE, and that flag lives on theCreateCompatibleRenderTargetcall this deletes.Results
Fixture at the 16 MiB texture ceiling, three runs each way, per resize step:
Two things worth a reviewer's attention
The swap chain alone is a regression, and structurally so. Reading
845e90a6's numbers in isolation reads as a failed migration; it is the setup for108a695a, which is where the win is.SetSourceSizeis deliberately absent. DXGI documents it as "an effective resize without calling the more-expensiveResizeBuffers", which is exactly this job — but paired withDXGI_SCALING_NONEit renders a surface whose buffer is much taller than its window (a 38 px header rounded up to a 128 px buffer) as a fragment at the top-left on a field of background colour. It also measured as worth nothing: 0.428 ms against 0.426 ms per step. A test pins the call absent, not the type name, so the comment explaining why cannot satisfy its own pin.Validation
zig build test-desktop-platform— 211 pass, 1 skip, 0 fail on this branch standalone.🤖 Generated with Claude Code