perf(gc): stop the globalThis bootstrap from disabling the per-object layout fast path

changelog.d/7809-globalthis-bootstrap-layout-latch.md

@@ -0,0 +1,100 @@
+### Fixed
+
+- **Touching `globalThis` no longer disables the per-object GC layout fast path
+  for the rest of the process.** `churn` and `tree` were paying +28% / +29% for
+  a single `for…of` in `main()`, and every real TypeScript program was paying it
+  too.
+
+  `globalThis` is populated lazily on first touch, and *any* plain-object or
+  array property **miss** forces it: the miss walks the prototype chain, reaches
+  `builtin_prototype_value` → `js_get_global_this_builtin_value`, and runs the
+  several-hundred-builtin bootstrap. Perry's whole benchmark corpus happens
+  never to take that path — no `for…of`, no spread, no `Symbol`, no property
+  miss anywhere in `churn`/`tree`/`interp`/`shapes`/`asyncpipe`/`retain` — so
+  the cost was invisible to every number in the perf campaign while real
+  programs paid it before their first line of work.
+
+  It was **not** the ~1.15 MB the bootstrap allocates. GC behaviour is
+  effectively identical with and without it: 105 minors on `churn` either way,
+  and ~616 KB more copied across the entire run. The cost was a **global latch**.
+  The bootstrap builds hundreds of permanently-rooted plain objects, and each
+  one's first pointer field minted an entry in the per-object GC slot-layout
+  side tables. Those entries are immortal, so `PER_OBJECT_LAYOUTS_NONEMPTY` —
+  the emptiness proof that keeps `layout_forget_object` off the allocation,
+  death and relocation paths — could never go `false` again. Measured
+  `layout_forget_object` self time: `churn` 112 ms → 916 ms, `tree`
+  194 ms → 740 ms, which is essentially the whole regression in both.
+
+  This is #7510's lesson at 1000× the scale ("one immortal entry is enough to
+  nullify an is-empty accelerator"; there it was a single interned keys array).
+  Two changes, and **both are needed**:
+
+  1. `gc::ImmortalLayoutScope` around `populate_global_this_builtins`. Inside
+     it, an object that would mint a per-object pointer mask declares
+     `GC_LAYOUT_UNKNOWN` instead — the tag-checked payload scan, which is the
+     code's own fallback for the same situation and the universally safe state,
+     not a weaker one. For an object that is never reclaimed the mask bought
+     precision nobody spends, at the price of two `RefCell` round-trips and two
+     hash probes on every allocation the program would ever make. Bootstrap
+     residue: **1113 entries → 0**.
+
+     Deliberately **not** applied to typed-shape layouts
+     (`init_typed_shape_layout`): those describe raw-f64 slots, whose bit
+     patterns can alias a heap pointer, and a conservative scan would trace —
+     and under the copying collector *rewrite* — a slot holding a number. The
+     scope applies only where the mask being replaced is itself derived from
+     `layout_pointer_bearing_bits`, i.e. exactly the test `GC_LAYOUT_UNKNOWN`
+     re-runs per slot.
+
+  2. An **address filter** replacing `PER_OBJECT_LAYOUTS_NONEMPTY` as the hot
+     guard. Change 1 alone moved nothing measurable, and that is the important
+     finding: ordinary runtime init still leaves one or two long-lived records
+     behind, and for a single global bit two entries are exactly as bad as 1113.
+     An 8192-bit thread-local filter over the key addresses turns "is either
+     table empty?" into "can this *address* have an entry?", so a nursery
+     address the tables have never seen is proved absent in one multiply and one
+     load even while immortal records exist elsewhere.
+
+     The filter sits *behind* the flag, and both live in ONE thread-local
+     (`PerObjectLayoutHint`). All three arrangements were measured on the quiet
+     mini; the co-located one wins everywhere:
+
+     | | `churn` | `push_cls` | `tree` | `interp` | `churn`+`for…of` | `tree`+`for…of` |
+     |---|--:|--:|--:|--:|--:|--:|
+     | base | 0.422 | 0.356 | 1.627 | 1.888 | 0.539 | 2.151 |
+     | filter only (no flag) | 0.438 | **0.383** | 1.673 | 1.934 | 0.500 | 1.857 |
+     | flag + filter, 2 slots | 0.421 | 0.368 | 1.642 | 1.950 | 0.506 | 1.886 |
+     | **flag + filter, 1 slot** | **0.422** | **0.368** | **1.640** | **1.922** | **0.493** | **1.840** |
+
+     Dropping the flag is a loss: almost every workload is *disarmed*, and for
+     those the flag is one load where the filter is a multiply, a shift, a load
+     and a test — `push_cls` went past its budget. Keeping both as separate
+     thread-locals costs a second `_tlv_get_addr` on exactly the workloads that
+     are legitimately armed (`interp`, `iso_miss`). One struct behind the
+     existing named hot slot gives the cheap gate AND one resolution.
+
+     `false` is a proof of absence and nothing else rests on it; the filter is
+     rebuilt from the live keys once half its bits are set, so a workload that
+     genuinely churns per-object records cannot saturate it permanently.
+
+  Measured on the quiet mini (base = `b9415d780`, both arms built locally;
+  interleaved, best-of-5, exit-checked): `churn` + `for…of` **0.539 → 0.493**
+  (floor 0.422), `tree` + `for…of` **2.151 → 1.840** (floor 1.627). Every
+  protected bench stays inside budget. `interp` 1.888 → 1.922 and `iso_miss`
+  2.361 → 2.443 still pay the filter test without benefiting from it — they are
+  legitimately armed, so it never proves absence for them.
+
+  Gated by five tests in `gc::tests::layout_trace::per_object_tables`, written
+  so none of them can pass vacuously: the bootstrap must leave both tables empty
+  (with a subject-live check that `globalThis.Array` actually populated); the
+  same store *outside* a scope must still mint a mask; an object built inside a
+  scope must still trace its children through the fallback scan; a live record
+  must survive a filter rebuild and still be found and removed; and — the
+  accelerator's own subject-live assertion — the filter must still prove
+  unrelated addresses absent *while the global flag is armed*, which is the
+  exact condition under which it silently stopped accelerating before.
+
+  `PERRY_GC_DIAG=1` now prints
+  `[gc-globalthis-bootstrap] elapsed_us=… per_object_slot_masks=… per_object_typed_layouts=…`
+  once per thread, so the bootstrap's cost and its residue are observable rather
+  than inferred.

crates/perry-runtime/src/gc/hot_tls.rs

@@ -24,7 +24,9 @@ use super::barrier::{
     GC_BIRTH_EXTRA_FLAGS, INCREMENTAL_MARK_BARRIER_MINOR_ONLY, INCREMENTAL_MARK_BARRIER_VALID_PTRS,
 };
 use super::layout::{LayoutSlotMask, TypedLayoutDescriptor, SHAPE_LAYOUTS};
-use super::layout_tables::{LAYOUT_SLOT_MASKS, PER_OBJECT_LAYOUTS_NONEMPTY, TYPED_LAYOUTS};
+use super::layout_tables::{
+    PerObjectLayoutHint, LAYOUT_SLOT_MASKS, PER_OBJECT_LAYOUTS_NONEMPTY, TYPED_LAYOUTS,
+};
 use super::malloc::{ARENA_FREE_LIST, ARENA_FREE_LIST_NONEMPTY};
 use super::trace::ValidPointerSet;
 use std::cell::{Cell, RefCell};
@@ -120,9 +122,9 @@ pub(super) fn hot_shape_layouts() -> &'static RefCell<ShapeLayoutMap> {
 /// per-object layout record at all" question the allocation, store, death and
 /// trace paths all ask (#7510).
 #[inline(always)]
-pub(super) fn hot_per_object_layouts_nonempty() -> &'static Cell<bool> {
+pub(super) fn hot_per_object_layout_hint() -> &'static PerObjectLayoutHint {
     // SAFETY: paired with `per_object_layouts_nonempty_hot_addr` above.
-    unsafe { &*(crate::tls_hot::hot().per_object_layouts_nonempty as *const Cell<bool>) }
+    unsafe { &*(crate::tls_hot::hot().per_object_layouts_nonempty as *const PerObjectLayoutHint) }
 }
 
 // --- gc::malloc -------------------------------------------------------------

crates/perry-runtime/src/gc/layout.rs

@@ -909,11 +909,27 @@ pub(crate) fn layout_note_slot(parent_user: usize, slot_index: usize, value_bits
                         set_layout_state(header, GC_LAYOUT_SIDE_MASK);
                     }
                 } else if (*header)._reserved & GC_LAYOUT_STATE_MASK == GC_LAYOUT_POINTER_FREE {
-                    let mut mask = LayoutSlotMask::Inline(0);
-                    mask.set_slot(slot_index);
-                    masks.insert(parent_user, mask);
-                    mark_per_object_layouts_nonempty();
-                    set_layout_state(header, GC_LAYOUT_SIDE_MASK);
+                    if super::layout_tables::immortal_layout_scope_active() {
+                        // An object built inside an `ImmortalLayoutScope` is
+                        // rooted for the life of the process, so the entry it
+                        // would mint here is never removed — and one such
+                        // entry disables `PER_OBJECT_LAYOUTS_NONEMPTY` for
+                        // every allocation the program will ever make. Take
+                        // the same `GC_LAYOUT_UNKNOWN` fallback the `else`
+                        // arm below uses for this exact situation; see
+                        // `ImmortalLayoutScope` for why that is the safe
+                        // state and not a weaker one.
+                        set_layout_state(header, GC_LAYOUT_UNKNOWN);
+                    } else {
+                        let mut mask = LayoutSlotMask::Inline(0);
+                        mask.set_slot(slot_index);
+                        masks.insert(parent_user, mask);
+                        mark_per_object_layouts_nonempty();
+                        // The one insert site that holds its own `borrow_mut`,
+                        // so it maintains the address filter inline too.
+                        super::layout_tables::layout_addr_filter_note(parent_user);
+                        set_layout_state(header, GC_LAYOUT_SIDE_MASK);
+                    }
                 } else {
                     set_layout_state(header, GC_LAYOUT_UNKNOWN);
                 }
@@ -1338,6 +1354,18 @@ pub(super) unsafe fn layout_rebuild_from_slots_with_policy(
     if mask.is_empty() {
         set_layout_state(header, GC_LAYOUT_POINTER_FREE);
         slot_masks_remove(user_ptr as usize);
+    } else if super::layout_tables::immortal_layout_scope_active() {
+        // Same reasoning as the `layout_note_slot` branch: an object built
+        // inside an `ImmortalLayoutScope` never dies, so the mask it would
+        // install here is a permanent tenant of a side table whose emptiness
+        // is a process-wide fast path. Falling back to the tag-checked scan is
+        // sound *for this rebuild specifically* because the mask above is
+        // itself derived from `layout_pointer_bearing_bits` — exactly the test
+        // `GC_LAYOUT_UNKNOWN` re-runs per slot. (This is why the scope may not
+        // be applied to a TYPED descriptor, whose raw-f64 slots the tag test
+        // would misread; see `ImmortalLayoutScope`.)
+        set_layout_state(header, GC_LAYOUT_UNKNOWN);
+        slot_masks_remove(user_ptr as usize);
     } else {
         set_layout_state(header, GC_LAYOUT_SIDE_MASK);
         slot_masks_insert(user_ptr as usize, mask);