Async liveness: bounded write parks, a classified accept loop, resolv.conf search/ndots

bin/.gitignore

@@ -1,6 +1,12 @@
 # Self-compile artifacts. Only the driver script is checked in.
+#
+# The wildcards matter: tools/selfhost.sh writes manticore_self{,.ll,_stubs.c}
+# here, and a 954k-line .ll that is neither tracked nor ignored rides into the
+# next `git add -A` — which is exactly how it got committed once.
 manticore
 manticore.ll
 manticore.o
 manticore_stubs.c
 manticore_stubs.o
+manticore_self*
+manticore_g[0-9]*

docs/ROADMAP.md

@@ -12,25 +12,27 @@ _Last updated: 2026-07-15 · branch `main` · HEAD `a6e062a`._
 Pure-PHP, self-hosting PHP→native AOT compiler. The runtime is emitted as LLVM IR from PHP; `bin/manticore` compiles its
 own `src/`.
 
-**Gates (all green):**
-- `tests/aot/run.sh` — **467/467**
-- `tools/selfhost_fixpoint.sh` — fixpoint byte-identical · self-host · stability 5×2
-- `tools/difftest.sh` — **458 match / 0 diff** vs PHP 8.5.8
+**Gates (all green, macOS + Linux):** `tests/aot/run.sh` · `tools/difftest.sh` (0 diff vs
+PHP 8.5.8) · `tools/selfhost_fixpoint.sh` (fixpoint byte-identical · self-host · MIR golden ·
+stability 5×2) · `tools/docker/run_tests.sh --gate`. Counts move every session — run them
+rather than trusting a number written here.
+
+⚠ **`bin/build` green says nothing about `tools/selfhost.sh`.** The manifest build compiles
+`src/Runtime` as a LIBRARY with a flattened namespace; the self-host path takes everything as
+ONE module. They diverge on emitted symbol names, and only the stability gate covers the
+second. Corollary: **never ship a compiler fix together with tree code that needs that fix** —
+the previous generation then cannot build the tree at all, and only a cold seed recovers.
 
 Build: `bin/build --seed` (cold Zend bootstrap → native) / `bin/build`
 (self-host). See `.claude/CLAUDE.md` for the pipeline and key files.
 
-**~228 stdlib functions** implemented (array/string/type/math/ctype/`preg_*`/JSON/
-var/SPL/date/IO); see the README's Standard library table.
-
-**Recently completed** (2026-07): full `preg_*` family over host PCRE2 +
-`#[RefOut]` out-param auto-vivification · Monomorphize **callable dimension**
-(specialize callback-takers per concrete closure) · **de-cellify** at the
-concrete-array ← cell-array store boundary — fixes `uasort` with any comparator
-and dents the erased-array representation root · Ryu float formatting · JSON
-decode 3.3× + default-flag escaping · assoc key-leak fix (malloc −97%) ·
-reified-class generics · pin elimination (278→0). Detail in the session memory
-files under `.claude/.../memory/`.
+**Recently completed** (2026-07): the async runtime — structured concurrency, transparent
+netpoller I/O, and the liveness pass over it (bounded write parks, a classified `accept(2)`
+loop that backs off on EMFILE instead of spinning, resolv.conf `search`/`ndots`) ·
+`docs/superset.md`, the catalogue of everything with no Zend oracle · the full `preg_*` family
+over host PCRE2 + `#[RefOut]` · Monomorphize **callable dimension** · **de-cellify** at the
+concrete-array ← cell-array store boundary · Ryu float formatting · reified-class generics ·
+pin elimination. Detail in the session memory files under `.claude/.../memory/`.
 
 ## The gap matrix (probed, with repros)
 

docs/async.md

@@ -13,6 +13,11 @@ install, nothing to link — `use function Async\spawn;` is enough.
 
 This directory holds the examples and benchmarks.
 
+Everything here is **superset**: `php` has `Fiber` and nothing else of it, so `difftest` cannot
+check a single line — these cases carry hand-written expected output instead.
+[docs/superset.md](superset.md) catalogues that whole surface (concurrency, attributes, FFI,
+modules, types, memory) and the rules it lives under.
+
 ## Model (not Promises)
 
 The concurrency *model* is Go's — cheap tasks, channels, a netpoller, blocking-*looking* I/O
@@ -265,6 +270,34 @@ $sem = new Async\Semaphore(4);
 $sem->withPermit(fn() => work());
 ```
 
+**An accept loop needs one.** It is the one place a scheduler will happily let you
+spawn without bound: `while (true) { $g->spawn(serve(accept())); }` turns a
+connection burst into unbounded tasks, fds and memory, and the first thing to fail is
+something unrelated. Take the permit **before** the accept — then the queue stays in
+the kernel backlog, where it belongs, instead of in your heap.
+
+```php
+$gate = new Async\Semaphore(256);
+Async\group(function (Async\TaskGroup $g) use ($server, $gate) {
+    while (true) {
+        $gate->acquire();                                  // BEFORE the accept
+        $conn = stream_socket_accept($server);
+        if ($conn === false) { $gate->release(); continue; }
+        $g->spawn(function () use ($conn, $gate) {
+            try { serve($conn); } finally { $gate->release(); }   // in the CHILD
+        });
+    }
+});
+```
+
+`Async\stats()['live']` is the gauge to alert on, and `Async\dump()` names the tasks
+when it climbs. There is deliberately **no built-in ceiling on tasks per scope**:
+`spawn()` is the wrong place to fail (the exception would land in the acceptor, the
+one task that has to survive an overload), a hard error turns overload into a crash
+instead of flow control, and `live` counts timers and reporters that share nothing
+with the work being limited. The policy — park, shed, or answer 503 — belongs to the
+application, and a Semaphore owned by the scope that owns the work already expresses it.
+
 ### Scoped values
 
 ```php
@@ -282,15 +315,32 @@ Ordinary stream calls suspend the fiber instead of the process when a scheduler
 `connect(2)` runs non-blocking, BOTH TLS handshake directions are driven through
 `WANT_READ`/`WANT_WRITE` parks (client `SSL_connect` and server `SSL_accept` — so a TLS
 server serves concurrent clients), and a hostname is resolved over the netpoller:
-`/etc/hosts`, a per-run cache held by the scheduler, then A and AAAA queries across
-**every** nameserver in `resolv.conf` (two attempts each, 2 s apiece, TC → retry over
-TCP), falling back to the blocking `getaddrinfo` walk when none of that can answer.
+`/etc/hosts`, a per-run cache held by the scheduler, the `search` list and `ndots` rule from
+`resolv.conf` (so `db`, `redis`, `service.namespace` — the names compose and kubernetes hand
+you — resolve here instead of falling through to the blocking walk), then A and AAAA queries
+across **every** nameserver (two attempts each, 2 s apiece, TC → retry over TCP), falling
+back to the blocking `getaddrinfo` walk when none of that can answer. A search walk stops the
+moment NO server answers: a dead resolver must not be multiplied by the suffix count.
 Two spawned HTTPS fetches to DIFFERENT hosts run 0.17s vs 0.34s sequential.
 
-`stream_set_timeout()` and `stream_socket_accept($srv, $timeout)` are honoured under the
-scheduler: the wait is BOUNDED, so a hung peer no longer wedges the fiber forever (an
-unbounded park was the old behaviour, and a timeout used to fall back to a blocking `poll(2)`
-that stalled every other task).
+**Every wait is bounded.** `stream_set_timeout()` is the STREAM's timeout, as in php — reads
+*and* writes — and `stream_socket_accept($srv, $timeout)` is honoured under the scheduler.
+On expiry the operation reports a short read/write with `stream_get_meta_data()['timed_out']`
+true, never an exception. Unbounded parks were the old behaviour on both sides, and each was
+a liveness hole rather than a missing feature: a peer that stops reading wedged the writing
+fiber forever — the task never settled, its scope never closed, the fd was never released —
+and a timeout used to fall back to a blocking `poll(2)` that stalled every other task. The
+default when nothing is set is 60 s, php's `default_socket_timeout`.
+
+**`accept(2)` failures are classified.** A would-block parks; a peer that vanished between its
+SYN and our accept (`ECONNABORTED`/`EPROTO`/`EINTR`) retries immediately, since no readiness
+edge is coming for a connection that is already gone; resource exhaustion
+(`EMFILE`/`ENFILE`/`ENOBUFS`/`ENOMEM`) backs off on a timer and keeps serving what is already
+open; anything else is reported. The overload case is why this matters: `accept(2)` fails
+while the pending connection **stays queued**, so a level-triggered listener stays readable —
+re-arming readiness is a hot spin that starves every sibling task and never even trips the
+watchdog, because it suspends on every iteration. It shows up only as `stats()['wakes']`
+climbing with wall time instead of with connections.
 
 The seam is `\Runtime\AsyncHook` (five callbacks installed by the scheduler — readable,
 writable, close, bounded-readable, sleep; one null check per would-block when no scheduler is
@@ -385,10 +435,13 @@ is still there.
 
 ## Not yet
 
-Also: `writev`/`io_uring` to break the 2-syscall floor · DNS search-domain/`ndots` handling
-(a name needing a suffix falls back to the blocking walk today) · off-thread file I/O ·
-shared-memory multithreading (a future compiler superset). See the async roadmap memory for
-the plan.
+Also: `writev`/`io_uring` to break the 2-syscall floor · `resolv.conf`'s `timeout:`/`attempts:`
+options (the search list and `ndots` are in; those two still hard-code 2 s × 2) · the search
+list applied to `/etc/hosts` the way glibc does · off-thread file I/O · shared-memory
+multithreading (a future compiler superset) · **fiber stack sizing**: 8 MiB of *virtual*
+address space per task with a `PROT_NONE` guard page, pooled and reclaimed promptly, but
+nobody has measured where 10 000 concurrent tasks actually hits `vm.max_map_count` or a
+container's `ulimit -v`. See the async roadmap memory for the plan.
 
 **`#[Async]`** — an attribute that turns a call into a spawned `Task` of the function's
 return type. The only piece here that cannot be a library: it needs the compiler to split