diff --git a/README.md b/README.md index 5af1514..bd698d4 100644 --- a/README.md +++ b/README.md @@ -1,257 +1,219 @@ # Manticore Self-hosted PHP-to-native AOT compiler. Compiles a large subset of PHP 8.5+ to -standalone native binaries (arm64 / x86_64) through LLVM IR — no PHP runtime, -no shared libraries beyond libc. **The compiler is written in PHP and compiles -itself to a byte-identical fixpoint.** - -## Rationale - -`If you only knew the power of the dark side.` - -Manticore is a proof-of-concept project that aims to demonstrate the -feasibility of creating a self-hosted PHP compiler. By writing the compiler in -PHP and compiling it to a byte-identical fixpoint, Manticore showcases the -potential for creating a fully self-contained PHP runtime environment. - -## Status - -- ✅ **Self-host fixpoint holds.** `manticore build manticore.json` rebuilds the - compiler from PHP source; gen2 and gen3 emit byte-identical IR. -- ✅ **AOT suite 467/467** green (`tests/aot/`). -- ✅ **Differential parity 458/0** vs the Zend `php` interpreter (PHP 8.5.8) — - manticore output matches the reference on every plain-runnable case. -- ✅ **~228 standard-library functions** implemented (array 35, string 43, type - 30, math 28, ctype 11, `preg_*` 10, plus JSON / var-dump / SPL / date / I/O) — - each as a PHP-level stdlib function, an injected prelude helper, or an inlined - codegen builtin. See [Standard library](#standard-library). -- ✅ **Rebuild-stable** — 5×2 cold/self rebuilds, every binary smoke-clean (no - build-to-build heap-layout roulette). -- ✅ **Faster than Zend on every benchmark** — up to **44×** on compute/algorithms, - 1.5–8× on the stdlib-bound tail, ~24× faster cold start — see - [Benchmarks](#benchmarks). -- ✅ Refcount + copy-on-write (assoc + objects) and a Bacon–Rajan cycle - collector v1 (opt-in, zero-overhead unless `gc_collect_cycles()` is reached). -- ✅ Cargo-like module system: `manticore.json`, `.sig` module interfaces, - prebuilt stdlib object, distributable compiler (`bin/` + `lib/`, no sources). -- ✅ Extension system (MVP): manifest-declared native-library bindings — - glue compiled into the app + `-l` linked (proof: `zlib`/`crc32`). Real - extensions (curl / xml / pdo) build on the same mechanism. -- ✅ **Broad PHP 8.5 surface.** Classes / interfaces / traits / enums (incl. - enum methods + constants and interface-implementing enums), abstract + - anonymous classes, late static binding (`new static`, `static::method()`, - `parent::`/`self::` forwarding), magic methods - (`__get`/`__set`/`__call`/`__invoke`/`__clone`/`__destruct`), `clone`-with, - 8.4 **property hooks** + asymmetric visibility, closures + first-class callable - syntax (`f(...)`), by-ref / variadic params + **argument unpacking** (`f(...$a)`), - dynamic callables (`call_user_func`, string/array callables), the pipe operator - `|>`, `match`, DNF types, null-coalescing / nullsafe (`??`, `?->`), `global` / - `static` locals, heredoc / nowdoc, encapsed-string interpolation, - `define`/constants, generators (`yield` / `yield from`), exceptions with - `try`/`catch`/`finally` and stack traces (`getTrace` / `debug_backtrace`), - the full `preg_*` family (10 functions via the host PCRE2 library), - by-reference to an **array element / object property** (`f($a[0])`, `f($o->v)`) - and out-parameter auto-vivification (`preg_match($re, $s, $m)`), reference - returns (`function &f()`), and file I/O over libc. -- ✅ **Generics** (docblock-driven, so source stays valid PHP): `@template` with - bounds (`T of C`) + defaults (`T = X`), `@extends`/`@implements`, generic - **traits** (`@use T`, zero-cost), and **reified** `@var Box = new Box` - (a real specialized class, no boxing) — plus implicit **monomorphization** of - erased `array` / `callable` params. See [`docs/generics.md`](docs/generics.md). - -## Benchmarks - -Native AOT output vs the Zend `php` interpreter (PHP 8.5.3). Apple M1 Pro, -`-O2`, best of 5, each compiled program verified byte-equal to `php` first. -Every loop is **data-dependent and `$argc`-seeded** so the LLVM optimizer can't -fold it away — these numbers are real codegen throughput, not a deleted loop. -Reproduce with `bash bench/run.sh` (cases in `bench/cases/`). - -**Compute-bound** — native codegen dominates (the interpreter's per-op dispatch -is the tax we skip): +standalone native binaries (arm64 / x86_64) through LLVM IR — no PHP runtime, no +shared libraries beyond libc. **The compiler is written in PHP and compiles itself +to a byte-identical fixpoint.** -| Benchmark | Workload | `php` | manticore | Speedup | -|-----------|----------|------:|----------:|--------:| -| `oop` | 20 M polymorphic virtual `area()` (LCG-indexed) | 1.13 s | 0.04 s | **28×** | -| `fib` | recursive fib, data-dependent depth [30,33] | 1.86 s | 0.10 s | **19×** | -| `closures`| 2 M closure build + call | 0.19 s | 0.01 s | **19×** | -| `mathf` | 3 M `sqrt` + `sin` accumulate | 0.16 s | 0.01 s | **16×** | -| `loop` | 50 M-iter data-dependent accumulate | 0.68 s | 0.06 s | **11×** | -| `sieve` | Sieve of Eratosthenes to 2 M | 0.20 s | 0.02 s | **10×** | -| `array` | 30× build + sum a 500 K-element vec | 0.35 s | 0.07 s | **5.0×** | -| `strcat` | 30 M-iter string append | 0.37 s | 0.11 s | **3.4×** | - -**Algorithms** (Computer Language Benchmarks Game style — float math, tight -loops, 2-D arrays): - -| Benchmark | `php` | manticore | Speedup | -|-----------|------:|----------:|--------:| -| `spectralnorm` | 0.44 s | 0.01 s | **44×** | -| `mandelbrot` | 0.26 s | 0.03 s | **8.7×** | -| `matmul` | 0.08 s | 0.01 s | **8×** | -| `dijkstra` | 0.14 s | 0.03 s | **4.7×** | -| `nbody` | 0.17 s | 0.04 s | **4.2×** | - -**Library-bound** — time is spent in the PHP-level stdlib/prelude (arrays, -strings, JSON), so the win is smaller and tracks how optimized that helper is, -and how much it competes with PHP's hand-tuned C: +```bash +manticore compile app.php -o app && ./app # one file → one static binary +``` -| Benchmark | Workload | `php` | manticore | Speedup | -|-----------|----------|------:|----------:|--------:| -| `funcarr` | `array_map`/`filter`/`reduce` | 0.16 s | 0.02 s | **8.0×** | -| `strops` | `strtoupper`/`str_replace` loop | 0.07 s | 0.02 s | **3.5×** | -| `wordcount`| assoc map build + iterate | 0.07 s | 0.02 s | **3.5×** | -| `sort` | `sort()` 3 K ints × 200 | 0.12 s | 0.05 s | **2.4×** | -| `sprintf` | `sprintf` formatting loop | 0.09 s | 0.04 s | **2.2×** | -| `in_array`| `in_array` linear scan over a vec | 0.15 s | 0.07 s | **2.1×** | -| `explode` | `explode`/`implode` loop (fused) | 0.14 s | 0.07 s | **2.0×** | -| `assoc` | string-keyed map build + lookup | 0.15 s | 0.08 s | **1.9×** | -| `json` | `json_encode` loop | 0.12 s | 0.08 s | **1.5×** | - -- **Cold start** (`echo "hello"`): `php` 62 ms vs manticore **2.6 ms** (~24×) — - no interpreter/extension init. -- **Compile time**: `fib.php` → native binary in **~0.1 s** (parse → MIR → - LLVM → clang → link). -- **Output size**: a trivial program links to a **~50 KB** fully-static binary - (libc only); the self-hosted compiler is ~4.3 MB. - -The library-bound tail is the tightest race — these lean on stdlib/prelude -helpers competing with PHP's hand-tuned C, yet native still wins every one. -`json_encode` is a native single-buffer codegen builtin (a recursive cell walk -that formats ints/floats and escapes strings straight into one growing buffer), -`explode`/`implode` round-trips fuse into a single native `str_replace`, assoc -lookups cache each string's hash in its header (Zend `zend_string` style), and -string builders (`$s = $s . …`) append in place, not the former O(n²) copy. +The output has no interpreter to install, no `php.ini`, no extension list — you ship +the binary. What you write is ordinary PHP: the Zend interpreter is the reference, +and every plain-runnable test case is diffed against it. On top of that sits a +[superset](docs/superset.md) `php` cannot run at all — structured concurrency, FFI, +a module system, compile-time attributes. + +--- ## Requirements -Emitted binaries are fully static and depend on nothing but libc. The -*compiler* needs a real toolchain on the host: +**Emitted binaries need nothing but libc.** The *compiler* needs a real toolchain on +the host, because it ends in `clang` and `cc`: -- **`clang`** and **`cc`** on `PATH`, with **LLVM ≥ 15** — Manticore emits - opaque-pointer IR, which clang 14 rejects. -- **PHP 8.5** — only for the cold bootstrap (Zend runs the compiler source once - to seed the first native binary; emitted binaries make no PHP-runtime calls). -- **libpcre2** (`preg_*`) and **OpenSSL 3** (TLS, `hash`/`hmac`) development - packages, plus `pcre2-config` / `pkg-config` to locate them. +| What | Version | Why | +|---|---|---| +| `clang` + `cc` on `PATH` | **LLVM ≥ 15** | Manticore emits opaque-pointer IR; clang 14 rejects it | +| `php` | **8.5** | cold bootstrap only — Zend runs the compiler source once to seed the first native binary | +| libpcre2 (**dev** package) | 10.x | `preg_*` rides host PCRE2; needs `pcre2-config` | +| OpenSSL 3 (**dev** package) | 3.x | TLS, `hash`/`hmac`; needs `pkg-config` | -Both **macOS** (arm64 / x86_64) and **Linux** (glibc ≥ 2.33, arm64 / x86_64) are -supported — each builds the compiler and passes the full suite, including the -self-host fixpoint. +The `-dev` / `-devel` half matters: the headers are what the build looks for, not just +the runtime library. + +**Platforms:** macOS (arm64 / x86_64) and Linux (glibc ≥ 2.33, arm64 / x86_64). Each +builds the compiler and passes the full suite including the self-host fixpoint. Alpine +(musl) builds; see [`docs/install.md`](docs/install.md) for the current caveats. + +```bash +# macOS +brew install php pcre2 openssl@3 pkg-config + +# Debian / Ubuntu — plus clang from your distro or apt.llvm.org, and PHP 8.5 from sury.org +sudo apt-get install -y build-essential pkg-config libpcre2-dev libssl-dev +``` -**Per-OS package lists, Docker images and troubleshooting: -[`docs/install.md`](docs/install.md).** +Per-OS package lists, Docker images and troubleshooting: +**[`docs/install.md`](docs/install.md)**. -## Quick start +## Install + +There is no prebuilt binary to download — the compiler compiles itself. The installer +checks the toolchain above, tells you what is missing, then installs under +`$MANTICORE_HOME` (default `~/.manticore`): ```bash -# Install: the compiler builds itself into ~/.manticore (needs the toolchain above) curl -fsSL https://raw.githubusercontent.com/manticorephp/compiler/main/install.sh | bash export PATH="$HOME/.manticore/bin:$PATH" -# ...or, from a checkout: +manticore version # manticore 0.6.0 +``` + +Re-running the installer **upgrades in place**: an existing `manticore` rebuilds the +new version *with itself* (self-host, fast) — the Zend seed is only the cold first +boot. Knobs: `MANTICORE_HOME`, `MANTICORE_REF` (branch/tag), `MANTICORE_REPO`, +`MANTICORE_SRC` (build a local checkout instead of cloning). -# Cold bootstrap: Zend seeds the first native compiler (no binary needed yet) -bash bin/compile # → bin/manticore (~4.3 MB static binary) +Via Composer, which here is a delivery + build trigger rather than a runtime: -# Thereafter, the compiler rebuilds itself (Zend seed is only the cold start) -bash bin/build # self-host: bin/manticore compiles src/ -bash bin/build --verify # + fixpoint + suite gate +```bash +composer create-project manticorephp/compiler manticore # builds into ~/.manticore +vendor/bin/manticore-install # if it is already a dependency +``` -# Compile and run a program -bin/manticore compile path/to/app.php -o app && ./app +The installed layout is self-contained and needs no environment variables — the binary +finds its runtime relative to itself: -# Source from stdin -echo ' -o ` | PHP source → native binary (file arg or stdin) | -| `build [manticore.json]` | Build all manifest targets (libraries + applications) | -| `dump-ast ` | Parse → print the AST | -| `dump-mir ` | Lower → print the typed MIR | -| `dump-llvm-mir ` | Full MIR pipeline → print LLVM IR | -| `dump-llvm ` | Same as `dump-llvm-mir` (the AST backend was removed; MIR is the only backend) | -| `dump-sig ` | Print the module-interface `.sig` (exported symbol table) | +| `build [manticore.json]` | build all manifest targets (libraries + applications) | +| `analyze ` | the static checks a compiler can make and an interpreter never gets to | +| `dump-ast` / `dump-mir` / `dump-llvm-mir` | parse / typed MIR / LLVM IR | +| `dump-sig ` | the module interface (exported symbol table) | | `version` / `help` | — | Flags: `-o `, `-O<0|1|2|3|s|z>` (clang opt level, default `-O2`), -`--emit-library` (compile to a standalone `.o` with no `@main`), -`--memory=rc|arena|hybrid`. `build` also takes `--libs-only` (build the -library targets and stop). - -## Module system (`manticore.json`) - -A cargo-style manifest builds multi-file projects, links prebuilt libraries, and -self-reproduces the compiler. **Full end-user guide: [`docs/modules.md`](docs/modules.md).** - -```json -{ - "libraries": [{ - "name": "stdlib", - "src": "src/Runtime", - "output": "lib/manticore_stdlib.o", - "runtime": true - }], - "applications": [{ - "name": "compiler", - "src": "src", - "output": "bin/manticore", - "entry": "src/zzz_entry.php", - "stdlib": false - }] -} +`--emit-library` (a standalone `.o` with no `@main`), `--memory=rc|arena|hybrid`. +`compile` **analyzes by default** and prints warnings without failing the build — +`--no-analyze` turns it off, `--analyze-strict` makes error-severity findings fail the +compile (rc=65). + +⚠ The `dump-*` commands do **not** link the stdlib, so a call into it resolves as +`unknown`. When that matters, read the final binary. + +## The PHP you get + +Classes / interfaces / traits / enums (enum methods, constants, interface-implementing +enums), abstract + anonymous classes, late static binding (`new static`, +`static::method()`, `parent::`/`self::` forwarding), magic methods +(`__get`/`__set`/`__call`/`__invoke`/`__clone`/`__destruct`), `clone`-with, 8.4 +**property hooks** + asymmetric visibility, closures + first-class callable syntax +(`f(...)`), by-ref / variadic params + argument unpacking, dynamic callables, the pipe +operator `|>`, `match`, DNF types, `??` / `?->`, `global` / `static` locals, heredoc / +nowdoc, string interpolation, constants, generators (`yield` / `yield from`), +exceptions with `try`/`catch`/`finally` and real stack traces, references to an array +element / object property and out-parameter auto-vivification +(`preg_match($re, $s, $m)`), reference returns, attributes, Reflection, Fibers, and +file I/O over libc. + +**Generics** are docblock-driven, so the source stays valid PHP: `@template` with +bounds and defaults, `@extends`/`@implements`, generic traits (zero-cost), and +**reified** `@var Box = new Box` — a real specialized class, no boxing — plus +implicit monomorphization of erased `array` / `callable` params +([`docs/generics.md`](docs/generics.md)). + +**Standard library:** the `array_*` family in full, strings (incl. the whole `preg_*` +family over host PCRE2), type/reflection, math, `ctype_*`, JSON, `var_dump`/`print_r`, +SPL, date/time, sockets and streams, hashing and crypto. Each function is either a +PHP-level stdlib function (`src/Runtime/Stdlib/`, compiled into +`lib/manticore_stdlib.o` and auto-linked), an injected prelude helper, or an inlined +codegen builtin. No imports, no registration — they are simply there. + +Current gaps are tracked with repros in [`docs/ROADMAP.md`](docs/ROADMAP.md); the +headline ones are listed under [Limitations](#limitations). + +## Beyond PHP — the superset + +Parity is the north star and `tools/difftest.sh` enforces it. The rest — the surface +`php` cannot run, and difftest therefore cannot check — is catalogued in +**[`docs/superset.md`](docs/superset.md)**: concurrency, compile-time attributes, FFI, +the module system, the type system, the memory model. + +The headline is **structured concurrency** — Go's model, PHP's spelling, written in +PHP over two primitives of ours (native `Fiber` on `fcontext`, and `Io\Poll` over +kqueue/epoll): + +```php +Async\async(function () { + $a = Async\spawn(fn() => file_get_contents('https://example.com/one')); + $b = Async\spawn(fn() => file_get_contents('https://example.com/two')); + [$x, $y] = Async\awaitAll($a, $b); // ~1 RTT, not 2 +}); ``` -- **`applications[]`** — `src` directory, `output` binary, optional `entry` - (the file whose top level becomes `main`), optional `exclude`, optional - `libraries` (user library deps — omit ⇒ all, `[]` ⇒ none), optional - `stdlib: false` (opt out of the always-on stdlib runtime), optional - `composer` — `true` builds the project the way Composer sees it: its - `composer.json` autoload (psr-4/psr-0/classmap dirs) **and** every installed - package from `composer.lock` (`vendor//`). The object form - `{ "vendor": false }` takes only the project's own autoload. -- **`libraries[]`** — compiled to `.o` + an `.o.sig` interface; - an application links a user library by naming it. A `runtime: true` library - (the stdlib) is built but auto-linked into every app (see above). -- **`.sig`** files carry a module's public symbol table so a dependent target - resolves cross-unit calls *without* re-parsing the dependency's sources. A - distributed compiler ships `bin/` + `lib/manticore_stdlib.{o,sig}` only — no - PHP sources — and resolves the bundled stdlib by `.sig`. - -The **stdlib is the always-on runtime**: every `compile`/`build` program gets it -transparently (no manifest ceremony), independent of the `libraries` selection; -opt out with `"stdlib": false` (only the self-contained compiler, which embeds -`src/Runtime`, does). See [`docs/modules.md`](docs/modules.md). +Ordinary `fread`/`fwrite`/`stream_socket_accept`/`sleep` suspend the fiber instead of +the process — plain streams *are* the async API, TLS and DNS included. Every task is +owned by a scope, cancellation is delivered at the suspend point, a deadlock is +reported rather than exited, and `Async\dump()` names every live task and where it was +spawned. An 8-worker prefork HTTP server does **150–160k rps** (`wrk`, plaintext +keep-alive). See [`docs/async.md`](docs/async.md). **Native libraries** (zlib, libcurl, …) bind through FFI — `#[Library, Symbol]` -attributes compile to direct C calls; declare them as manifest `extensions`. -Mechanism + type mapping: [`docs/ffi.md`](docs/ffi.md). - -## Standard library - -**~228 PHP standard-library functions** are implemented across three tiers, all -exposed to user programs transparently (no imports, no registration): +attributes compile to direct C calls, declared as manifest `extensions`; mechanism and +type mapping in [`docs/ffi.md`](docs/ffi.md). The **module system** +([`docs/modules.md`](docs/modules.md)) is a cargo-style `manticore.json` with +`applications` and `libraries`, `.sig` module interfaces so a dependent target resolves +cross-unit calls without re-parsing sources, and a distributable compiler that ships +`bin/` + `lib/` with no PHP sources at all. -| Family | Count | Examples | -|--------|------:|----------| -| `array_*` | 35 | `array_map`/`filter`/`reduce`, `usort`/`uasort`/`uksort`, `array_merge`, `array_column`, `array_diff`, `array_unique` | -| String | 43 | `str_replace`, `substr`, `explode`/`implode`, `sprintf`, `preg_*` (10, via host PCRE2), `str_pad`, `wordwrap`, `levenshtein` | -| Type / reflection | 30 | `is_*`, `gettype`, `get_class`, `get_object_vars`, `class_exists`, `method_exists` | -| Math | 28 | `abs`, `sqrt`, trig, `intdiv`, `fmod`, `round`, `pow`, `max`/`min` | -| `ctype_*` | 11 | `ctype_digit`, `ctype_alpha`, … | -| Var / JSON / SPL / date / I/O | remainder | `var_dump`, `var_export`, `print_r`, `json_encode`, `SplStack`/`SplQueue`, `time`/`date`, `fopen`/`fread` over libc | +## Performance -Each function is one of: a **PHP-level stdlib function** (`src/Runtime/Stdlib/`, -compiled into `lib/manticore_stdlib.o` and auto-linked), an **injected prelude -helper** (`prelude/`, inlined into each program), or an **inlined codegen -builtin** (`src/Compile/Mir/Passes/EmitLlvmBuiltins.php`, emitted as a primitive -/ libc call / LLVM intrinsic). See `docs/ROADMAP.md` for the gap matrix. +Native AOT output vs the Zend interpreter, Apple M1 Pro, `-O2`, best of 5, every +compiled program verified byte-equal to `php` first. Loops are data-dependent and +`$argc`-seeded so LLVM cannot fold them away. Representative slice — reproduce the +full set with `bash bench/run.sh` (cases in `bench/cases/`): -## Compiler pipeline +| Benchmark | Workload | `php` | manticore | Speedup | +|---|---|---:|---:|---:| +| `spectralnorm` | float math, tight loops | 0.44 s | 0.01 s | **44×** | +| `oop` | 20 M polymorphic virtual calls | 1.13 s | 0.04 s | **28×** | +| `fib` | recursive, data-dependent depth | 1.86 s | 0.10 s | **19×** | +| `sieve` | Eratosthenes to 2 M | 0.20 s | 0.02 s | **10×** | +| `funcarr` | `array_map`/`filter`/`reduce` | 0.16 s | 0.02 s | **8.0×** | +| `strcat` | 30 M-iter string append | 0.37 s | 0.11 s | **3.4×** | +| `sort` | `sort()` 3 K ints × 200 | 0.12 s | 0.05 s | **2.4×** | +| `json` | `json_encode` loop | 0.12 s | 0.08 s | **1.5×** | + +Compute-bound work wins big (no per-op dispatch). The library-bound tail is the +tightest race — those helpers compete with PHP's hand-tuned C — and native still wins +every one. Also: **cold start** 2.6 ms vs 62 ms (~24×), a trivial program links to a +**~50 KB** fully-static binary, and `fib.php` compiles to native in ~0.1 s. + +## How it is built ``` PHP source @@ -260,11 +222,11 @@ PHP source → LowerFromAst ─┐ → ConstFold │ → DeadStore │ - → InferTypes │ MIR (src/Compile/Mir) — flat, typed, SSA-ish IR - → InlineClosures │ The only backend (the AST backend was removed); - → Monomorphize │ EmitLlvm builds IR text via the src/Codegen/Llvm helpers. - → NarrowReturns │ Monomorphize specializes erased-array / callable params - → CheckTypeDefs │ per concrete call-site shape (see docs/design). + → InferTypes │ MIR (src/Compile/Mir) — flat, typed, SSA-ish IR. + → InlineClosures │ The only backend; Monomorphize specializes erased-array + → Monomorphize │ and callable params per concrete call-site shape. + → NarrowReturns │ + → CheckTypeDefs │ → DemoteCharLocals │ → InferEffects │ → InferAllocKind │ @@ -276,117 +238,64 @@ PHP source → cc link static binary (libc only) ``` -## Memory model - -Full guide + how to control it: [`docs/memory.md`](docs/memory.md). - -- **Reference counting** on strings, objects, vecs, and assoc arrays, with - **copy-on-write** for assoc snapshots/stores (Zend-style). Deterministic - frees, no GC pauses. -- **Cycle collector v1** — synchronous Bacon–Rajan, **opt-in**: zero overhead - unless a program reaches `gc_collect_cycles()`. (v1 limit: manual trigger; - static/global roots not scanned.) -- **Allocation modes** (`--memory` / `MANTICORE_MEMORY`): `hybrid` *(default)*, - `rc`, `arena` (bump-pointer, scope-freed). Escape analysis (`InferAllocKind`) - routes each allocation between arena (confined) and heap-rc (escaping). -- The unified `PhpArray` is one runtime type (vec + assoc collapsed) with FNV - bucket indexing for hashed keys. +**Memory** ([`docs/memory.md`](docs/memory.md)): reference counting on strings, +objects, vecs and assoc arrays with copy-on-write, so frees are deterministic and there +are no GC pauses; a synchronous Bacon–Rajan cycle collector that is opt-in and +zero-overhead until `gc_collect_cycles()` is reached; and three allocation modes +(`--memory` / `MANTICORE_MEMORY`) — `hybrid` (default, escape analysis routes each +allocation between arena and heap-rc), `rc`, `arena`. -## Source layout - -Pure PHP, one class/interface/trait/enum per file, path mirrors FQN. +**Source layout** — pure PHP, one class per file, path mirrors FQN: ``` bin/ build & run scripts + the output binary - compile cold seed: Zend builds a throwaway seed, which then runs - `build manticore.json` → native bin/manticore + stdlib + compile cold seed (Zend → throwaway seed → native compiler + stdlib) build self-host rebuild via the manifest (+ --seed, --verify) -lib/ prebuilt stdlib object + .sig (gitignored build artifacts) -src/Lexer/ tokenizer -src/Parser/ recursive-descent + Pratt parser; AST node types -src/Compile/ AST → MIR lowering, MIR passes, and the EmitLlvm backend - Mir/ the typed IR (Node/Type/Module) + Passes/ pipeline - Runtime/, TypeHint/, MemoryAbi.php, MemoryOp.php -src/Codegen/Llvm/ low-level LLVM-IR text builders (Module/Block/Type/Value) - used by EmitLlvm + the runtime hosts; no semantic logic - (new emission belongs in Compile/Mir/Passes/EmitLlvm*) -src/Runtime/ PHP-level stdlib + runtime helpers compiled into binaries, - plus libc / OS / Json bindings -src/Ffi/ FFI binding attributes -src/Os/ OS / syscall layer -src/Manticore/ driver (Main.php), Sig.php (module interfaces), build command -src/Cli/ CLI dispatch -tools/ build + gate scripts (selfhost, difftest, …) -tests/aot/ primary harness: cases/*.php + expected/*.out -docs/ROADMAP.md status, gap matrix, planning method (start here) -docs/design/ design docs (module-system, type-system-v2, generators, - late-static-binding, monomorphization, …) +lib/ prebuilt stdlib object + .sig + prelude (build artifacts, gitignored) +prelude/ PHP injected into every program (Fiber, async runtime, Resource, …) +src/Lexer/ tokenizer +src/Parser/ recursive-descent + Pratt parser; AST node types +src/Compile/ AST → MIR lowering, MIR passes, EmitLlvm backend +src/Codegen/ low-level LLVM-IR text builders (no semantic logic) +src/Runtime/ PHP-level stdlib + libc / OS / FFI bindings compiled into binaries +src/Manticore/ driver (Main.php), Sig.php, the build command +tools/ build + gate scripts (selfhost, difftest, docker, …) +tests/aot/ the harness: cases/*.php + expected/*.out, auto-discovered +examples/ runnable demos (examples/async/ has the concurrency ones) +docs/ the guides; docs/ROADMAP.md is the status + gap matrix ``` -`src/zzz_entry.php` sorts last and holds the top-level `main_driver()` call the -binary's `main` lowers to. - -## Self-hosting & gates +## Gates ```bash -bash tests/aot/run.sh # AOT suite (467 cases) +bash tests/aot/run.sh # the AOT suite (cases/ + expected/, auto-discovered) bash tests/aot/run.sh -k hello # filter by substring -bash tools/difftest.sh # parity vs `php` (PHP 8.5.8) -bash tools/selfhost_fixpoint.sh # fixpoint + self-host suite + stability +bash tools/difftest.sh # parity vs `php` +bash tools/selfhost_fixpoint.sh # fixpoint + self-host suite + rebuild stability +bash tools/docker/run_tests.sh --gate # the same, on Linux ``` -`bin/compile` cold-seeds (Zend → throwaway seed → native compiler via the -manifest); `bin/build` self-hosts. `selfhost_fixpoint.sh` asserts gen2 IR == -gen3 IR, runs the suite through the self-built compiler, and rebuilds 5×2 to -catch layout roulette. - -## Known limitations - -- **Integer overflow wraps** (two's-complement) instead of promoting to float - as PHP does — `PHP_INT_MAX + 1` gives `PHP_INT_MIN`, not a float. -- **Dynamic name resolution** is not yet supported — `new $cls()`, `$cls::m()`, - `$o->$m()`, `$o->$prop`, a computed-string `$f()`, `$obj instanceof $cls`, and - `ReflectionClass`. (Literal / first-class / `call_user_func($strVar, …)` - callables do work.) -- **`extract()`** is not implemented (it needs dynamic symbol-table writes the - typed frame does not model). `compact()` works. -- **`goto` into a loop body** is unsupported (plain forward/backward `goto` - works). -- **Cycle collector** is manual-trigger only; static/global roots not scanned. -- **Multi-object linking composes** (resolved): a binary linked from two - manticore objects (user `.o` + prebuilt `stdlib.o`) is correct and - byte-identical — class ids are content-hashed (stable across objects) and - drops route through a per-class `linkonce_odr` descriptor + indirect drop_fn, - so a class one object doesn't know still drops correctly. The compiler still - self-builds self-contained (stdlib embedded) for simplicity; user programs - link the cached `stdlib.o`. - -## Roadmap / next steps - -1. **Parity tail.** Close the remaining `tools/difftest.sh` gaps toward the full - PHP 8.5 surface: integer overflow → float promotion, the `sprintf` flag - corners (`%b`, width+precision, `%e` exponent), `extract`/`compact`. Each is a - scoped stdlib/codegen fix, not an architectural one. -2. **Representation soundness.** Continue the typed⇄cell array reabstraction the - monomorphization + de-cellify work opened (`docs/design/unknown-cell-soundness.md`, - `docs/design/monomorphize-callable-dim.md`): erased-array boundaries now - specialize + de-cellify at stores; broaden the same discipline to the last - raw-guessing consumers. -3. **Build cache.** A content-addressed cache (`~/.manticore/cache`, keyed by - srchash + compiler ABI + target triple) to skip re-lowering/re-clang of - unchanged modules — a speed feature (multi-object linking already composes). -4. **Extension system.** MVP shipped (manifest `extensions`, glue compiled in, - `-l` linked; proof: `zlib`/`crc32`, and the `preg_*` family over host - PCRE2). Next: static-archive linking (keeps binaries fully static) + real - extensions (curl / xml / pdo) on the same FFI mechanism. -5. **Module system depth.** Weak library symbols (app can override), composer - packaging + `dependencies` resolution (`vendor/`, lockfile), cross-library - `.sig` classes. -6. **Memory.** Cycle-collector roots for statics/globals + automatic trigger; - broaden arena/hybrid escape routing. +`selfhost_fixpoint.sh` asserts gen2 IR == gen3 IR, runs the suite through the +self-built compiler, and rebuilds repeatedly to catch build-to-build layout roulette. +The Linux gate is not optional for anything touching `src/Runtime/`, syscalls or errno. + +## Limitations + +- **Integer overflow wraps** (two's-complement) instead of promoting to float — + `PHP_INT_MAX + 1` gives `PHP_INT_MIN`. +- **`extract()`** is not implemented (dynamic symbol-table writes the typed frame does + not model). `compact()` works. +- **`goto` into a loop body** is unsupported (plain forward/backward `goto` works). +- **Cycle collector** is manual-trigger only and does not scan static/global roots. +- **A `.sig` carries functions only** — classes, interfaces, traits, enums and + constants do not yet cross a compiled-library boundary. +- **Regular-file I/O blocks the async loop** by design; see + [`docs/async.md`](docs/async.md#-what-is-not-async) for the measurements and + `Async\readFile()`. + +The full, current gap matrix with repros lives in +[`docs/ROADMAP.md`](docs/ROADMAP.md). ## License Licensed under the [MIT License](LICENSE). - -[issue #1]: https://github.com/manticorephp/compiler/issues/1 diff --git a/bin/.gitignore b/bin/.gitignore index c9807fa..abfdb3a 100644 --- a/bin/.gitignore +++ b/bin/.gitignore @@ -1,6 +1,12 @@ # Self-compile artifacts. Only the driver script is checked in. +# +# The wildcards matter: tools/selfhost.sh writes manticore_self{,.ll,_stubs.c} +# here, and a 954k-line .ll that is neither tracked nor ignored rides into the +# next `git add -A` — which is exactly how it got committed once. manticore manticore.ll manticore.o manticore_stubs.c manticore_stubs.o +manticore_self* +manticore_g[0-9]* diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index 71c3a00..969cd22 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -12,25 +12,27 @@ _Last updated: 2026-07-15 · branch `main` · HEAD `a6e062a`._ Pure-PHP, self-hosting PHP→native AOT compiler. The runtime is emitted as LLVM IR from PHP; `bin/manticore` compiles its own `src/`. -**Gates (all green):** -- `tests/aot/run.sh` — **467/467** -- `tools/selfhost_fixpoint.sh` — fixpoint byte-identical · self-host · stability 5×2 -- `tools/difftest.sh` — **458 match / 0 diff** vs PHP 8.5.8 +**Gates (all green, macOS + Linux):** `tests/aot/run.sh` · `tools/difftest.sh` (0 diff vs +PHP 8.5.8) · `tools/selfhost_fixpoint.sh` (fixpoint byte-identical · self-host · MIR golden · +stability 5×2) · `tools/docker/run_tests.sh --gate`. Counts move every session — run them +rather than trusting a number written here. + +⚠ **`bin/build` green says nothing about `tools/selfhost.sh`.** The manifest build compiles +`src/Runtime` as a LIBRARY with a flattened namespace; the self-host path takes everything as +ONE module. They diverge on emitted symbol names, and only the stability gate covers the +second. Corollary: **never ship a compiler fix together with tree code that needs that fix** — +the previous generation then cannot build the tree at all, and only a cold seed recovers. Build: `bin/build --seed` (cold Zend bootstrap → native) / `bin/build` (self-host). See `.claude/CLAUDE.md` for the pipeline and key files. -**~228 stdlib functions** implemented (array/string/type/math/ctype/`preg_*`/JSON/ -var/SPL/date/IO); see the README's Standard library table. - -**Recently completed** (2026-07): full `preg_*` family over host PCRE2 + -`#[RefOut]` out-param auto-vivification · Monomorphize **callable dimension** -(specialize callback-takers per concrete closure) · **de-cellify** at the -concrete-array ← cell-array store boundary — fixes `uasort` with any comparator -and dents the erased-array representation root · Ryu float formatting · JSON -decode 3.3× + default-flag escaping · assoc key-leak fix (malloc −97%) · -reified-class generics · pin elimination (278→0). Detail in the session memory -files under `.claude/.../memory/`. +**Recently completed** (2026-07): the async runtime — structured concurrency, transparent +netpoller I/O, and the liveness pass over it (bounded write parks, a classified `accept(2)` +loop that backs off on EMFILE instead of spinning, resolv.conf `search`/`ndots`) · +`docs/superset.md`, the catalogue of everything with no Zend oracle · the full `preg_*` family +over host PCRE2 + `#[RefOut]` · Monomorphize **callable dimension** · **de-cellify** at the +concrete-array ← cell-array store boundary · Ryu float formatting · reified-class generics · +pin elimination. Detail in the session memory files under `.claude/.../memory/`. ## The gap matrix (probed, with repros) diff --git a/docs/async.md b/docs/async.md index 457e9d5..f956eb3 100644 --- a/docs/async.md +++ b/docs/async.md @@ -13,6 +13,11 @@ install, nothing to link — `use function Async\spawn;` is enough. This directory holds the examples and benchmarks. +Everything here is **superset**: `php` has `Fiber` and nothing else of it, so `difftest` cannot +check a single line — these cases carry hand-written expected output instead. +[docs/superset.md](superset.md) catalogues that whole surface (concurrency, attributes, FFI, +modules, types, memory) and the rules it lives under. + ## Model (not Promises) The concurrency *model* is Go's — cheap tasks, channels, a netpoller, blocking-*looking* I/O @@ -265,6 +270,34 @@ $sem = new Async\Semaphore(4); $sem->withPermit(fn() => work()); ``` +**An accept loop needs one.** It is the one place a scheduler will happily let you +spawn without bound: `while (true) { $g->spawn(serve(accept())); }` turns a +connection burst into unbounded tasks, fds and memory, and the first thing to fail is +something unrelated. Take the permit **before** the accept — then the queue stays in +the kernel backlog, where it belongs, instead of in your heap. + +```php +$gate = new Async\Semaphore(256); +Async\group(function (Async\TaskGroup $g) use ($server, $gate) { + while (true) { + $gate->acquire(); // BEFORE the accept + $conn = stream_socket_accept($server); + if ($conn === false) { $gate->release(); continue; } + $g->spawn(function () use ($conn, $gate) { + try { serve($conn); } finally { $gate->release(); } // in the CHILD + }); + } +}); +``` + +`Async\stats()['live']` is the gauge to alert on, and `Async\dump()` names the tasks +when it climbs. There is deliberately **no built-in ceiling on tasks per scope**: +`spawn()` is the wrong place to fail (the exception would land in the acceptor, the +one task that has to survive an overload), a hard error turns overload into a crash +instead of flow control, and `live` counts timers and reporters that share nothing +with the work being limited. The policy — park, shed, or answer 503 — belongs to the +application, and a Semaphore owned by the scope that owns the work already expresses it. + ### Scoped values ```php @@ -282,15 +315,32 @@ Ordinary stream calls suspend the fiber instead of the process when a scheduler `connect(2)` runs non-blocking, BOTH TLS handshake directions are driven through `WANT_READ`/`WANT_WRITE` parks (client `SSL_connect` and server `SSL_accept` — so a TLS server serves concurrent clients), and a hostname is resolved over the netpoller: -`/etc/hosts`, a per-run cache held by the scheduler, then A and AAAA queries across -**every** nameserver in `resolv.conf` (two attempts each, 2 s apiece, TC → retry over -TCP), falling back to the blocking `getaddrinfo` walk when none of that can answer. +`/etc/hosts`, a per-run cache held by the scheduler, the `search` list and `ndots` rule from +`resolv.conf` (so `db`, `redis`, `service.namespace` — the names compose and kubernetes hand +you — resolve here instead of falling through to the blocking walk), then A and AAAA queries +across **every** nameserver (two attempts each, 2 s apiece, TC → retry over TCP), falling +back to the blocking `getaddrinfo` walk when none of that can answer. A search walk stops the +moment NO server answers: a dead resolver must not be multiplied by the suffix count. Two spawned HTTPS fetches to DIFFERENT hosts run 0.17s vs 0.34s sequential. -`stream_set_timeout()` and `stream_socket_accept($srv, $timeout)` are honoured under the -scheduler: the wait is BOUNDED, so a hung peer no longer wedges the fiber forever (an -unbounded park was the old behaviour, and a timeout used to fall back to a blocking `poll(2)` -that stalled every other task). +**Every wait is bounded.** `stream_set_timeout()` is the STREAM's timeout, as in php — reads +*and* writes — and `stream_socket_accept($srv, $timeout)` is honoured under the scheduler. +On expiry the operation reports a short read/write with `stream_get_meta_data()['timed_out']` +true, never an exception. Unbounded parks were the old behaviour on both sides, and each was +a liveness hole rather than a missing feature: a peer that stops reading wedged the writing +fiber forever — the task never settled, its scope never closed, the fd was never released — +and a timeout used to fall back to a blocking `poll(2)` that stalled every other task. The +default when nothing is set is 60 s, php's `default_socket_timeout`. + +**`accept(2)` failures are classified.** A would-block parks; a peer that vanished between its +SYN and our accept (`ECONNABORTED`/`EPROTO`/`EINTR`) retries immediately, since no readiness +edge is coming for a connection that is already gone; resource exhaustion +(`EMFILE`/`ENFILE`/`ENOBUFS`/`ENOMEM`) backs off on a timer and keeps serving what is already +open; anything else is reported. The overload case is why this matters: `accept(2)` fails +while the pending connection **stays queued**, so a level-triggered listener stays readable — +re-arming readiness is a hot spin that starves every sibling task and never even trips the +watchdog, because it suspends on every iteration. It shows up only as `stats()['wakes']` +climbing with wall time instead of with connections. The seam is `\Runtime\AsyncHook` (five callbacks installed by the scheduler — readable, writable, close, bounded-readable, sleep; one null check per would-block when no scheduler is @@ -385,10 +435,13 @@ is still there. ## Not yet -Also: `writev`/`io_uring` to break the 2-syscall floor · DNS search-domain/`ndots` handling -(a name needing a suffix falls back to the blocking walk today) · off-thread file I/O · -shared-memory multithreading (a future compiler superset). See the async roadmap memory for -the plan. +Also: `writev`/`io_uring` to break the 2-syscall floor · `resolv.conf`'s `timeout:`/`attempts:` +options (the search list and `ndots` are in; those two still hard-code 2 s × 2) · the search +list applied to `/etc/hosts` the way glibc does · off-thread file I/O · shared-memory +multithreading (a future compiler superset) · **fiber stack sizing**: 8 MiB of *virtual* +address space per task with a `PROT_NONE` guard page, pooled and reclaimed promptly, but +nobody has measured where 10 000 concurrent tasks actually hits `vm.max_map_count` or a +container's `ulimit -v`. See the async roadmap memory for the plan. **`#[Async]`** — an attribute that turns a call into a spawned `Task` of the function's return type. The only piece here that cannot be a library: it needs the compiler to split diff --git a/docs/superset.md b/docs/superset.md new file mode 100644 index 0000000..6c6f553 --- /dev/null +++ b/docs/superset.md @@ -0,0 +1,270 @@ +# The superset — what Manticore adds on top of PHP + +Manticore's north star is that PHP **feels like PHP**: the interpreter is the oracle, and +`tools/difftest.sh` compares our output against it case by case. Everything on that side of +the line is *parity* work, and it is not what this document is about. + +This document is the other side: the surface that has **no oracle**, because `php` cannot run +it at all. A green difftest says nothing about any of it. That single fact shapes every rule +below. + +## The rules the superset lives under + +1. **No oracle ⇒ its own tests.** A superset feature's `tests/aot/expected/*.out` is + hand-written and reviewed, never captured from `php`. Difftest classifies such a case as + manticore-only *only when `php` produces no stdout for it* — so those cases must print + nothing before their first Manticore-only call, or a skip silently becomes a DIFF. +2. **Inert under Zend wherever possible.** Generics are docblocks, FFI bindings and layout + directives are attributes: `php` ignores all of it, so the same file still runs under the + interpreter. That is what keeps a Manticore project a PHP project rather than a fork of the + language. Where a feature genuinely cannot be inert (`Async\`, `Io\Poll`), it is confined to + its own namespace and demand-gated. +3. **Zero cost when unused.** The async runtime is not linked into a program that never names + `Async\`; the cycle collector costs nothing until `gc_collect_cycles()` is called; the + watchdog is one float compare per resume when off. A superset that taxes the programs that + ignore it is a tax on parity. +4. **No php.ini, no PECL, no C extension ABI.** Configuration is the manifest and, for the few + runtime knobs, environment variables. Anything a C extension would have done is either FFI + or a manifest target. + +--- + +## 1. Concurrency + +The largest piece, and the one with the least Zend to compare against: `php` has `Fiber`, and +nothing else here. + +### `Async\` — structured concurrency (`prelude/async.php`, [docs/async.md](async.md)) + +Pure PHP over two primitives; **no new compiler intrinsics**. What makes it a superset rather +than a library you could publish on Packagist is that the primitives underneath it are ours: + +| what | Zend | here | +|---|---|---| +| green threads | `Fiber` (userland scheduler, no I/O integration) | `Fiber` on native `fcontext` (~2.8× a Zend fiber switch), stack pooled, guard page | +| readiness | `stream_select` only | `Io\Poll` — kqueue / epoll / poll, edge-aware, a real reactor | +| blocking I/O | blocks the process | suspends the fiber (§1.3) | +| task ownership | none | every task belongs to a scope; no fire-and-forget | +| cancellation | none | delivered as `CancelledException` **at the suspend point**, sticky, re-raised at every later suspend | +| deadlock | `rc=0`, silently | `DeadlockException` with the full task table | +| signals | `pcntl_signal_dispatch()` you must call | a daemon task parked on `EVFILT_SIGNAL` / `signalfd(2)` | + +Guarantees, the API table and the idioms are in [docs/async.md](async.md). The parts worth +naming *as superset* — nothing in PHP expresses them: + +- **Scope-owned tasks.** `async()` opens a root scope; `TaskGroup::run()` opens a child. A + scope does not return until its children are joined, and the scope lives on the task, so two + concurrent `group()` calls cannot adopt each other's children. +- **Cancellation as a value.** The scope is the source (`$g->cancel()`), a `CancellationToken` + is its read-only half — a helper can observe cancellation without being handed the power to + cause it. `Context::token()/deadline()/remaining()/value()` read the calling task's scope + ambiently, which is where PHP would otherwise reach for a global. +- **Deadlines compose by tightening only.** A 30 s inner `timeout()` inside a 2 s outer one + dies at 2 s. +- **`shield()`** — the one thing that holds cancellation back, so cleanup that must itself + suspend (a close frame, a 503) can run. +- **CSP channels + `select`.** `Channel` is an `IteratorAggregate`, so consumption is a + `foreach` that ends when the channel is closed and drained; `next(): Received` is the comma-ok + form for when `null` is a legal payload. `select` / `selectNow` / `selectWithin` are Go's + three forms, returning a `Selected` object because PHP has no multi-return. +- **`Semaphore` / `Mutex` / `Once`** — a critical section that may suspend in the middle, + which is exactly what a `lock` cannot be in a language with no scheduler. + +### 1.2 Diagnostics you cannot build from outside the engine + +A cooperative loop's two failure modes are "hung" and "one task is holding it", and neither is +observable from library code: + +- `Async\dump()` — every live task, what it is parked on (`io-read fd=9 +deadline`), and + **where it was spawned**. The `file:line` is folded in *by the compiler* at the `Async\` call + site, so it works with no annotation; `->named('http')` only adds a label. +- `Async\dumpOn(SIGQUIT)` — the same table out of an **already hung** process. +- `Async\watchdog(50.0)` / `MANTICORE_ASYNC_WATCHDOG=50` — names the task that held the loop + too long, after the fact (a cooperative loop cannot preempt), rate-limited per task. +- `Async\stats()` — `spawned`/`settled`/`cancelled`/`wakes`/`reactor_waits`/`timer_fires`/ + `watchdog` plus the `live`/`ready`/`io_parked`/`timers` gauges. `live` is the gauge to alert + on; `wakes` climbing with wall time rather than with work is what a spin looks like. + +### 1.3 Transparent I/O — the same stdlib, different blocking behaviour + +This is the piece users notice least and depend on most: `fread`, `fwrite`, +`stream_socket_accept`, `stream_select`, `sleep`, `file_get_contents('https://…')` and +everything layered on them **suspend the fiber instead of the process** when a scheduler is +running. Plain streams *are* the async API; there is no `async fread`. + +Under it, and all superset: + +- **Network setup is async too** — non-blocking `connect(2)`, both TLS handshake directions + driven through `WANT_READ`/`WANT_WRITE` parks (so a TLS *server* serves concurrent clients), + and name resolution over the netpoller: `/etc/hosts`, a per-run cache held by the scheduler, + `search`/`ndots` expansion from `resolv.conf`, then A and AAAA across every nameserver + (two attempts each, TC → retry over TCP), with the blocking `getaddrinfo` walk as the last + resort. +- **Every wait is bounded.** `stream_set_timeout()` sets the stream's timeout for reads *and* + writes; a park that expires records `timed_out` and reports a short read/write. An unbounded + park is a liveness hole, not a feature — a peer that stops reading would otherwise wedge a + fiber, its scope and its fd forever. +- **`accept(2)` failures are classified**, not retried blindly: would-block parks, a peer that + vanished retries, resource exhaustion (EMFILE/ENFILE/ENOBUFS/ENOMEM) backs off on a timer, + and anything else is reported. Under EMFILE the pending connection stays queued, so a + level-triggered listener stays readable — re-arming readiness there is a hot spin that + starves every sibling. +- **`stream_select`/`socket_select` are reactor-native** — one park, one wake-up per readiness + edge, instead of a backoff loop. +- **The seam is one interface**: `\Runtime\AsyncHook`, a handful of callbacks the scheduler + installs. With no scheduler running it is one null check per would-block, which is why the + stdlib pays nothing for being async-aware. + +### 1.4 Process model — `Process\` + +`Process\fork/pid/ppid`, `Process\workers(int)`, `Process\supervise(int, callable)` sit +**beside** `Async\`, deliberately not inside it: none of them runs a scheduler, and forking +must happen before a reactor exists. `supervise()` forks N workers, restarts one that crashes +and forwards `SIGTERM` to the group — the shape a real service needs, and the reason the +pcntl layer exists at all. + +### 1.5 What is deliberately NOT async + +Regular-file I/O blocks the loop: `O_NONBLOCK` is a no-op for regular files on both targets, +and there is no thread pool (rejected: non-atomic rc, a non-thread-safe arena, a process-global +exception slot) and no io_uring (Linux-only would leave macOS behind). `Async\readFile()` / +`Async\writeFile()` chunk and yield. Saying this plainly is part of the superset's contract: +a runtime that claims "everything is async" and blocks anyway is worse than one that names +the exception. + +--- + +## 2. Compile-time directives — attributes ([docs/attributes.md](attributes.md)) + +PHP attributes are inert metadata to Zend, which is exactly what makes them the right carrier +for compiler instructions: the file still parses and runs under `php`. + +| attribute | what it buys | Zend equivalent | +|---|---|---| +| `#[Manticore\Attr\Struct]` | a class with **no object header** — a value laid out like a C struct | none | +| `#[TypeDef]` | a named layout / type alias the compiler reasons about | none | +| `#[Manticore\Attr\RefOut]` | an out-parameter that is auto-vivified for the callee (how `preg_match($s, $p, $m)` fills `$m` natively) | the engine's own C-level by-ref | +| `#[Ffi\Library]` | which native library a binding links against | `ext/ffi` + `php.ini` | +| `#[Ffi\Symbol]` | the C symbol behind a PHP function, per target | `FFI::cdef` string | +| `#[Ffi\CType]` | the C type of a return/param — **not cosmetic**: a C `int` return is not sign-extended by every libc, so `-1` reads as `4294967295` without it | C declaration | +| `#[Ffi\Borrow]` / `BorrowMut` / `Take` / `Give` / `StaticPtr` | ownership across the FFI boundary, so the compiler knows who frees what | nothing — you get a leak or a double free | + +The ownership family is the part with no analogue anywhere in PHP: it lets a native pointer +participate in the same rc discipline as a PHP value instead of being an opaque handle the +programmer must remember to free. + +--- + +## 3. FFI without an extension ([docs/ffi.md](ffi.md)) + +A binding is a PHP function with attributes; the call is a direct native call, not a +marshalling layer. There is no `ext/ffi`, no `FFI::cdef` string to parse at runtime, no +`php.ini` to enable. Libraries are linked **statically** into the output binary, so a compiled +program has no runtime dependency on them — which is what lets `preg_*` ride host PCRE2 and +TLS ride OpenSSL while the binary stays self-contained. + +The trade named honestly: opaque handles are `\Ffi\Ptr`, and memory that crosses the boundary +obeys the ownership attributes above rather than the garbage collector. + +--- + +## 4. Modules and builds ([docs/modules.md](modules.md)) + +PHP has no build system, so this whole layer is superset: + +- **`manticore.json`** — a manifest of *applications* (an entry point → a binary) and + *libraries* (a source tree → a `.o` + a `.sig`). `bin/manticore build` is the whole build. +- **`.sig` module interfaces** — a compiled library's exported signatures, so a consumer type + checks against it without re-reading its source. (Known limit: a `.sig` carries **functions + only** — classes, interfaces, traits, enums and constants do not cross a library boundary yet.) +- **Composer discovery** — a `vendor/` tree resolves as sources to compile, not as an autoload + map to evaluate at runtime. +- **Distribution** — a single static binary; nothing to install on the target, no runtime, no + extension list. + +--- + +## 5. Types ([docs/generics.md](generics.md), [docs/design/type-system-v2.md](design/type-system-v2.md)) + +The type system is inferred and then *used* — for layout, for unboxing, for monomorphization — +rather than checked and discarded: + +- **Docblock generics first** (`@template`, bounds, defaults, `@extends`/`@implements`, + generic traits), because a docblock is inert under Zend. Inline `<…>` is an extension for + code that has already committed to Manticore. +- **Reified class generics** — a real specialized class where you need one. +- **Implicit generics / monomorphization** — a function taking a callback is specialized per + concrete closure with no annotation at all. This is the difference between a callback costing + a dynamic dispatch and costing nothing. +- **`bin/manticore analyze`** — the static checks a compiler can make that an interpreter never + gets the chance to: unsound patterns, hazards that only appear natively. +- **`dump-ast` / `dump-mir` / `dump-llvm-mir` / `dump-sig`** — every stage of the pipeline is + inspectable. ⚠ The dumps do not link the stdlib, so a call into it resolves as `unknown`; + read the final binary when that matters. + +--- + +## 6. Memory ([docs/memory.md](memory.md)) + +PHP gives you a refcount you cannot see and a GC you cannot steer. Here the model is a choice: + +- **hybrid (default)** — escape analysis routes each value to an arena or to refcounting. +- **rc** — deterministic, immediate frees. +- **arena** — bulk reclaim at scope/program exit, no per-object frees; ideal for a + compile-and-exit tool (Manticore's own batch runs). +- `gc_collect_cycles()` — a synchronous **Bacon–Rajan** cycle collector, opt-in and zero cost + until called. Current limits: manual trigger only, and it does not scan static/global roots. + +Selected with `MANTICORE_MEMORY`; finer-grained control (a per-function `#[Arena]`, explicit +scopes) is designed, not wired. + +--- + +## 7. Runtime knobs + +No `php.ini`. The environment variables that exist, and nothing more: + +| variable | effect | +|---|---| +| `MANTICORE_MEMORY` | memory model — `hybrid` / `rc` / `arena` | +| `MANTICORE_ASYNC_WATCHDOG` | loop-hog watchdog threshold, in ms | +| `MANTICORE_PRELUDE` | where the prelude lives (a binary built to a temp path cannot find it argv0-relative) | +| `MANTICORE_STDLIB` / `_O` / `_SIG` | override the stdlib object / signature the driver links | +| `MANTICORE_PROFILE`, `MANTICORE_DEBUG_VERIFY`, `MANTICORE_TYPECHECK`, `MANTICORE_REFLECT_REPORT`, `MANTICORE_UNKNOWN_PROP_TRACE` | compiler diagnostics | +| `MANTICORE_ARENA_ARRAYS`, `MANTICORE_EMPTY_SINGLETON` | allocation experiments | + +--- + +## 8. Designed, not built + +Kept here so the boundary stays honest: + +- **`#[Async]`** — an attribute that turns a call into a spawned `Task`. The only piece of + the async story that cannot be a library: it needs the compiler to split the function and to + type the call site. The typing half already works (`Type::typeArgs` + + `InferCalls::genericReturnType()` resolve `Task` with no reification), so what remains is + a lowering pass and three decisions about methods, inheritance and calling one outside + `async()`. See [docs/design/async-attribute.md](design/async-attribute.md). +- **`#[CompileTime]`** — evaluate a function at compile time. +- **Shared-memory threads** — a future compiler superset, and a much larger one: it invalidates + the non-atomic rc, the arena and the process-global exception slot that everything above is + built on. +- **Off-thread / `io_uring` file I/O** — see §1.5 for why not yet. + +--- + +## Where each piece is verified + +| area | how | +|---|---| +| `Async\`, `Io\Poll`, `Fiber` | `tests/aot/cases/async_*.php` (`bash tests/aot/run.sh -k async`) — manticore-only, hand-written expected | +| transparent I/O bounds | `async_write_timeout`, `async_stream_timeout`, `async_accept_idle_park` | +| accept classification | `async_accept_errno` — asserted through the errno **selector**, never a raw number, so it reads the same on both hosts | +| resolver | `dns_resolv_conf`, `async_dns_resolver` — text-fixture parsers, no network | +| FFI / attributes | the stdlib itself is the test: every `Runtime\Libc` binding rides them | +| modules | `manifest_app`, plus the compiler's own self-host build | +| everything with a Zend answer | `tools/difftest.sh` | + +The network-dependent check lives outside the suite: +`bin/manticore compile examples/async/tls_async_smoke.php` — two overlapped HTTPS fetches to +different hosts plus `dns_get_record` over the parked UDP exchange. diff --git a/examples/async/http_transparent.php b/examples/async/http_transparent.php index cb38d21..9275cc7 100644 --- a/examples/async/http_transparent.php +++ b/examples/async/http_transparent.php @@ -25,6 +25,10 @@ if ($worker === 0) { echo "transparent http on :8080 (", WORKERS, " workers)\n"; } + // DELIBERATELY UNBOUNDED — this file exists to produce a `wrk` number, and an + // acquire/release on the hot path muddies it. A real server needs a ceiling: + // examples/async/server.php shows the shape (a Semaphore taken BEFORE the + // accept). Note the ceiling is PER WORKER — WORKERS multiplies it. async(function () use ($server) { while (true) { $conn = stream_socket_accept($server); // plain accept — auto-suspends diff --git a/examples/async/server.php b/examples/async/server.php index 74aa297..31ae8b3 100644 --- a/examples/async/server.php +++ b/examples/async/server.php @@ -22,6 +22,10 @@ use Async\TaskGroup; const ADDR = 'tcp://127.0.0.1:8081'; +// The ceiling on connections served AT ONCE, per worker. Tasks are cheap, which is +// exactly why an accept loop needs a brake: without one a connection burst becomes +// unbounded tasks, fds and memory, and the first thing to fail is unrelated. +const MAX_CONNS = 256; final class Stats { @@ -74,15 +78,30 @@ function worker(int $index): void // Every connection is a child of THIS scope, so the scope cannot close // while one is in flight — and cancelling it cancels all of them. + // + // The permit is taken BEFORE the accept: at the ceiling this worker simply + // stops accepting, and the queue stays in the kernel's backlog (then in the + // client's SYN retry), which is what backpressure means for a server. It has + // to be released in the CHILD's finally — the parent's would run once, at + // scope exit. acquire() checks cancellation, so SIGTERM still unwinds here. + $gate = new \Async\Semaphore(MAX_CONNS); try { - group(function (TaskGroup $g) use ($server, $stats) { + group(function (TaskGroup $g) use ($server, $stats, $gate) { while (true) { + $gate->acquire(); $conn = \stream_socket_accept($server); if ($conn === false) { + $gate->release(); continue; } \stream_set_blocking($conn, false); - $g->spawn(fn() => serveConnection($conn, $stats)); + $g->spawn(function () use ($conn, $stats, $gate) { + try { + serveConnection($conn, $stats); + } finally { + $gate->release(); + } + }); } }); } catch (CancelledException $e) { diff --git a/prelude/async.php b/prelude/async.php index 3738aa0..bcfccdc 100644 --- a/prelude/async.php +++ b/prelude/async.php @@ -3324,7 +3324,13 @@ function connect(string $addr): \Resource return $conn; } - /** @internal Accept the next connection, suspending until one arrives. */ + /** + * @internal Accept the next connection, suspending until one arrives. + * + * Deliberately UNBOUNDED, unlike read/write below: an idle listener is not a + * stalled peer. Bounding it would force every `while (true) { accept(); }` loop + * to tell "no client yet" apart from "listener broken" on the same return. + */ function accept(\Resource $server): \Resource { while (true) { @@ -3361,7 +3367,13 @@ function read(\Resource $conn, int $length): string // watcher a <= 0 AFTER a readability wake is EOF or a hard error // (ECONNRESET) rather than would-block, so we report closed instead of // spinning forever on a reset connection (no errno binding to tell apart). - $sched->waitReadable($conn); + // The park is BOUNDED (stream_set_timeout's value, else 60 s): raw is not a + // licence to wedge a fiber on a peer that never answers. + $rsecs = $conn->rtimeoutMs > 0 ? (float)$conn->rtimeoutMs / 1000.0 : 60.0; + if ($sched->waitReadableWithin($conn, $rsecs) !== true) { + $conn->timedOut = true; + return ''; + } $buf = $sched->readBuf($length); $n = \Async\sys_recv($fd, $buf, $length, 0); if ($n > 0) { @@ -3388,8 +3400,14 @@ function write(\Resource $conn, string $data): int break; } // n < 0 = back-pressure. Wait for writability, retry ONCE; a second - // <= 0 is a hard error (EPIPE / reset), not would-block. - $sched->waitWritable($conn); + // <= 0 is a hard error (EPIPE / reset), not would-block. The wait is + // BOUNDED — a peer that stops reading must end the write short, not + // park this fiber for the life of the process. + $wsecs = $conn->wtimeoutMs > 0 ? (float)$conn->wtimeoutMs / 1000.0 : 60.0; + if ($sched->waitWritableWithin($conn, $wsecs) !== true) { + $conn->timedOut = true; + break; + } $n = \Async\sys_send($fd, $chunk, $len - $total, 0); if ($n > 0) { $total = $total + $n; diff --git a/prelude/resource.php b/prelude/resource.php index 2b6a045..cbbfa6f 100644 --- a/prelude/resource.php +++ b/prelude/resource.php @@ -227,6 +227,14 @@ final class Resource * with a 0 timeout, so a read with no data ready returns '' instead of waiting. */ public bool $blocking = true; + /** + * SOCKET/TLS WRITE timeout in ms; 0 = the same 60 s default the read side uses. + * stream_set_timeout() sets it together with $rtimeoutMs (php's timeout is the + * stream's, not the read's). A write park that expires records $timedOut and + * reports a SHORT WRITE — never an exception. Appended LAST on purpose: a new + * field ahead of an existing one moves every offset a stale stdlib.o still uses. + */ + public int $wtimeoutMs = 0; public function __construct(int $kind, string $type, int $addr, bool $persistent = false) { diff --git a/src/Compile/Mir/Passes/EmitLlvm.php b/src/Compile/Mir/Passes/EmitLlvm.php index ab94ea3..bfb37be 100644 --- a/src/Compile/Mir/Passes/EmitLlvm.php +++ b/src/Compile/Mir/Passes/EmitLlvm.php @@ -1502,6 +1502,13 @@ private function isFreshStringTemp(Node $node): bool { if ($node->type->kind !== Type::KIND_STRING) { return false; } $k = $node->kind; + // ⚠ A TERNARY is deliberately NOT here, even though + // {@see EmitLlvmControl::retainBorrowedStrArm} gives its borrowed arms a + // +1: an arm that is not itself string-TYPED (a cell/unknown carrier that + // coerces to the string) gets no retain, so treating the result as a fresh + // temp released what nobody owned — preg_full and socket_sendmsg read + // freed bytes. Retaining without releasing leaks; releasing without + // retaining corrupts. return $k === Node::KIND_CONCAT || $k === Node::KIND_CALL || $k === Node::KIND_METHOD_CALL || $k === Node::KIND_STATIC_CALL || $k === Node::KIND_INVOKE; diff --git a/src/Compile/Mir/Passes/EmitLlvmControl.php b/src/Compile/Mir/Passes/EmitLlvmControl.php index 49141b2..5251836 100644 --- a/src/Compile/Mir/Passes/EmitLlvmControl.php +++ b/src/Compile/Mir/Passes/EmitLlvmControl.php @@ -295,6 +295,46 @@ private function emitDeadLabel(): string return $label . ":\n"; } + /** + * A string-typed conditional yields an OWNED (+1) value from EVERY arm, so a + * borrowed arm — an alias, a property / element read — is retained here. + * + * Without it `$out = $out === '' ? $s : ($out . ',' . $s);` stored $s's buffer + * BORROWED: the next iteration's `$s = trim(...)` freed it and the allocator + * handed the same block back, so the accumulated value silently became the + * newest element repeated. Assignment retains a bare alias ($x = $s) but never + * looked inside a conditional, and the mixed shape — one owned arm, one + * borrowed — is the `string|false` idiom, so it is everywhere. Its return-path + * twin is {@see EmitLlvmModule::returnedLocalNames}. + * + * ⚠ The retain is NOT paired with a release: the consumers still treat a + * conditional as borrowed ({@see EmitLlvm::isFreshStringTemp}, + * {@see InsertMemoryOps::isOwnedObj}). Teaching them it is owned was tried and + * reverted — an arm that is not itself string-TYPED (a cell/unknown carrier + * that coerces to the string) gets no retain here, so the release side then + * freed what nobody owned and preg/socket cases read freed bytes. Retaining + * without releasing leaks a string; releasing without retaining corrupts one, + * and the concat arm already leaked before this. Pairing them needs every arm + * covered regardless of its static type — worth doing, not worth guessing at. + * + * A CELL-typed conditional is excluded: its arms are boxed, and a cell's + * ownership is the tag-guarded __mir_cell_drop discipline, not this one. + */ + private function retainBorrowedStrArm(Ternary $n, ?Node $arm, string $i64reg): string + { + if ($arm === null || $n->type->kind !== Type::KIND_STRING) { return ''; } + if ($arm->type->kind !== Type::KIND_STRING) { return ''; } + // A fresh temp already carries the +1; a literal is immortal (its retain is + // a sentinel no-op, but skip the call rather than emit a useless one). + if ($this->isFreshStringTemp($arm) || $arm->kind === Node::KIND_STRING_CONST) { + return ''; + } + $this->rt->needsStrRc = true; + $p = $this->ssa->allocReg(); + return ' ' . $p . ' = inttoptr i64 ' . $i64reg . " to ptr\n" + . ' call void @__mir_rc_retain_str(ptr ' . $p . ")\n"; + } + private function emitTernary(Ternary $n): string { $t = $n; @@ -341,11 +381,13 @@ private function emitTernary(Ternary $n): string } else { $thenVal = $rawCond; } + $out .= $this->retainBorrowedStrArm($n, $t->then ?? $t->cond, $thenVal); $out .= ' store i64 ' . $thenVal . ', ptr ' . $res . "\n"; $out .= ' br label %' . $endLabel . "\n"; $out .= $elseLabel . ":\n"; $out .= $this->emitNode($t->else_); $out .= $wantCell ? $this->boxToCell($t->else_->type) : $this->coerceToI64(); + $out .= $this->retainBorrowedStrArm($n, $t->else_, $this->lastValue); $out .= ' store i64 ' . $this->lastValue . ', ptr ' . $res . "\n"; $out .= ' br label %' . $endLabel . "\n"; $out .= $endLabel . ":\n"; diff --git a/src/Compile/Mir/Passes/LowerFromAst.php b/src/Compile/Mir/Passes/LowerFromAst.php index adc4db5..41164e7 100644 --- a/src/Compile/Mir/Passes/LowerFromAst.php +++ b/src/Compile/Mir/Passes/LowerFromAst.php @@ -1971,12 +1971,19 @@ private function staticPropRef(string $rawClass, string $rawName): ?StaticProp_ * each backed by a module global cell `@__sl_`. A binding * with an initialiser also gets a once-init guard cell so the init * runs on the first call only. + * + * The name is SANITIZED, like a static property's: inside a namespaced + * function the cell would otherwise be `@Ns\fn__sl_x`, and a backslash in + * an unquoted LLVM identifier is a parse error ("expected '=' in global + * variable"). Every static local in this tree happened to sit in a + * global-namespace file, so the whole class of function was unbuildable + * without anyone finding out. */ private function lowerStaticLocal(\Parser\Ast\StaticLocalStmt $stmt): Node { $base = $this->currentLowerClass !== '' - ? '@' . $this->currentLowerClass . '__' . $this->currentLowerFn - : '@' . $this->currentLowerFn; + ? '@' . $this->sanitizeSym($this->currentLowerClass . '__' . $this->currentLowerFn) + : '@' . $this->sanitizeSym($this->currentLowerFn); $nodes = []; foreach ($stmt->decls as $d) { $cell = $base . '__sl_' . $d->name; diff --git a/src/Compile/Mir/Passes/LowerPrelude.php b/src/Compile/Mir/Passes/LowerPrelude.php index 43ea75d..80ed92f 100644 --- a/src/Compile/Mir/Passes/LowerPrelude.php +++ b/src/Compile/Mir/Passes/LowerPrelude.php @@ -581,6 +581,10 @@ private function predefinedConstant(string $name): ?Node 'SOCKET_EBADF' => 9, 'SOCKET_EINVAL' => $isDarwin ? 22 : 22, 'SOCKET_EMFILE' => $isDarwin ? 24 : 24, + 'SOCKET_ENFILE' => $isDarwin ? 23 : 23, + 'SOCKET_ENOBUFS' => $isDarwin ? 55 : 105, + 'SOCKET_ENOMEM' => $isDarwin ? 12 : 12, + 'SOCKET_EPROTO' => $isDarwin ? 100 : 71, 'SOCKET_EACCES' => $isDarwin ? 13 : 13, ]; if (isset($sock[$name])) { return new IntConst($sock[$name], Type::int_()); } diff --git a/src/Runtime/Stdlib/Dns.php b/src/Runtime/Stdlib/Dns.php index 5aa54f0..dc3a428 100644 --- a/src/Runtime/Stdlib/Dns.php +++ b/src/Runtime/Stdlib/Dns.php @@ -10,28 +10,135 @@ // reuse only socket primitives and behave identically on every host. Names are // parsed with compression-pointer support. The record arrays mirror php's shapes. +/** + * resolv.conf's text, '' when it cannot be read. $path is a seam: every parser below + * takes TEXT, so the whole of resolv.conf handling is testable offline, with no + * network and no env var (the AOT runner injects none). + */ +function __mc_resolv_text(string $path = '/etc/resolv.conf'): string +{ + $c = \file_get_contents($path); + return $c === false ? '' : $c; +} + +/** + * Every IPv4 `nameserver` in $text, comma-joined; '' when there is none. A + * comma-joined STRING rather than an array so the value can cross any boundary + * unharmed (the stdlib array-repr rule) — callers explode() it. + */ +function __mc_resolv_nameservers(string $text): string +{ + $out = ''; + foreach (\explode("\n", $text) as $line) { + $line = \trim($line); + if (\strpos($line, 'nameserver ') !== 0) { continue; } + $ip = \trim(\substr($line, 11)); + // IPv4 only here (a v6 nameserver needs a v6 UDP socket path). + if ($ip === '' || \strpos($ip, ':') !== false) { continue; } + $out = $out === '' ? $ip : ($out . ',' . $ip); + } + return $out; +} + +/** + * The search list in $text, comma-joined; '' when there is none. `search a b c` and + * `domain d` both feed it and the LAST such line wins — glibc's rule, and the reason + * a file carrying both is not ambiguous. Capped at 6 entries (glibc's MAXSEARCH), + * lowercased, trailing dots stripped. + */ +function __mc_resolv_search(string $text): string +{ + $out = ''; + foreach (\explode("\n", $text) as $line) { + $line = \trim($line); + if ($line === '' || $line[0] === '#' || $line[0] === ';') { continue; } + $isSearch = \strpos($line, 'search') === 0; + $isDomain = \strpos($line, 'domain') === 0; + if (!$isSearch && !$isDomain) { continue; } + // `searchfoo bar` is not a search line: the keyword must end at a separator. + $sep = \strlen($line) > 6 ? $line[6] : ' '; + if ($sep !== ' ' && $sep !== "\t") { continue; } + $rest = \trim(\substr($line, 6)); + if ($rest === '') { continue; } + $out = ''; + $n = 0; + foreach (\explode(' ', \str_replace("\t", ' ', $rest)) as $tokRaw) { + $tok = \rtrim(\strtolower(\trim((string)$tokRaw)), '.'); + if ($tok === '') { continue; } + $out = $out === '' ? $tok : ($out . ',' . $tok); + $n = $n + 1; + if ($n >= 6) { break; } + if ($isDomain) { break; } // `domain` names exactly one suffix + } + } + return $out; +} + +/** + * `options ndots:N` from $text — last one wins, clamped to 0..15 (glibc's range); + * 1 when absent. Other options (timeout:, attempts:, rotate, edns0) are ignored; + * mapping timeout/attempts onto __mc_dns_query's 2 s × 2 tries is a follow-up. + */ +function __mc_resolv_ndots(string $text): int +{ + $nd = 1; + foreach (\explode("\n", $text) as $line) { + $line = \trim($line); + if (\strpos($line, 'options') !== 0) { continue; } + foreach (\explode(' ', \str_replace("\t", ' ', \substr($line, 7))) as $tokRaw) { + $tok = \trim((string)$tokRaw); + if (\strpos($tok, 'ndots:') !== 0) { continue; } + $v = (int)\substr($tok, 6); + if ($v < 0) { $v = 0; } + if ($v > 15) { $v = 15; } + $nd = $v; + } + } + return $nd; +} + +/** + * The names to try for $host, in query order, comma-joined. + * + * An absolute name (`host.`) is used as-is. Otherwise the `ndots` rule decides which + * comes first: a name with at least $ndots dots is likely already qualified, so the + * bare name leads; a shorter one is a container/cluster short name (`db`, `redis`, + * `service.namespace`) and the search suffixes lead. Inside compose and kubernetes + * that shape is the NORM, and without it every such lookup fell through the async + * resolver into the blocking getaddrinfo walk — stalling the whole loop, in exactly + * the deployment async exists for. + */ +function __mc_dns_candidates(string $host, string $searchCsv, int $ndots): string +{ + $h = \rtrim($host, '.'); + if ($h === '' || \strlen($host) !== \strlen($h)) { + return $h; // rooted (or empty) — no search list applies + } + $suffixes = []; + foreach (\explode(',', $searchCsv) as $sRaw) { + $s = \trim((string)$sRaw); + if ($s !== '') { $suffixes[] = $s; } + } + if (\count($suffixes) === 0) { return $h; } + $qualified = \substr_count($h, '.') >= $ndots; + $out = $qualified ? $h : ''; + foreach ($suffixes as $s) { + $cand = $h . '.' . (string)$s; + $out = $out === '' ? $cand : ($out . ',' . $cand); + } + return $qualified ? $out : ($out . ',' . $h); +} + /** * EVERY IPv4 `nameserver` from /etc/resolv.conf, comma-joined; '8.8.8.8' when the - * file lists none. A comma-joined STRING rather than an array so the value can cross - * any boundary unharmed (the stdlib array-repr rule) — callers explode() it. + * file lists none. * * Resolvers are tried IN ORDER with a per-server timeout, which is what makes a dead * first entry survivable; a single-server resolver failed the whole lookup. */ function __mc_dns_nameservers(): string { - $c = \file_get_contents('/etc/resolv.conf'); - $out = ''; - if ($c !== false) { - foreach (\explode("\n", $c) as $line) { - $line = \trim($line); - if (\strpos($line, 'nameserver ') !== 0) { continue; } - $ip = \trim(\substr($line, 11)); - // IPv4 only here (a v6 nameserver needs a v6 UDP socket path). - if ($ip === '' || \strpos($ip, ':') !== false) { continue; } - $out = $out === '' ? $ip : ($out . ',' . $ip); - } - } + $out = \__mc_resolv_nameservers(\__mc_resolv_text()); return $out === '' ? '8.8.8.8' : $out; } @@ -374,7 +481,25 @@ function __mc_dns_exchange_tcp(string $ns, string $query, float $timeout = 5.0): */ function __mc_dns_query(string $host, int $qtype): string { - $servers = \explode(',', \__mc_dns_nameservers()); + return \__mc_dns_query_via($host, $qtype, \__mc_dns_nameservers()); +} + +/** A reply's RCODE: 0 NOERROR, 3 NXDOMAIN; -1 when the message is too short. */ +function __mc_dns_rcode(string $msg): int +{ + if (\strlen($msg) < 4) { return -1; } + return \ord($msg[3]) & 0x0F; +} + +/** + * {@see __mc_dns_query} against an explicit nameserver list, so a caller walking a + * search list parses resolv.conf ONCE instead of once per candidate. Records the + * reply's RCODE for that walk to read back. + */ +function __mc_dns_query_via(string $host, int $qtype, string $nsCsv): string +{ + \__mc_dns_rcode_hold(true, -1); + $servers = \explode(',', $nsCsv); $query = \__mc_dns_build_query($host, $qtype); foreach ($servers as $nsRaw) { $ns = (string)$nsRaw; @@ -387,8 +512,12 @@ function __mc_dns_query(string $host, int $qtype): string if ($resp === '') { continue; } // timed out — retry this server if (\__mc_dns_truncated($resp)) { $full = \__mc_dns_exchange_tcp($ns, $query, 2.0); - if ($full !== '') { return $full; } + if ($full !== '') { + \__mc_dns_rcode_hold(true, \__mc_dns_rcode($full)); + return $full; + } } + \__mc_dns_rcode_hold(true, \__mc_dns_rcode($resp)); return $resp; } } diff --git a/src/Runtime/Stdlib/Io.php b/src/Runtime/Stdlib/Io.php index b93d913..50df15a 100644 --- a/src/Runtime/Stdlib/Io.php +++ b/src/Runtime/Stdlib/Io.php @@ -319,7 +319,6 @@ function __mc_stream_sendv(\Resource $s, array $chunks): int \poke_i64($iov, $i * 16 + 8, \strlen($sc)); } if (\Runtime\AsyncHook::active()) { - $h = \Runtime\AsyncHook::writable(); $sent = 0; while ($sent < $total) { $w = \Runtime\Libc\sys_writev($s->addr, $iov, $n); @@ -332,15 +331,9 @@ function __mc_stream_sendv(\Resource $s, array $chunks): int if ($w === 0) { break; } // -1 → EWOULDBLOCK (the plain-socket write path never sees a hard // error here: EPIPE/ECONNRESET surface as 0 after the next park). - $h($s); - $w = \Runtime\Libc\sys_writev($s->addr, $iov, $n); - if ($w > 0) { - $sent = $sent + $w; - if ($sent >= $total) { break; } - \__mc_iov_advance($iov, $n, $w); - continue; - } - break; + // The park is BOUNDED: on expiry this reports what it managed to send. + // No post-park retry — a wake is a hint, so the loop head re-writes anyway. + if (\__mc_wait_write($s) === 0) { break; } } \Runtime\Libc\free($iov); return $sent; @@ -392,6 +385,40 @@ function __mc_wait_read(\Resource $s): int return 1; // readable, POLLHUP (recv drains + reports EOF), or poll error } +/** + * The writable twin of {@see __mc_wait_read}: wait (bounded) for a network stream to + * drain before a send. Returns 1 to proceed, 0 on expiry (records $timedOut). + * + * An UNBOUNDED writable park — what every write site did — wedged the writing fiber + * forever on a peer that stops reading (a zero TCP window, the reverse-slowloris): + * the task never settles, its scope never closes, the fd is never released, and + * nothing says why, because from the scheduler's side that task is legitimately + * waiting on I/O. Expiry here is a SHORT WRITE, never an exception — php's fwrite + * reports the byte count and leaves timed_out to say what happened. + */ +function __mc_wait_write(\Resource $s): int +{ + if (\Runtime\AsyncHook::active()) { + $hf = \Runtime\AsyncHook::writableFor(); + $secs = ($s->wtimeoutMs > 0 ? (float)$s->wtimeoutMs : 60000.0) / 1000.0; + // `=== true`: an untyped hook slot returns a tagged cell, read by tag. + if ($hf($s, $secs) === true) { + return 1; + } + $s->timedOut = true; + return 0; + } + $to = $s->blocking ? ($s->wtimeoutMs > 0 ? $s->wtimeoutMs : 60000) : 0; + $rc = \__mc_poll_one($s->addr, true, $to); + if ($rc === 0) { + $s->timedOut = $s->blocking; // a non-blocking full buffer is not a timeout + return 0; + } + // POLLERR/POLLHUP (__mc_poll_one folds both into -1): stop, but do NOT record a + // timeout — a dead fd is not a slow peer, and $timedOut is sticky. + return $rc < 0 ? 0 : 1; +} + /** * Pull ONE chunk from the socket into the buffer. Returns bytes added; 0 means * the peer closed OR the read timed out (both stop the reader; feof vs timed_out @@ -467,18 +494,33 @@ function __mc_stream_fill(\Resource $s, int $want): int // / WANT_WRITE (3) whenever a record is only partly on the wire. That is NOT // end-of-stream — reporting 0 here truncated the response at a record // boundary. Park on the reactor and read the rest. + // + // BOUNDED by an ABSOLUTE deadline (the caller's rtimeoutMs, else 60 s), the way + // __mc_tls_drive_connect bounds the handshake. Both directions parked unbounded + // here, and the loop can ALTERNATE between them — so a per-park budget would let + // a peer that sends half a record and stops live twice-forever, with timed_out + // false. __mc_wait_read above already honours the deadline; this loop must not + // then hand it away. if ($got <= 0 && $s->kind === \Resource::KIND_TLS && \Runtime\AsyncHook::active()) { + $deadline = \__mc_microtime_f() + + ($s->rtimeoutMs > 0 ? (float)$s->rtimeoutMs : 60000.0) / 1000.0; + $rf = \Runtime\AsyncHook::readableFor(); + $wf = \Runtime\AsyncHook::writableFor(); while ($got <= 0) { $err = \Runtime\Openssl\getError($s->ssl, $got); - if ($err === 2) { - $h = \Runtime\AsyncHook::readable(); - $h($s); - } elseif ($err === 3) { - $h = \Runtime\AsyncHook::writable(); - $h($s); - } else { + if ($err !== 2 && $err !== 3) { break; // clean shutdown (SSL_ERROR_ZERO_RETURN) or a hard error } + $left = $deadline - \__mc_microtime_f(); + if ($left <= 0.0) { + $s->timedOut = true; + break; + } + $ready = $err === 2 ? $rf($s, $left) : $wf($s, $left); + if ($ready !== true) { + $s->timedOut = true; + break; + } $got = __mc_transport_recv($s, $buf, $want); } } @@ -1295,10 +1337,14 @@ function fwrite(\Resource $stream, string|array $data, ?int $length = null): int // process never blocks. A writability wake is a HINT: the retry may report // back-pressure again (another writer drained the window first), so a // would-block parks again instead of ending the write short. Only a real - // errno (EPIPE/ECONNRESET) stops us. TLS keeps the retry-once shape — errno - // says nothing about SSL_write's WANT_WRITE. + // errno (EPIPE/ECONNRESET) — or the stream's WRITE DEADLINE — stops us. TLS + // keeps the retry-once shape: errno says nothing about SSL_write's WANT_WRITE. + // + // The deadline is PER PARK, exactly as on the read side (each fill gets a + // whole rtimeoutMs). A peer draining slowly but steadily is alive and php + // keeps writing to it; an absolute per-call budget would fail a big fwrite + // over a slow link where php succeeds. if (\Runtime\AsyncHook::active()) { - $h = \Runtime\AsyncHook::writable(); $total = 0; while ($total < $len) { $chunk = $total === 0 ? $data : \substr($data, $total); @@ -1312,7 +1358,7 @@ function fwrite(\Resource $stream, string|array $data, ?int $length = null): int break; // EPIPE / ECONNRESET — a real error, not back-pressure } } - $h($stream); + if (\__mc_wait_write($stream) === 0) { break; } // deadline: short write if ($stream->kind !== \Resource::KIND_SOCKET) { $n = \__mc_transport_send($stream, $chunk, $len - $total); if ($n > 0) { $total = $total + $n; } else { break; } diff --git a/src/Runtime/Stdlib/Net.php b/src/Runtime/Stdlib/Net.php index a4780d7..8bf6909 100644 --- a/src/Runtime/Stdlib/Net.php +++ b/src/Runtime/Stdlib/Net.php @@ -142,15 +142,19 @@ function __mc_poll_one(int $fd, bool $forWrite, int $timeoutMs): int } /** - * php.net's stream_set_timeout: bound how long a read on $stream waits for data. + * php.net's stream_set_timeout: bound how long an operation on $stream waits. * 0/0 keeps the default (php's default_socket_timeout, 60s); a positive value caps * each read at that long, after which the read returns empty and * stream_get_meta_data()['timed_out'] is true. + * + * The value is the STREAM's timeout, not the read's — php applies it to writes too, + * so a peer that stops draining bounds the send instead of wedging it forever. */ function stream_set_timeout(\Resource $stream, int $seconds, int $microseconds = 0): bool { $ms = $seconds * 1000 + \intdiv($microseconds, 1000); $stream->rtimeoutMs = $ms > 0 ? $ms : 0; + $stream->wtimeoutMs = $ms > 0 ? $ms : 0; return true; } @@ -352,26 +356,46 @@ function __mc_resolve_async(string $host): string if ($cached !== '') { return $cached; } - $ip = \__mc_resolve_query($host, 1); // A - if ($ip === '') { + // resolv.conf is read ONCE per lookup and its pieces threaded down — the search + // walk would otherwise re-read and re-parse it for every candidate and qtype. + $text = \__mc_resolv_text(); + $ns = \__mc_resolv_nameservers($text); + if ($ns === '') { $ns = '8.8.8.8'; } + $cands = \__mc_dns_candidates($host, \__mc_resolv_search($text), \__mc_resolv_ndots($text)); + foreach (\explode(',', $cands) as $cRaw) { + $qname = (string)$cRaw; + if ($qname === '') { continue; } + $ip = \__mc_resolve_query($qname, 1, $host, $ns); // A + if ($ip !== '') { return $ip; } // AAAA next: an IPv6-only name is common enough (and the literal we return // goes straight back into getaddrinfo, which takes v6 literals as happily as // v4 — so nothing downstream needs to know). - $ip = \__mc_resolve_query($host, 28); // AAAA + $ip = \__mc_resolve_query($qname, 28, $host, $ns); // AAAA + if ($ip !== '') { return $ip; } + // Keep walking on NXDOMAIN *and* on a NOERROR with no address (a name with + // only MX/TXT is not an address, and a search list exists to keep going). + // Stop when NOBODY answered: a timeout must not be multiplied by the number + // of suffixes — that turns one dead resolver into a minutes-long lookup. + if (\__mc_dns_rcode_last() < 0) { break; } } - return $ip; + return ''; } /** - * One QTYPE's worth of resolution: query, take the first address record, and cache it - * under its own TTL. '' when the type has no answer. + * One QTYPE's worth of resolution: query $qname, take the first address record, and + * cache it under its own TTL. '' when the type has no answer. + * + * $cacheKey is the name the CALLER asked for, never the expanded candidate: caching + * each suffix separately would fill the table with per-suffix duplicates and miss on + * the next lookup of the same short name. */ -function __mc_resolve_query(string $host, int $qtype): string +function __mc_resolve_query(string $qname, int $qtype, string $cacheKey, string $nsCsv): string { - $resp = \__mc_dns_query($host, $qtype); + $resp = \__mc_dns_query_via($qname, $qtype, $nsCsv); if ($resp === '') { return ''; } + $host = $cacheKey; /** @var array> $recs */ $recs = \__mc_dns_parse($resp, $qtype); foreach ($recs as $rec) { @@ -915,6 +939,32 @@ function __mc_net_errno(bool $write, int $val): int return $e; } +/** + * The RCODE of the last DNS query, same idiom, -1 when NO server answered — which + * is what stops a search-list walk before a dead resolver is multiplied by the + * suffix count. + * + * ⚠ It lives HERE, in a global-namespace file, rather than beside the resolver in + * Dns.php: a `static` inside a NAMESPACED function used to emit `@Ns\fn__sl_x`, + * which is not valid LLVM. That is fixed in the compiler + * ({@see \Compile\Mir\Passes\LowerFromAst::lowerStaticLocal}) — but a compiler + * generation WITHOUT the fix cannot build a tree that needs it, so `bin/build` + * on any older binary would die here and only a cold seed could recover. The + * tree stays buildable by every generation; the fix stays for user code. + */ +function __mc_dns_rcode_hold(bool $write, int $val): int +{ + static $r = -1; + if ($write) { $r = $val; } + return $r; +} + +/** The RCODE of the last query, or -1 when nobody answered. */ +function __mc_dns_rcode_last(): int +{ + return \__mc_dns_rcode_hold(false, 0); +} + /** strerror($errno) as an owned string ('' for 0). */ function __mc_errno_msg(int $errno): string { @@ -1259,29 +1309,63 @@ function stream_socket_accept(\Resource $server, ?float $timeout = null) // (the listener is non-blocking, so a lost race is EWOULDBLOCK, not a stall). // A $timeout is honoured by the reactor's BOUNDED wait; it used to fall into // the branch below, whose poll(2) blocked the whole scheduler. The deadline - // is absolute, so a re-park after a lost race cannot extend it. + // is absolute, so a re-park after a lost race cannot extend it — a deadline + // of -1.0 means there is none. + // + // WHY THE FAILURE IS CLASSIFIED ({@see __mc_accept_class}): this loop used to + // re-park on ANY accept failure. Out of descriptors that is a hot spin, not a + // wait — accept(2) fails while the pending connection stays queued, so the + // listener stays readable and the park returns immediately, forever. It does + // not even trip the watchdog (it suspends every iteration); it shows only as + // an exploding `wakes` counter, while every sibling task starves. + $deadline = ($timeout !== null && $timeout >= 0.0) + ? \__mc_microtime_f() + $timeout : -1.0; $fd = \Runtime\Libc\sys_accept($server->addr, \int_to_ptr(0), \int_to_ptr(0)); - if ($fd < 0 && $timeout !== null && $timeout >= 0.0) { - $hf = \Runtime\AsyncHook::readableFor(); - $deadline = \__mc_microtime_f() + $timeout; - while ($fd < 0) { - $left = $deadline - \__mc_microtime_f(); - if ($left <= 0.0) { - return false; - } - // `=== true`: the hook is an untyped slot, so its result arrives as - // a tagged cell and is read by tag, never as a raw word. - if ($hf($server, $left) !== true) { - return false; + $backoff = 0.0; + $fast = 0; + while ($fd < 0) { + $e = \__mc_errno(); + $cls = \__mc_accept_class($e); + if ($cls === 3) { + \__mc_net_errno(true, $e); // never a SILENT false + return false; + } + if ($cls === 1 && $fast < 8) { + $fast = $fast + 1; + $fd = \Runtime\Libc\sys_accept($server->addr, \int_to_ptr(0), \int_to_ptr(0)); + continue; + } + if ($cls === 2) { + // Sleep, do NOT re-arm readiness: the listener stays readable until + // an accept succeeds, so readiness carries no information here. + \__mc_net_errno(true, $e); + $backoff = \__mc_accept_backoff($backoff); + if ($deadline >= 0.0) { + $left = $deadline - \__mc_microtime_f(); + if ($left <= 0.0) { return false; } + if ($backoff > $left) { $backoff = $left; } } + $sleep = \Runtime\AsyncHook::sleeper(); + $sleep($backoff); $fd = \Runtime\Libc\sys_accept($server->addr, \int_to_ptr(0), \int_to_ptr(0)); + continue; } - } else { - $h = \Runtime\AsyncHook::readable(); - while ($fd < 0) { + if ($deadline >= 0.0) { + $left = $deadline - \__mc_microtime_f(); + if ($left <= 0.0) { return false; } + $hf = \Runtime\AsyncHook::readableFor(); + // `=== true`: the hook is an untyped slot, so its result arrives as + // a tagged cell and is read by tag, never as a raw word. + if ($hf($server, $left) !== true) { return false; } + } else { + $h = \Runtime\AsyncHook::readable(); $h($server); - $fd = \Runtime\Libc\sys_accept($server->addr, \int_to_ptr(0), \int_to_ptr(0)); } + // Readiness reached ⇒ the overload episode (if any) is over. Both + // counters must reset, or one EMFILE burst slows the loop for good. + $backoff = 0.0; + $fast = 0; + $fd = \Runtime\Libc\sys_accept($server->addr, \int_to_ptr(0), \int_to_ptr(0)); } } else { if ($timeout !== null && $timeout >= 0.0) { @@ -1292,6 +1376,7 @@ function stream_socket_accept(\Resource $server, ?float $timeout = null) } $fd = \Runtime\Libc\sys_accept($server->addr, \int_to_ptr(0), \int_to_ptr(0)); if ($fd < 0) { + \__mc_net_errno(true, \__mc_errno()); return false; } } diff --git a/src/Runtime/Stdlib/Sockets.php b/src/Runtime/Stdlib/Sockets.php index 69f4344..1cea726 100644 --- a/src/Runtime/Stdlib/Sockets.php +++ b/src/Runtime/Stdlib/Sockets.php @@ -36,6 +36,7 @@ function __mc_sock_fd(\Socket $s): int * 0 SOL_SOCKET 1 SO_ERROR 2 SO_RCVTIMEO 3 SO_SNDTIMEO 4 AF_INET6 * 5 O_NONBLOCK 6 F_GETFL 7 F_SETFL 8 SO_LINGER 9 SO_REUSEADDR * 10 EWOULDBLOCK 11 EAGAIN 12 EINPROGRESS 13 ETIMEDOUT + * 14 EMFILE 15 ENFILE 16 ECONNABORTED 17 ENOBUFS 18 ENOMEM 19 EPROTO */ function __mc_sock_const(int $which): int { @@ -52,6 +53,9 @@ function __mc_sock_const(int $which): int static $eAgain = 0; static $eInProgress = 0; static $eTimedOut = 0; + static $eConnAborted = 0; + static $eNoBufs = 0; + static $eProto = 0; if ($ready === 0) { $isDarwin = \__mc_host_is_darwin(); @@ -71,6 +75,11 @@ function __mc_sock_const(int $which): int // MEASURED: Darwin ETIMEDOUT 60 (cc probe + php SOCKET_ETIMEDOUT); // Linux asm-generic/errno.h 110 (glibc and musl agree). $eTimedOut = $isDarwin ? 60 : 110; + // The accept(2) classifier's set. MEASURED the same way; EMFILE/ENFILE/ + // ENOMEM live in asm-generic/errno-BASE.h, whose numbers Darwin shares. + $eConnAborted = $isDarwin ? 53 : 103; + $eNoBufs = $isDarwin ? 55 : 105; + $eProto = $isDarwin ? 100 : 71; $ready = 1; } @@ -87,7 +96,13 @@ function __mc_sock_const(int $which): int if ($which === 10) { return $eWouldBlock; } if ($which === 11) { return $eAgain; } if ($which === 12) { return $eInProgress; } - return $eTimedOut; + if ($which === 13) { return $eTimedOut; } + if ($which === 14) { return 24; } // EMFILE — 24 everywhere + if ($which === 15) { return 23; } // ENFILE — 23 everywhere + if ($which === 16) { return $eConnAborted; } + if ($which === 17) { return $eNoBufs; } + if ($which === 18) { return 12; } // ENOMEM — 12 everywhere + return $eProto; } /** @@ -301,6 +316,54 @@ function __mc_sock_wouldblock(): bool return $e === 0 || $e === \__mc_sock_const(10) || $e === \__mc_sock_const(11) || $e === 4; } +/** + * How an accept(2) failure must be handled: + * 0 would-block → park on readiness (today's only behaviour) + * 1 transient → retry NOW. EINTR, and the peer that vanished between its SYN + * and our accept (ECONNABORTED/EPROTO): no readiness edge is + * coming for a connection that is already gone. + * 2 overload → out of a process- or host-wide resource (EMFILE/ENFILE/ + * ENOBUFS/ENOMEM). accept(2) fails while the pending connection + * STAYS QUEUED, so a level-triggered listener stays readable: the + * park returns at once, accept fails again, and the loop spins + * without ever suspending — a task that starves every sibling. + * Back off on a timer and keep serving what is already open. + * 3 fatal → EBADF/EINVAL/ENOTSOCK … the listener itself is broken. + * + * ⚠ Kept SEPARATE from {@see __mc_sock_wouldblock}, and with the OPPOSITE polarity: + * there an unknown errno parks (a read must not truncate live data on an errno + * nobody set), here an unknown errno is fatal (on accept the alternative is an + * infinite loop). Folding EMFILE into the would-block set would make every + * socket_read/socket_send spin on it too. + */ +function __mc_accept_class(int $errno): int +{ + if ($errno === 0 || $errno === \__mc_sock_const(10) || $errno === \__mc_sock_const(11)) { + return 0; + } + if ($errno === 4 || $errno === \__mc_sock_const(16) || $errno === \__mc_sock_const(19)) { + return 1; + } + if ($errno === \__mc_sock_const(14) || $errno === \__mc_sock_const(15) + || $errno === \__mc_sock_const(17) || $errno === \__mc_sock_const(18)) { + return 2; + } + return 3; +} + +/** + * The next overload backoff step: 1 ms, doubling to a 50 ms ceiling. Slower than + * __mc_select_poll_park's 0.2 ms/10 ms on purpose — fd exhaustion clears when some + * other connection closes, not when the next packet lands, so the point is to stop + * burning wakes rather than to keep latency low. + */ +function __mc_accept_backoff(float $prev): float +{ + if ($prev <= 0.0) { return 0.001; } + $next = $prev * 2.0; + return $next > 0.05 ? 0.05 : $next; +} + /** Mark $fd non-blocking under a scheduler, once, so a would-block can be parked. */ function __mc_sock_async_prep(\Socket $socket): bool { @@ -343,9 +406,25 @@ function socket_accept(\Socket $socket): \Socket|false $async = \__mc_sock_async_prep($socket); $fd = \Runtime\Libc\sys_accept($socket->fd, \int_to_ptr(0), \int_to_ptr(0)); // Under a scheduler a would-block PARKS instead of blocking the loop; a wake is - // a hint, so the accept is retried until it lands or the wait expires. - while ($fd < 0 && $async && \__mc_sock_wouldblock()) { + // a hint, so the accept is retried until it lands or the wait expires. A + // transient failure (the peer gave up between its SYN and this accept) retries + // straight away — no readiness edge is coming for a connection already gone — + // but only a bounded number of times, or a listener stuck in that state spins. + // + // EMFILE is deliberately NOT backed off here: php's socket_accept reports the + // errno and returns false, and ext/sockets leaves the serving policy to the + // caller. The stream layer, which owns its accept loop, does back off. + $fast = 0; + while ($fd < 0 && $async) { + $cls = \__mc_accept_class(\__mc_errno()); + if ($cls === 1 && $fast < 8) { + $fast = $fast + 1; + $fd = \Runtime\Libc\sys_accept($socket->fd, \int_to_ptr(0), \int_to_ptr(0)); + continue; + } + if ($cls !== 0) { break; } if (!\__mc_sock_park($socket->fd, false, 0.0)) { break; } + $fast = 0; $fd = \Runtime\Libc\sys_accept($socket->fd, \int_to_ptr(0), \int_to_ptr(0)); } if ($fd < 0) { diff --git a/tests/aot/cases/async_accept_errno.php b/tests/aot/cases/async_accept_errno.php new file mode 100644 index 0000000..74865e0 --- /dev/null +++ b/tests/aot/cases/async_accept_errno.php @@ -0,0 +1,49 @@ + $c) { + echo $names[$i], ' => ', $class[__mc_accept_class($c)], "\n"; +} + +// The classifier is SEPARATE from __mc_sock_wouldblock, and deliberately of the +// opposite polarity: there an unknown errno parks (a read must not truncate live +// data), here it is fatal (the alternative is an infinite loop). Widening the +// would-block set would make every socket_read/socket_send spin on EMFILE too. +echo "emfile-is-overload: ", (__mc_accept_class(__mc_sock_const(14)) === 2 ? 'yes' : 'no'), "\n"; +echo "unknown-is-fatal: ", (__mc_accept_class(9999) === 3 ? 'yes' : 'no'), "\n"; + +// 1 ms, doubling, capped at 50 ms. +$steps = []; +$b = 0.0; +for ($i = 0; $i < 8; $i = $i + 1) { + $b = __mc_accept_backoff($b); + $steps[] = (string)(int)round($b * 1000.0); +} +echo "backoff-ms: ", implode(',', $steps), "\n"; +echo "done\n"; diff --git a/tests/aot/cases/async_accept_idle_park.php b/tests/aot/cases/async_accept_idle_park.php new file mode 100644 index 0000000..bf12558 --- /dev/null +++ b/tests/aot/cases/async_accept_idle_park.php @@ -0,0 +1,90 @@ +await(); + $c = $client->await(); + $t = $ticker->await(); + $wakes = \Async\stats()['wakes'] - $before; + return $a . ' ' . $c . ' ticks=' . (string)$t + . ' parked=' . ($wakes < 200 ? 'yes' : 'no'); +}); +echo "listening: ", ($server !== false ? 'yes' : 'no'), "\n"; +echo $report, "\n"; + +// The bounded branch of the same loop still expires, and still leaves the scheduler +// live (the deadline is absolute — a re-park after a lost race cannot extend it). +$bounded = async(function () use ($server): string { + $waiter = spawn(function () use ($server): string { + $t0 = microtime(true); + $conn = stream_socket_accept($server, 0.15); + return ($conn === false ? 'false' : 'conn') + . ' bounded=' . ((microtime(true) - $t0) < 1.0 ? 'yes' : 'no'); + }); + $ticker = spawn(function (): int { + $n = 0; + for ($i = 0; $i < 3; $i = $i + 1) { + \Async\delay(0.01); + $n = $n + 1; + } + return $n; + }); + return $waiter->await() . ' ticks=' . (string)$ticker->await(); +}); +echo $bounded, "\n"; + +fclose($server); +echo "done\n"; diff --git a/tests/aot/cases/async_write_timeout.php b/tests/aot/cases/async_write_timeout.php new file mode 100644 index 0000000..7e28012 --- /dev/null +++ b/tests/aot/cases/async_write_timeout.php @@ -0,0 +1,100 @@ +await(); + $ticks = $ticker->await(); + $deaf->await(); + return $w . ' ticks=' . (string)$ticks; +}); +echo "listening: ", ($server !== false ? 'yes' : 'no'), "\n"; +echo $report, "\n"; + +// stream_set_timeout is the stream's, not the read's: both directions carry it. +stream_set_timeout($server, 3); +echo 'r=', $server->rtimeoutMs, ' w=', $server->wtimeoutMs, "\n"; + +fclose($server); +echo "done\n"; diff --git a/tests/aot/cases/dns_resolv_conf.php b/tests/aot/cases/dns_resolv_conf.php new file mode 100644 index 0000000..e34c7d6 --- /dev/null +++ b/tests/aot/cases/dns_resolv_conf.php @@ -0,0 +1,72 @@ +step(1), " ", $s->step(2), "\n"; + +// A second instance shares the method's static, as in PHP. +$t = new Seq(); +echo $t->step(10), "\n"; diff --git a/tests/aot/expected/async_accept_errno.out b/tests/aot/expected/async_accept_errno.out new file mode 100644 index 0000000..879f6c7 --- /dev/null +++ b/tests/aot/expected/async_accept_errno.out @@ -0,0 +1,15 @@ +nobody-set => park +EWOULDBLOCK => park +EAGAIN => park +EINTR => retry +ECONNABORTED => retry +EPROTO => retry +EMFILE => backoff +ENFILE => backoff +ENOBUFS => backoff +ENOMEM => backoff +bogus => fatal +emfile-is-overload: yes +unknown-is-fatal: yes +backoff-ms: 1,2,4,8,16,32,50,50 +done diff --git a/tests/aot/expected/async_accept_idle_park.out b/tests/aot/expected/async_accept_idle_park.out new file mode 100644 index 0000000..c167003 --- /dev/null +++ b/tests/aot/expected/async_accept_idle_park.out @@ -0,0 +1,4 @@ +listening: yes +accepted connected ticks=5 parked=yes +false bounded=yes ticks=3 +done diff --git a/tests/aot/expected/async_write_timeout.out b/tests/aot/expected/async_write_timeout.out new file mode 100644 index 0000000..f7f78ad --- /dev/null +++ b/tests/aot/expected/async_write_timeout.out @@ -0,0 +1,4 @@ +listening: yes +short=yes timed_out=yes bounded=yes ticks=4 +r=3000 w=3000 +done diff --git a/tests/aot/expected/dns_resolv_conf.out b/tests/aot/expected/dns_resolv_conf.out new file mode 100644 index 0000000..1651163 --- /dev/null +++ b/tests/aot/expected/dns_resolv_conf.out @@ -0,0 +1,28 @@ +ns: 10.0.0.1,10.0.0.2 +search: a.example,b.example +ndots: 2 +domain-only: corp.example +search-then-domain: corp.example +domain-then-search: x.example,y.example +not-a-search-line: [] +trailing-dots: a.example,b.example +capped: s1,s2,s3,s4,s5,s6 +ndots-default: 1 +ndots-clamped: 15 +ndots-last-wins: 5 +ndots-tab: 4 +short: web.a.example,web.b.example,web +qualified: a.b.c,a.b.c.a.example,a.b.c.b.example +ndots2-short: svc.ns.a.example,svc.ns.b.example,svc.ns +rooted: host +no-search: web +rcode-noerror: 0 +rcode-nxdomain: 3 +rcode-short: -1 +file-ns: 192.0.2.53 +file-search: svc.cluster.local,cluster.local +file-candidates: redis.svc.cluster.local,redis.cluster.local,redis +missing-file: [] +live-nonempty: yes +live-no-spaces: yes +done diff --git a/tests/aot/expected/static_local_namespaced.out b/tests/aot/expected/static_local_namespaced.out new file mode 100644 index 0000000..ad59152 --- /dev/null +++ b/tests/aot/expected/static_local_namespaced.out @@ -0,0 +1,5 @@ +123 +-1 +42 +101 103 +113