Conversation
jserv
changed the title
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
A MAP_SHARED file mapping is backed by a live host mmap overlay onto the guest slab. Once anything truncates that file the page is gone, and the next host-side access raises SIGBUS. elfuse installs no handler, so a guest can kill the host with three ordinary syscalls: mmap a file shared, truncate it, then hand the address to something that reads it. This adds the recovery mechanism and installs the handler. Nothing uses the pad yet, so behavior is unchanged; the callers follow. HOST_SIGBUS_GUARD arms a per-thread setjmp landing site around an access. It uses the mask-free _setjmp because the handler carries SA_NODEFER, which leaves SIGBUS unblocked and so leaves no mask to restore; sigsetjmp would add two sigprocmask traps to every guarded copy. The macro carries the rules a caller has to meet, the load-bearing one being that the guarded statement may hold no lock, since the jump skips every release. The jump happens from a resume function the handler redirects the interrupted context to, not from the handler itself. Jumping straight out never returns through the wrapper a sanitizer installs, which leaves ThreadSanitizer's per-thread signal bookkeeping inconsistent: measured, the first recovered SIGBUS worked and the second deadlocked. An unguarded SIGBUS is told apart by si_addr, measured on Darwin to carry the faulting address for a hardware fault and 0 for an external kill -BUS. si_code cannot make that call, since both report 1 and sys/signal.h marks BUS_ADRERR and BUS_OBJERR NOTIMP. Sent signals are re-raised so kill -BUS still kills on the first one; genuine faults return into the restored disposition, so the process dies on the original siginfo with si_addr and the faulting PC intact.
Every syscall taking a path pays guest_read_path, which copies the string out of guest memory with a direct memcpy. When the page has lost its backing the copy raises SIGBUS and takes the host down; measured, open() on such a path killed elfuse with exit 138. The copy helpers now run their memcpy and memchr inside the recovery pad and report the fault instead, so the guest sees EFAULT. guest_read_str_small and guest_read_str separate a fault from an ordinary failure by returning -2 rather than -1. The distinction matters to one caller: guest_read_path retries a -1 through a larger buffer, which for a fault would only fault again at the same address. Both are negative, so the callers testing < 0 are unaffected. The fixture maps a file shared, truncates it from a child, and hands the address to open(). It is added to the matrix, whose expected counts move by one.
A futex word can live in a MAP_SHARED file mapping, which is how a process-shared mutex in a shm file is laid out, and the atomics that read and update it run from host user mode. Once the file is truncated the page is gone and an unguarded atomic kills the host, so a guest can take elfuse down with a futex call on a range it truncated itself. Two leaf helpers wrap the load and the compare-exchange in the recovery pad, and every futex path that touched the word directly now goes through them: WAIT, WAITV, REQUEUE, WAKE_OP, and the PI lock, trylock and unlock. Each reports EFAULT the same way it already reported an unresolvable uaddr. Wrapping at the leaf is what keeps the bucket locks intact. The jump lands inside the helper rather than unwinding past a caller, so every path that holds a lock still reaches its own unlock, including the ones that also unlink a waiter and destroy its condvar. The CAS helper takes NULL where the caller retries regardless of who won the race, so those sites do not carry a result nobody reads.
getrandom and read(/dev/urandom) both filled the guest buffer through a resolved guest pointer, so a vanished page killed the host on the store. Measured, both exited 138; the readv form did too, by a second path. Neither can use the recovery pad. arc4random_buf takes a libc lock and the urandom cache copies under its per-fd lock, and the jump out of a guard skips every release: wrapping arc4random_buf stranded that lock and hung the next fork. They fill a host buffer instead and hand it to guest_write, which does the same copy with the pad around a plain memcpy, so the fault surfaces as EFAULT. The staging chunk is clamped to the end of the current guest page. A page is the granularity at which backing vanishes, so a chunk that cannot straddle one either lands whole or faults whole, and the byte count returned is never short of what reached the guest. readv on /dev/urandom needs the same treatment per entry, so validate_iov_total now hands back the iovec array it already walked rather than making the caller read it a second time. The count is capped before the array is sized, and the heap copy is freed on both exits. Both entropy paths reuse the proved slice_clamp for the chunking rather than spelling the same minimum twice.
process_vm_readv and writev memmove between two guest buffers from host user mode, so either end can be a vanished page and an unguarded copy kills the host. A bare guard is not enough here. The chunk runs to the end of the contiguous region and can be megabytes, and a guard can only say whether the whole thing survived, so one vanished page would discard everything the copy had already written from the reported total. The copy reports its own progress instead, and the caller advances the iovec offsets by what actually moved before returning the partial result Linux would. Progress is only meaningful if a step cannot itself be torn, so each is cut at the next page boundary on whichever side reaches one first and lies within a single page at both ends. A page is the granularity at which backing vanishes, so such a step either lands whole or faults whole. A flat stride would not do: unaligned against the real boundaries by up to a page, it could tear and lose what it had written. The step uses memmove because sys_process_vm accepts the caller's own pid, and then both iovecs resolve into one address space and a guest can hand it overlapping ranges. Copying those forward is undefined behavior, and measured against the pre-fix build it corrupted guest memory at every overlap delta tried. Exactness has one gap, and it is a race rather than a miscount: a truncate landing mid-step takes the page after part of that step is written. The error is bounded by one page and is always an under-report, which callers absorb because a short count means resume-from-here and re-copying is idempotent. si_addr cannot close it, since nothing specifies the order memmove touches its range in. The overlap case in test-process-vm pins the count and that nothing outside the destination is written. It deliberately pins nothing about the content: that is unspecified, and measured against the qemu reference kernel it matches neither memmove nor a page-wise move.
gdb_invalidate_written_code is the one host-user-mode touch of guest memory left outside the recovery pad, and it would fault the same way a memcpy does if the range sat on a vanished MAP_SHARED page. It is left that way deliberately: reaching it requires --gdb, so no guest can drive it, and the ranges it flushes are code the debugger just wrote rather than file-backed data. Recording that here so the next audit of these call sites does not have to re-derive it, and does not read the omission as an oversight to copy.
The guardrail measured getpid, clock_gettime and a one-byte urandom read. All three are served inline by the shim or the vDSO and never touch the guest copy helpers, so nothing watched the helpers themselves. Arming the SIGBUS pad with sigsetjmp instead of _setjmp cost 45 percent on every path-taking syscall, 4.2 us to 6.1 us on stat, and the guardrail stayed green: no lane covered it, and the existing ceilings sit 4x over observed anyway. The new stat-path lane is the densest guest_read_path caller a one-line bench can reach, and it is checked as a ratio to getpid from the same run. It is the only lane whose cost is a slope rather than a step: the other three fall off a fast path and land past any plausible ceiling, while this one moves by a percentage, which no absolute number catches without being tuned to one machine. The three existing ceilings are left wide for that reason. run_case discards a warmup pass. Without it the first case absorbed every one-time cost in the run and getpid read 62 to 72 ns cold against 50 warm, a 40 percent swing on the very lane the ratio divides by. A failing pass is re-measured once before it is reported, because the lanes do not degrade together: getpid stays inside the process while stat makes a real host filesystem call, so filesystem load moves one and not the other. A clean tree measured 205x against a 105x ceiling purely because Gatekeeper was scanning binaries a fresh build had produced, and the same binary came back at 87x once quiet. Only a measurement over its limit is retried; a bench that exited non-zero or a lane that reported no number fails for a reason a second run reproduces. Verified: restoring the sigsetjmp regression fails the static lane on both passes, 124.6x then 128.7x. The dynamic lane only moved 407x to 488x, because sysroot resolution dilutes a per-copy cost across 13.6 us; it is kept for gross regressions and noted as the weaker detector.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary by cubic
Catches host SIGBUS around direct guest-memory touches so MAP_SHARED truncations no longer kill the host. Previously a vanished page aborted the process; now syscalls return EFAULT with precise partial counts, recovered faults resume outside the handler, and sent SIGBUS is re-raised.
Written for commit f91b895. Summary will update on new commits.