The defect
Every blocking syscall identifies "the thread that is blocking" by the scheduler's recorded current
thread, never by the thread that is actually executing:
sched.block_current();
if let Some(thread) = sched.current_thread_mut() {
thread.blocked_in_syscall = true;
}
...
let still_blocked = scheduler::with_scheduler(|sched| {
sched.current_thread_mut().map(|t| t.state == ThreadState::Blocked)
});
If the recorded current has skewed away from the executing thread even once, the prologue stamps
Blocked + blocked_in_syscall onto a foreign thread's record and the HLT loop then polls that
foreign record. The victim waits on a condition its own waker will never touch — a permanent hang, and
the skew is now permanent because the wrong record carries the flag. This is the mechanism behind a
run-3 residual observed while gating #545/#557-adjacent work: idle's record carried
blocked_in_syscall, which nothing legitimately sets (idle never issues a blocking syscall), and that
flag is what steered a later tick into the kernel-frame save branch.
Why this is a separate issue, not folded into #557's fixes
PR #558 (fix/ctx-routing-and-abort-rollback, merged) removed two concrete skew sources inside
kernel/src/interrupts/context_switch.rs (rollback over-restore on save failure, and the
SignalDeliveryResult::Terminated arm leaving the frame/CR3 committed). Those fixes make the recorded
current correct at the moment they run. This issue is about the fact that "recorded current always
names the thread actually executing on this CPU" is not an enforced, tree-wide invariant — nothing
prevents a different writer, elsewhere in the tree, from introducing the same class of skew again.
1. There is no independent identity to switch to. Both candidate "executing thread" sources are
the same recorded state, written by the same dispatcher:
| source |
what it reads |
written by |
scheduler::current_thread_id() (scheduler.rs:3544) |
cpu_state[cpu].current_thread |
schedule() / switch_to_idle() / abort_dispatch_and_resume() |
per_cpu::current_thread() (per_cpu.rs:380) |
GS-relative current-thread pointer |
per_cpu::set_current_thread(), same call sites |
The syscall entry point itself resolves the caller the same way — handler.rs:585 does
let current_thread_id = match crate::task::scheduler::current_thread_id(). So "capture the id at
syscall entry" captures the recorded current too; it narrows the window (one read instead of N) but
does not make the identity authoritative. A real fix needs an identity the dispatcher cannot get wrong:
Linux's shape is current derived from the kernel stack (thread_info at the stack base, i.e. RSP & ~(THREAD_SIZE-1)), or equivalently a per-thread cookie written when the kernel stack is allocated and
read back from TSS.RSP0/RSP. Breenix has neither today — introducing one is a design change to
thread creation, kernel_stack_top publication and every RSP0 publisher.
2. The call-site fan-out is large and spread across the syscall surface. Direct
current_thread_mut() uses today:
| file |
current_thread_mut() |
notes |
kernel/src/syscall/handlers.rs |
45 |
read/write/waitpid/pause/nanosleep prologues + epilogues |
kernel/src/syscall/socket.rs |
20 |
recv/recvfrom/accept/connect (plus 10 current_thread()) |
kernel/src/syscall/fs.rs |
5 |
|
kernel/src/syscall/futex.rs |
4 |
|
kernel/src/task/waitqueue.rs |
2 |
|
kernel/src/task/scheduler.rs |
2 |
|
kernel/src/task/completion.rs |
1 |
|
| tree-wide |
104 |
|
plus 42 still_blocked re-check sites across six files (socket.rs, handlers.rs, fs.rs,
futex.rs, signal.rs, time.rs), each of which is a HLT loop that must poll the same record it
stamped. Every one of these is on a blocking syscall path — changing them all in one branch would
swamp the review surface of whatever fix is landing alongside it.
3. It is not #558's defect class. #558's B1/B2 removed skew sources inside
kernel/src/interrupts/context_switch.rs. The recorded-current identity is a tree-wide invariant
question ("the recorded current always names the thread executing on this CPU") whose remaining
violators are outside that file — the known candidate is switch_to_idle() called from an exit/fault
path while the caller keeps running. Fixing the prologues without establishing that invariant would
paper over the symptom; establishing it needs its own design pass plus an oracle that can catch a
violation the way the leak oracles caught #470's.
What the follow-up work should contain
- An authoritative executing-thread identity (kernel-stack-derived
current, or a cookie in the
thread's kernel stack validated against cpu_state[cpu].current_thread).
- A debug-build assertion / trace counter that fires when the recorded current disagrees with that
identity — the oracle that would have named the run-3 skew instead of leaving it unattributed.
- Conversion of the blocking prologues, epilogues and
still_blocked polls to the authoritative id
(the ~79 syscall-path call sites above), keeping current_thread_mut() only where "whatever the
scheduler currently records" is genuinely what is meant.
- An audit of the remaining recorded-current writers outside
context_switch.rs
(switch_to_idle, switch_to_idle_best_effort, exception-cleanup paths) for callers that keep
running after handing the CPU away.
Related open issues: #527 (exec lock inversion), #545 (x86 TCP-recv loopback-delivery hang) — the
hang class this defect makes permanent.
Deferred out of PR #558 (fix/ctx-routing-and-abort-rollback), documented as a scope call in review;
not fixed there because it is a design-level, tree-wide invariant change rather than a targeted fix.
The defect
Every blocking syscall identifies "the thread that is blocking" by the scheduler's recorded current
thread, never by the thread that is actually executing:
If the recorded current has skewed away from the executing thread even once, the prologue stamps
Blocked + blocked_in_syscallonto a foreign thread's record and the HLT loop then polls thatforeign record. The victim waits on a condition its own waker will never touch — a permanent hang, and
the skew is now permanent because the wrong record carries the flag. This is the mechanism behind a
run-3 residual observed while gating #545/#557-adjacent work: idle's record carried
blocked_in_syscall, which nothing legitimately sets (idle never issues a blocking syscall), and thatflag is what steered a later tick into the kernel-frame save branch.
Why this is a separate issue, not folded into #557's fixes
PR #558 (
fix/ctx-routing-and-abort-rollback, merged) removed two concrete skew sources insidekernel/src/interrupts/context_switch.rs(rollback over-restore on save failure, and theSignalDeliveryResult::Terminatedarm leaving the frame/CR3 committed). Those fixes make the recordedcurrent correct at the moment they run. This issue is about the fact that "recorded current always
names the thread actually executing on this CPU" is not an enforced, tree-wide invariant — nothing
prevents a different writer, elsewhere in the tree, from introducing the same class of skew again.
1. There is no independent identity to switch to. Both candidate "executing thread" sources are
the same recorded state, written by the same dispatcher:
scheduler::current_thread_id()(scheduler.rs:3544)cpu_state[cpu].current_threadschedule()/switch_to_idle()/abort_dispatch_and_resume()per_cpu::current_thread()(per_cpu.rs:380)per_cpu::set_current_thread(), same call sitesThe syscall entry point itself resolves the caller the same way —
handler.rs:585doeslet current_thread_id = match crate::task::scheduler::current_thread_id(). So "capture the id atsyscall entry" captures the recorded current too; it narrows the window (one read instead of N) but
does not make the identity authoritative. A real fix needs an identity the dispatcher cannot get wrong:
Linux's shape is
currentderived from the kernel stack (thread_infoat the stack base, i.e.RSP & ~(THREAD_SIZE-1)), or equivalently a per-thread cookie written when the kernel stack is allocated andread back from
TSS.RSP0/RSP. Breenix has neither today — introducing one is a design change tothread creation,
kernel_stack_toppublication and every RSP0 publisher.2. The call-site fan-out is large and spread across the syscall surface. Direct
current_thread_mut()uses today:current_thread_mut()kernel/src/syscall/handlers.rskernel/src/syscall/socket.rscurrent_thread())kernel/src/syscall/fs.rskernel/src/syscall/futex.rskernel/src/task/waitqueue.rskernel/src/task/scheduler.rskernel/src/task/completion.rsplus 42
still_blockedre-check sites across six files (socket.rs,handlers.rs,fs.rs,futex.rs,signal.rs,time.rs), each of which is a HLT loop that must poll the same record itstamped. Every one of these is on a blocking syscall path — changing them all in one branch would
swamp the review surface of whatever fix is landing alongside it.
3. It is not #558's defect class. #558's B1/B2 removed skew sources inside
kernel/src/interrupts/context_switch.rs. The recorded-current identity is a tree-wide invariantquestion ("the recorded current always names the thread executing on this CPU") whose remaining
violators are outside that file — the known candidate is
switch_to_idle()called from an exit/faultpath while the caller keeps running. Fixing the prologues without establishing that invariant would
paper over the symptom; establishing it needs its own design pass plus an oracle that can catch a
violation the way the leak oracles caught #470's.
What the follow-up work should contain
current, or a cookie in thethread's kernel stack validated against
cpu_state[cpu].current_thread).identity — the oracle that would have named the run-3 skew instead of leaving it unattributed.
still_blockedpolls to the authoritative id(the ~79 syscall-path call sites above), keeping
current_thread_mut()only where "whatever thescheduler currently records" is genuinely what is meant.
context_switch.rs(
switch_to_idle,switch_to_idle_best_effort, exception-cleanup paths) for callers that keeprunning after handing the CPU away.
Related open issues: #527 (exec lock inversion), #545 (x86 TCP-recv loopback-delivery hang) — the
hang class this defect makes permanent.
Deferred out of PR #558 (
fix/ctx-routing-and-abort-rollback), documented as a scope call in review;not fixed there because it is a design-level, tree-wide invariant change rather than a targeted fix.