Skip to content

Blocking-syscall prologues identify the executing thread via recorded scheduler current, not an authoritative identity (OQ1 residual) #560

Description

@ryanbreen

The defect

Every blocking syscall identifies "the thread that is blocking" by the scheduler's recorded current
thread, never by the thread that is actually executing:

sched.block_current();
if let Some(thread) = sched.current_thread_mut() {
    thread.blocked_in_syscall = true;
}
...
let still_blocked = scheduler::with_scheduler(|sched| {
    sched.current_thread_mut().map(|t| t.state == ThreadState::Blocked)
});

If the recorded current has skewed away from the executing thread even once, the prologue stamps
Blocked + blocked_in_syscall onto a foreign thread's record and the HLT loop then polls that
foreign record. The victim waits on a condition its own waker will never touch — a permanent hang, and
the skew is now permanent because the wrong record carries the flag. This is the mechanism behind a
run-3 residual observed while gating #545/#557-adjacent work: idle's record carried
blocked_in_syscall, which nothing legitimately sets (idle never issues a blocking syscall), and that
flag is what steered a later tick into the kernel-frame save branch.

Why this is a separate issue, not folded into #557's fixes

PR #558 (fix/ctx-routing-and-abort-rollback, merged) removed two concrete skew sources inside
kernel/src/interrupts/context_switch.rs (rollback over-restore on save failure, and the
SignalDeliveryResult::Terminated arm leaving the frame/CR3 committed). Those fixes make the recorded
current correct at the moment they run. This issue is about the fact that "recorded current always
names the thread actually executing on this CPU" is not an enforced, tree-wide invariant — nothing
prevents a different writer, elsewhere in the tree, from introducing the same class of skew again.

1. There is no independent identity to switch to. Both candidate "executing thread" sources are
the same recorded state, written by the same dispatcher:

source what it reads written by
scheduler::current_thread_id() (scheduler.rs:3544) cpu_state[cpu].current_thread schedule() / switch_to_idle() / abort_dispatch_and_resume()
per_cpu::current_thread() (per_cpu.rs:380) GS-relative current-thread pointer per_cpu::set_current_thread(), same call sites

The syscall entry point itself resolves the caller the same way — handler.rs:585 does
let current_thread_id = match crate::task::scheduler::current_thread_id(). So "capture the id at
syscall entry" captures the recorded current too; it narrows the window (one read instead of N) but
does not make the identity authoritative. A real fix needs an identity the dispatcher cannot get wrong:
Linux's shape is current derived from the kernel stack (thread_info at the stack base, i.e. RSP & ~(THREAD_SIZE-1)), or equivalently a per-thread cookie written when the kernel stack is allocated and
read back from TSS.RSP0/RSP. Breenix has neither today — introducing one is a design change to
thread creation, kernel_stack_top publication and every RSP0 publisher.

2. The call-site fan-out is large and spread across the syscall surface. Direct
current_thread_mut() uses today:

file current_thread_mut() notes
kernel/src/syscall/handlers.rs 45 read/write/waitpid/pause/nanosleep prologues + epilogues
kernel/src/syscall/socket.rs 20 recv/recvfrom/accept/connect (plus 10 current_thread())
kernel/src/syscall/fs.rs 5
kernel/src/syscall/futex.rs 4
kernel/src/task/waitqueue.rs 2
kernel/src/task/scheduler.rs 2
kernel/src/task/completion.rs 1
tree-wide 104

plus 42 still_blocked re-check sites across six files (socket.rs, handlers.rs, fs.rs,
futex.rs, signal.rs, time.rs), each of which is a HLT loop that must poll the same record it
stamped. Every one of these is on a blocking syscall path — changing them all in one branch would
swamp the review surface of whatever fix is landing alongside it.

3. It is not #558's defect class. #558's B1/B2 removed skew sources inside
kernel/src/interrupts/context_switch.rs. The recorded-current identity is a tree-wide invariant
question ("the recorded current always names the thread executing on this CPU") whose remaining
violators are outside that file — the known candidate is switch_to_idle() called from an exit/fault
path while the caller keeps running. Fixing the prologues without establishing that invariant would
paper over the symptom; establishing it needs its own design pass plus an oracle that can catch a
violation the way the leak oracles caught #470's.

What the follow-up work should contain

  1. An authoritative executing-thread identity (kernel-stack-derived current, or a cookie in the
    thread's kernel stack validated against cpu_state[cpu].current_thread).
  2. A debug-build assertion / trace counter that fires when the recorded current disagrees with that
    identity — the oracle that would have named the run-3 skew instead of leaving it unattributed.
  3. Conversion of the blocking prologues, epilogues and still_blocked polls to the authoritative id
    (the ~79 syscall-path call sites above), keeping current_thread_mut() only where "whatever the
    scheduler currently records" is genuinely what is meant.
  4. An audit of the remaining recorded-current writers outside context_switch.rs
    (switch_to_idle, switch_to_idle_best_effort, exception-cleanup paths) for callers that keep
    running after handing the CPU away.

Related open issues: #527 (exec lock inversion), #545 (x86 TCP-recv loopback-delivery hang) — the
hang class this defect makes permanent.

Deferred out of PR #558 (fix/ctx-routing-and-abort-rollback), documented as a scope call in review;
not fixed there because it is a design-level, tree-wide invariant change rather than a targeted fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions