Feature request: subProcessAttach: false — track children and emit DAP events without injecting the adapter

[cesare-montresor]

Summary

Add a subProcessAttach flag (default true) that, when set to false, decouples subprocess tracking from adapter injection. With subProcess: true, subProcessAttach: false, debugpy tracks child processes, emits debugpyAttach events, and forwards stdout/stderr as DAP output events but does not inject pydevd, does not suspend the child.

This collapses two long-standing pain points into a single orthogonal flag:

1. No way to be notified of child spawns without triggering a full attach (and all the breakage that comes with it in spawn-mode multiprocessing).
2. No way to receive child stdout/stderr as DAP output events without full adapter injection.

---

Current behavior the forced binary choice

subProcess is currently all-or-nothing:

subProcess	effect
true	track + inject pydevd + suspend child waiting for client
false	silence everything, no events, no output, no visibility

There is no middle ground. For spawn-mode multiprocessing (PyTorch, mp.spawn, custom plugin/trial runners), subProcess: true breaks the child process entirely:

• The semaphore tracker helper (from multiprocessing.semaphore_tracker import main) is intercepted first and reported as the subprocess to attach the actual worker is never reached (ptvsd#1887, still skipped in the current test suite).
• SemLock objects cannot be pickled across the spawn boundary after debugpy patches __main__ imports (ptvsd#2108).
• Any custom IPC layer (pipes, mp.Pipe, shared-memory tensors) breaks because the child is suspended waiting for a DAP client the parent never intends to connect.

The only escape is subProcess: false, which makes child output completely invisible to the DAP layer. The only workaround we have found is intercepting raw PTY bytes via the VSCode extension API, which doesn't look stable.

vscode.window.onDidWriteTerminalData(e => {
    collectOutput(e.data); // no PID, no stdout/stderr distinction, VSCode-only
});

This is insufficient for our use case: it is unstructured, conflates all processes sharing the terminal, provides no process identity, and is unavailable to non-VSCode DAP clients.

---

Proposed solution

Split the single subProcess boolean into two independent concerns:

// launch.json
{
  "subProcess": true,       // keep tracking children (existing flag)
  "subProcessAttach": false // new: don't inject pydevd, don't suspend
}

Or programmatically:

debugpy.configure(subprocess=True, subprocess_attach=False)

Behavior matrix:

subProcess	subProcessAttach	result
false	---	existing: silence everything
true	true (default)	existing: track + inject + suspend
true	false	new: track + debugpyAttach event + output events, no injection, no suspension

With subProcessAttach: false, the adapter should:

1. Detect child spawns via the existing pydevd subprocess hook.
2. Emit a debugpyAttach event (existing extension to DAP) so the client is notified and can optionally attach manually.
3. Forward child stdout/stderr as DAP output events tagged with source.pid (already in the DAP spec's OutputEvent).
4. Not inject pydevd into the child.
5. Not suspend the child.

The child runs freely. The parent's IPC layer is unaffected. A DAP client that wants to attach can still do so on receiving the debugpyAttach event.

---

Why this covers both missing features

Spawn-mode notification without breakage:
The debugpyAttach event already carries a fully-formed attach config with the child PID. With subProcessAttach: false, this event is emitted but no automatic attach follows, the client decides what to do. This is the "notify but don't attach" mode that spawn-mode frameworks need.

Child output in the DAP layer:
With the adapter already aware of the child (via the subprocess hook), forwarding its stdout/stderr as output events requires only an OS-level pipe before the child's Python runtime initializes — no pydevd, no tracing overhead. This replaces onDidWriteTerminalData with a proper, PID-tagged, per-stream DAP event visible to all clients.

---

Prior art / related issues

• ptvsd#1887 (2019) spawn + semaphore tracker false-positive; test still skipped on Linux.
• ptvsd#2108 (2020) __main__ unpickling bug with spawn.
• ptvsd#1658 debug console missing output from multiprocessing children; workaround was "console": "integratedTerminal" (i.e. fall back to PTY).
• ptvsd#1585 output dropped when child exits before adapter flushes output events.
• ptvsd#1659 partial fix: fd-level capture for launch when children share parent fds; never extended to spawn.
• debugpy#42 subprocess hangs suspended when parent PID can't be resolved.
• debugpy#81 subprocess always waits for client; only escape is subProcess: false.
• debugpy#264 parent hangs when stdout=PIPE and forked child is being debugged.
• debugpy#1717 debugpy does not send output events of category stdout to non-VSCode DAP clients in some configurations.

---

Who this helps

Any setup using spawn start method that needs child visibility without full adapter injection:

• PyTorch distributed / mp.spawn
• Custom ML trial runners, plugin systems, hyperparameter search frameworks
• multiprocessing.Pool / ProcessPoolExecutor with spawn on Linux/macOS
• Non-VSCode DAP clients (Neovim/nvim-dap, JetBrains, CLI) that have no access to onDidWriteTerminalData

[cesare-montresor]

This is my own solution for "seamless" integration between vscode, debugpy, cuda and spwan.

def init_debugger(port: int, wait_for_client: bool = False) -> None:
    try:
        import debugpy
        debugpy.listen(("localhost", port))
        logger.info(f"Debugger listening on port {port}")  # extension sees this
        print(f"Debugger listening on port {port}", flush=True)  # belt and braces
        if wait_for_client:
            logger.info("Waiting for debugger to attach...")
            debugpy.wait_for_client()        # blocks AFTER the log line is out
            logger.info("Debugger attached")
    except Exception as e:
        logger.warning(f"Failed to initialize debugger on port {port}: {e}")

This small extension captures the custom output and attach to it automatically.
debugpy-spawn.zip

While this currently works, it's actually pretty fast, it's held together with glue tape and sticks.