Skip to content

Add Data Loop denial diagnostics - #916

Draft
Richie Gomez (richiemsft) wants to merge 10 commits into
mainfrom
data-loop-decoder
Draft

Add Data Loop denial diagnostics#916
Richie Gomez (richiemsft) wants to merge 10 commits into
mainfrom
data-loop-decoder

Conversation

@richiemsft

@richiemsft Richie Gomez (richiemsft) commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

📖 Description

Adds a deterministic Data Loop JSON sibling for every successful Learning Mode denial analysis.

The Data Loop document is a bounded, deduplicated superset of canonical denial events and noncanonical decoder outcomes. Signatures retain provider/event identity, PID, safe operational identifiers, typed accessType and resourceType when available, sorted sanitized properties, and occurrence counts. Usernames are redacted from account fields and profile/resource paths before truncation.

Canonical-denial signatures take priority over diagnostic signatures at group, byte, and guarded-transport bounds. Overflow accounting distinguishes canonical occurrences that could not retain a signature. Schema metadata failures remain fatal; recoverable per-event failures use closed payload, decoder-limit, and unsupported-encoding categories.

Canonical and Data Loop files are staged and promoted as one transaction. Native capture, guarded WPR fallback, and --audit relocation use the same paired-output contract.

🔗 References

Stacked on #847.

🔍 Validation

  • Rust unit tests for the Data Loop model, ETL decoder, paired output, and audit relocation
  • Clippy with warnings denied across the affected crates
  • Release wxc-exec.exe and plm.exe build
  • Guarded-WPR VM validation with canonical and typed Data Loop groups, zero overflow, and no username leakage

✅ Checklist

📋 Issue Type

  • Bug fix
  • Feature
  • Task

GitHub Actions runs the PR validation build automatically. The ADO pipeline
(MXC-PR-Build) is the Azure version of the PR pipeline, kept in parity with the GitHub
Actions build; it runs on merge to main, and Microsoft reviewers with write access can trigger it
on a PR with /azp run. See docs/pull-requests.md.

If the dependency-feed-check check fails on a new dependency, the crate must be added to
the feed before the PR can pass. See docs/pull-requests.md
for the steps.

Microsoft Reviewers: Open in CodeFlow

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Route wxc-exec --audit through captureDenials so native PSEC/V2 remains preferred and guarded WPR provides compatible analysis and ETL retention. Reuse canonical denials for adjusted policy generation and support retainEtl consistently across both capture providers.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Update repository guidance for captureDenials-backed audit routing and guarded retainEtl parity.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
@richiemsft
Richie Gomez (richiemsft) changed the base branch from unified-audit-capture to main August 17, 2026 19:44
Emit a bounded, username-redacted Data Loop JSON sibling containing canonical denial occurrences and classified decoder outcomes. Keep paired output transactional across native, guarded, and audit capture paths.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Copilot AI balanced review requested due to automatic review settings August 17, 2026 19:54
@microsoft-github-policy-service microsoft-github-policy-service Bot added the Copilot-Instructions PR modifies Copilot instruction files (.github/copilot-instructions.md or .github/instructions/) label Aug 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds bounded, redacted Data Loop diagnostics alongside Learning Mode denial output and integrates them across native capture, guarded WPR, and audit workflows.

Changes:

  • Adds the Data Loop model, aggregation, sanitization, and paired output.
  • Adds process-scoped guarded-WPR ETL relogging and retention.
  • Reworks --audit to consume captureDenials artifacts directly.

Reviewed changes

Copilot reviewed 29 out of 30 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
.github/copilot-instructions.md Updates Learning Mode architecture guidance.
README.md Revises audit-mode documentation.
docs/learning-mode/capabilities.md Documents Data Loop and retained ETL behavior.
src/Cargo.lock Records the new dependency edge.
src/backends/appcontainer/common/src/appcontainer_runner.rs Supports guarded ETL transfer.
src/backends/appcontainer/common/src/base_container_runner.rs Integrates retained guarded captures.
src/backends/appcontainer/common/src/capture_output.rs Adds paired denial/Data Loop output.
src/backends/appcontainer/common/src/guarded_capture.rs Extends guarded-capture interfaces.
src/backends/learning_mode/windows/src/capability_dacl.rs Adds diagnostic metadata to recovered denials.
src/backends/learning_mode/windows/src/etl_decode.rs Aggregates Data Loop outcomes and selects relog events.
src/backends/learning_mode/windows/src/etl_filter.rs Adds process-scoped ETL relogging.
src/backends/learning_mode/windows/src/extractors.rs Adds sanitization and diagnostic classification.
src/backends/learning_mode/windows/src/lib.rs Exports ETL filtering.
src/backends/learning_mode/windows/src/tdh_decode.rs Categorizes recoverable decoding failures.
src/core/learning_mode_core/src/analyze.rs Carries and bounds Data Loop results.
src/core/learning_mode_core/src/data_loop.rs Defines the Data Loop model and writer.
src/core/learning_mode_core/src/lib.rs Exports Data Loop APIs.
src/core/learning_mode_core/src/model.rs Makes classifications sortable.
src/core/mxc_engine/src/guarded_capture.rs Implements guarded trace transfer.
src/core/mxc_engine/src/lib.rs Exports audit runner resolution.
src/core/mxc_engine/src/run.rs Adds audit-specific resolver entry point.
src/core/wxc/Cargo.toml Adds Learning Mode core dependency.
src/core/wxc/src/audit.rs Relocates and post-processes capture artifacts.
src/core/wxc/src/main.rs Routes audit through captureDenials.
src/host/plm/readme.md Documents guarded analysis and validation.
src/host/plm/src/analysis.rs Returns canonical documents after writing.
src/host/plm/src/elevated.rs Adds filtered analysis/trace transfer protocol.
src/host/plm/src/log.rs Consumes guardian-returned analysis directly.
src/host/plm/src/secure_scratch.rs Manages filtered ETL scratch storage.
src/host/plm/src/stop.rs Extracts canonical-denial post-processing.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +96 to +97
if !is_learning_mode_event(header.ProviderId, header.EventDescriptor.Id) {
return Ok(());
.map_err(|error| error.error)
.with_context(|| format!("failed to replace {}", output_path.display()))?;
Ok(())
Ok(document)
Comment on lines +762 to +766
let Some(filetime) = normalized_filetime(header.TimeStamp, acc) else {
return;
};
if acc.event_in_scope(header.ProcessId, filetime) {
if acc.relog_selected_event_indices.len() >= MAX_PROCESSED_EVENTS {
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Copilot-Instructions PR modifies Copilot instruction files (.github/copilot-instructions.md or .github/instructions/)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants