Add Data Loop denial diagnostics - #916
Draft
Richie Gomez (richiemsft) wants to merge 10 commits into
Draft
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Route wxc-exec --audit through captureDenials so native PSEC/V2 remains preferred and guarded WPR provides compatible analysis and ETL retention. Reuse canonical denials for adjusted policy generation and support retainEtl consistently across both capture providers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Update repository guidance for captureDenials-backed audit routing and guarded retainEtl parity. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Richie Gomez (richiemsft)
force-pushed
the
unified-audit-capture
branch
from
August 17, 2026 16:29
a300213 to
5e1090c
Compare
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Richie Gomez (richiemsft)
changed the base branch from
unified-audit-capture
to
main
August 17, 2026 19:44
Emit a bounded, username-redacted Data Loop JSON sibling containing canonical denial occurrences and classified decoder outcomes. Keep paired output transactional across native, guarded, and audit capture paths. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 4eff3ee2-323c-494d-99b9-f7b54e495216
Richie Gomez (richiemsft)
force-pushed
the
data-loop-decoder
branch
from
August 17, 2026 19:54
dbf31fe to
abf1219
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adds bounded, redacted Data Loop diagnostics alongside Learning Mode denial output and integrates them across native capture, guarded WPR, and audit workflows.
Changes:
- Adds the Data Loop model, aggregation, sanitization, and paired output.
- Adds process-scoped guarded-WPR ETL relogging and retention.
- Reworks
--auditto consumecaptureDenialsartifacts directly.
Reviewed changes
Copilot reviewed 29 out of 30 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
.github/copilot-instructions.md |
Updates Learning Mode architecture guidance. |
README.md |
Revises audit-mode documentation. |
docs/learning-mode/capabilities.md |
Documents Data Loop and retained ETL behavior. |
src/Cargo.lock |
Records the new dependency edge. |
src/backends/appcontainer/common/src/appcontainer_runner.rs |
Supports guarded ETL transfer. |
src/backends/appcontainer/common/src/base_container_runner.rs |
Integrates retained guarded captures. |
src/backends/appcontainer/common/src/capture_output.rs |
Adds paired denial/Data Loop output. |
src/backends/appcontainer/common/src/guarded_capture.rs |
Extends guarded-capture interfaces. |
src/backends/learning_mode/windows/src/capability_dacl.rs |
Adds diagnostic metadata to recovered denials. |
src/backends/learning_mode/windows/src/etl_decode.rs |
Aggregates Data Loop outcomes and selects relog events. |
src/backends/learning_mode/windows/src/etl_filter.rs |
Adds process-scoped ETL relogging. |
src/backends/learning_mode/windows/src/extractors.rs |
Adds sanitization and diagnostic classification. |
src/backends/learning_mode/windows/src/lib.rs |
Exports ETL filtering. |
src/backends/learning_mode/windows/src/tdh_decode.rs |
Categorizes recoverable decoding failures. |
src/core/learning_mode_core/src/analyze.rs |
Carries and bounds Data Loop results. |
src/core/learning_mode_core/src/data_loop.rs |
Defines the Data Loop model and writer. |
src/core/learning_mode_core/src/lib.rs |
Exports Data Loop APIs. |
src/core/learning_mode_core/src/model.rs |
Makes classifications sortable. |
src/core/mxc_engine/src/guarded_capture.rs |
Implements guarded trace transfer. |
src/core/mxc_engine/src/lib.rs |
Exports audit runner resolution. |
src/core/mxc_engine/src/run.rs |
Adds audit-specific resolver entry point. |
src/core/wxc/Cargo.toml |
Adds Learning Mode core dependency. |
src/core/wxc/src/audit.rs |
Relocates and post-processes capture artifacts. |
src/core/wxc/src/main.rs |
Routes audit through captureDenials. |
src/host/plm/readme.md |
Documents guarded analysis and validation. |
src/host/plm/src/analysis.rs |
Returns canonical documents after writing. |
src/host/plm/src/elevated.rs |
Adds filtered analysis/trace transfer protocol. |
src/host/plm/src/log.rs |
Consumes guardian-returned analysis directly. |
src/host/plm/src/secure_scratch.rs |
Manages filtered ETL scratch storage. |
src/host/plm/src/stop.rs |
Extracts canonical-denial post-processing. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+96
to
+97
| if !is_learning_mode_event(header.ProviderId, header.EventDescriptor.Id) { | ||
| return Ok(()); |
| .map_err(|error| error.error) | ||
| .with_context(|| format!("failed to replace {}", output_path.display()))?; | ||
| Ok(()) | ||
| Ok(document) |
Comment on lines
+762
to
+766
| let Some(filetime) = normalized_filetime(header.TimeStamp, acc) else { | ||
| return; | ||
| }; | ||
| if acc.event_in_scope(header.ProcessId, filetime) { | ||
| if acc.relog_selected_event_indices.len() >= MAX_PROCESSED_EVENTS { |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📖 Description
Adds a deterministic Data Loop JSON sibling for every successful Learning Mode denial analysis.
The Data Loop document is a bounded, deduplicated superset of canonical denial events and noncanonical decoder outcomes. Signatures retain provider/event identity, PID, safe operational identifiers, typed
accessTypeandresourceTypewhen available, sorted sanitized properties, and occurrence counts. Usernames are redacted from account fields and profile/resource paths before truncation.Canonical-denial signatures take priority over diagnostic signatures at group, byte, and guarded-transport bounds. Overflow accounting distinguishes canonical occurrences that could not retain a signature. Schema metadata failures remain fatal; recoverable per-event failures use closed payload, decoder-limit, and unsupported-encoding categories.
Canonical and Data Loop files are staged and promoted as one transaction. Native capture, guarded WPR fallback, and
--auditrelocation use the same paired-output contract.🔗 References
Stacked on #847.
🔍 Validation
wxc-exec.exeandplm.exebuild✅ Checklist
Cargo.lock, thedependency-feed-checkcheck passes (see docs/pull-requests.md)📋 Issue Type
GitHub Actions runs the PR validation build automatically. The ADO pipeline
(
MXC-PR-Build) is the Azure version of the PR pipeline, kept in parity with the GitHubActions build; it runs on merge to
main, and Microsoft reviewers with write access can trigger iton a PR with
/azp run. See docs/pull-requests.md.If the
dependency-feed-checkcheck fails on a new dependency, the crate must be added tothe feed before the PR can pass. See docs/pull-requests.md
for the steps.
Microsoft Reviewers: Open in CodeFlow