feat(EC-1836): add ec.oci.parsed_blob builtin for cached JSON parsing

[robnester-rh]

Summary

• Registers new OPA builtin ec.oci.parsed_blob(ref) that fetches an OCI blob, parses it as JSON, and caches the parsed AST in a per-component sync.Map
• Eliminates redundant json.unmarshal calls when the same blob is evaluated across multiple policy namespaces (currently 9x for SBOMs)
• Reuses ociBlobInternal for fetching (raw blob caching already handled), then parses once and caches the ast.Term

Test plan

• Valid JSON blob returns parsed object
• Invalid JSON blob returns nil (no error propagated to OPA)
• Caching returns pointer-identical result on second call, single Layer fetch
• Unexpected URI type returns nil
• Remote error returns nil
• TestFunctionsRegistered includes ec.oci.parsed_blob

resolves: EC-1836

🤖 Generated with Claude Code

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Enterprise

Run ID: a3854ddf-ebe1-4b40-8ed4-f662d74b667b

📥 Commits

Reviewing files that changed from the base of the PR and between 20d275f and 790ebe2.

📒 Files selected for processing (5)

• docs/modules/ROOT/pages/ec_oci_parsed_blob.adoc
• docs/modules/ROOT/pages/rego_builtins.adoc
• docs/modules/ROOT/partials/rego_nav.adoc
• internal/rego/oci/oci.go
• internal/rego/oci/oci_test.go

✅ Files skipped from review due to trivial changes (3)

• docs/modules/ROOT/partials/rego_nav.adoc
• docs/modules/ROOT/pages/ec_oci_parsed_blob.adoc
• docs/modules/ROOT/pages/rego_builtins.adoc

🚧 Files skipped from review as they are similar to previous changes (2)

• internal/rego/oci/oci_test.go
• internal/rego/oci/oci.go

---

📝 Walkthrough

Walkthrough

A new OPA rego builtin ec.oci.parsed_blob is added to fetch an OCI blob from a registry, unmarshal its content as JSON, and return the parsed value. The implementation includes per-component caching with singleflight deduplication, registration as a memoized nondeterministic builtin, comprehensive test coverage, and full user documentation.

Changes

ociParsedBlob OPA Builtin

Layer / File(s)	Summary
Contract and caching infrastructure internal/rego/oci/oci.go	Adds the ociParsedBlobName constant and extends ComponentCache with parsedBlobCache (sync.Map) and parsedBlobFlight (singleflight.Group) to enable per-component, per-ref-string caching and deduplication of concurrent requests.
Builtin registration and implementation internal/rego/oci/oci.go	Registers the ociParsedBlobName builtin via registerOCIParsedBlob() as memoized and nondeterministic, implements ociParsedBlob() to validate string ref input, fetch the raw blob via ociBlobInternal, unmarshal JSON, convert to OPA AST term, and return the cached result, and wires registration in init().
Test coverage internal/rego/oci/oci_test.go	Adds TestOCIParsedBlob with subtests for valid JSON parsing, invalid JSON returning nil, memoization and cache reuse with identical term instances, unsupported URI term types, and remote layer fetch errors; extends TestFunctionsRegistered to assert the builtin is registered.
Documentation and navigation docs/modules/ROOT/pages/ec_oci_parsed_blob.adoc, docs/modules/ROOT/pages/rego_builtins.adoc, docs/modules/ROOT/partials/rego_nav.adoc	Creates a detailed reference page for ec.oci.parsed_blob documenting the ref input parameter and value return type, adds an index entry to the Rego builtins table, and adds a navigation link in the Rego documentation partial.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 9.09% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: adding a new OPA builtin function for cached JSON parsing of OCI blobs.
Description check	✅ Passed	The description is well-related to the changeset, explaining the purpose, implementation details, and test plan for the new ec.oci.parsed_blob builtin.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[qodo-for-conforma]

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0)

Context used

✅ Compliance rules (platform): 27 rules

1. No singleflight for parsing 🐞 Bug ➹ Performance

Description

ociParsedBlob uses parsedBlobCache but does not use singleflight, so concurrent cache misses
for the same ref can still run JSON unmarshal + AST conversion multiple times (and potentially
emit repeated error logs). This diverges from the module’s established pattern (e.g.,
ociBlobInternal, ociImageFiles) that avoids thundering-herd work per key.

Code

internal/rego/oci/oci.go[R640-679]

+	cc := componentCacheFromContext(bctx.Context)
+
+	if cached, found := cc.parsedBlobCache.Load(refStr); found {
+		logger.Debug("Parsed blob served from cache")
+		return cached.(*ast.Term), nil
+	}
+
+	rawTerm, err := ociBlobInternal(bctx, a, true)
+	if err != nil || rawTerm == nil {
+		return nil, nil
+	}
+
+	rawStr, ok := rawTerm.Value.(ast.String)
+	if !ok {
+		logger.Error("blob value is not a string")
+		return nil, nil
+	}
+
+	var parsed any
+	if err := json.Unmarshal([]byte(string(rawStr)), &parsed); err != nil {
+		logger.WithFields(log.Fields{
+			"action": "unmarshal",
+			"error":  err,
+		}).Error("failed to unmarshal blob as JSON")
+		return nil, nil
+	}
+
+	value, err := ast.InterfaceToValue(parsed)
+	if err != nil {
+		logger.WithFields(log.Fields{
+			"action": "convert to ast",
+			"error":  err,
+		}).Error("failed to convert parsed JSON to AST value")
+		return nil, nil
+	}
+
+	term := ast.NewTerm(value)
+	cc.parsedBlobCache.Store(refStr, term)
+	logger.Debug("Parsed blob cached")
+	return term, nil

Relevance

⭐⭐⭐ High

Repo pattern uses singleflight on cached heavy work; reviewers accepted adding singleflight to avoid
thundering herd in PR #3125.

PR-#3125
PR-#3073

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The new builtin performs cache check + parse + store without singleflight, while existing
heavy-cache code paths in this file explicitly use singleflight.Group to avoid duplicated work on
concurrent misses.

internal/rego/oci/oci.go[629-680]
internal/rego/oci/oci.go[512-621]
internal/rego/oci/oci.go[1047-1143]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`ociParsedBlob` caches parsed results in `cc.parsedBlobCache`, but concurrent callers can miss the cache and all parse the same blob at once because there is no `singleflight.Group` guarding the parse path.

## Issue Context
Other heavy OCI operations in this package use `singleflight` (e.g., blob fetch and image/blob files extraction) to ensure only one goroutine performs the work per cache key.

## Fix Focus Areas
- internal/rego/oci/oci.go[629-680]
- internal/rego/oci/oci.go[885-891]

## Implementation notes
- Add `parsedBlobFlight singleflight.Group` to `ComponentCache`.
- Wrap the parse path in `cc.parsedBlobFlight.Do(refStr, func() (any, error) { ... })`.
- Inside the `Do` callback, double-check `cc.parsedBlobCache.Load(refStr)` before doing work (matching the existing pattern used by `ociBlobInternal`).
- Store and return the parsed `*ast.Term` from within the singleflight callback.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

2. Misleading per-component caching 🐞 Bug ⚙ Maintainability

Description

The builtin’s description claims results are cached “per component”, but ociParsedBlob uses
componentCacheFromContext which falls back to a global defaultComponentCache when
WithComponentCache isn’t set, making the cache effectively process-global in that case. This
mismatch can lead to unintended cache lifetime/memory retention and incorrect assumptions by callers
about isolation.

Code

internal/rego/oci/oci.go[R114-119]

+	ast.RegisterBuiltin(&ast.Builtin{
+		Name:             decl.Name,
+		Description:      "Fetch a blob from an OCI registry and return the parsed JSON value. Results are cached per component.",
+		Decl:             decl.Decl,
+		Nondeterministic: decl.Nondeterministic,
+	})

Relevance

⭐⭐⭐ High

Team has accepted fixes to misleading comments/docs; component-cache fallback behavior documented in
PR #3120.

PR-#3120
PR-#3043

ⓘ Recommendations generated based on similar findings in past PRs

Evidence

The PR adds a description promising per-component caching, but the cache target is
componentCacheFromContext, which explicitly falls back to a global default cache when the context
isn’t annotated; the repo has at least one evaluator creation path that does not apply
WithComponentCache.

internal/rego/oci/oci.go[100-120]
internal/rego/oci/oci.go[629-680]
internal/rego/oci/oci.go[902-909]
cmd/validate/image.go[345-362]
internal/evaluation_target/input/input.go[39-69]

Agent prompt

The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`registerOCIParsedBlob` describes `ec.oci.parsed_blob` results as cached per component, but the implementation caches into `defaultComponentCache` when a component-scoped cache is not installed on the context.

## Issue Context
- `componentCacheFromContext` returns a process-global fallback when the context lacks a component cache.
- Some call paths install `WithComponentCache`, but others construct OPA evaluators without it.

## Fix Focus Areas
- internal/rego/oci/oci.go[100-120]
- internal/rego/oci/oci.go[902-909]
- internal/rego/oci/oci.go[629-680]
- cmd/validate/image.go[345-362]
- internal/evaluation_target/input/input.go[39-69]

## Implementation options (pick one)
1) **Clarify contract**: Update the builtin description to reflect reality, e.g. “cached in the context’s component cache (if present), otherwise cached in a global fallback cache.”
2) **Enforce component scoping**: Only read/write `parsedBlobCache` when the context actually contains a `*ComponentCache`; otherwise skip storing into the fallback cache (rely on OPA memoization within a single eval).
3) **Ensure callers install component cache**: Wrap the contexts passed into OPA evaluators with `WithComponentCache` in all relevant entry points so the description remains true.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

---

[qodo-for-conforma]

PR Summary by Qodo

Add ec.oci.parsed_blob OPA builtin with per-component JSON parse caching
✨ Enhancement 🧪 Tests 🕐 40+ Minutes

Walkthroughs

Description

• Register new OPA builtin "ec.oci.parsed_blob(ref)" returning parsed JSON from an OCI blob.
• Cache parsed AST terms per component to avoid repeated JSON unmarshalling across namespaces.
• Add unit tests covering success, invalid JSON, caching behavior, and error cases.

Diagram

graph TD
  P["OPA policy eval"] --> B["Builtin: ec.oci.parsed_blob"] --> C["ComponentCache"]
  C -- "miss" --> O["ociBlobInternal"] --> R{{"OCI registry"}}
  O --> J["json.Unmarshal"] --> T["ast.Term"] --> C
  C -- "hit" --> B

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Rely on OPA builtin memoization only (no explicit parsed cache)

• ➕ Less custom caching logic and fewer cache invalidation concerns
• ➖ Memoization scope may not cover repeated evaluations across policy namespaces/components as desired
• ➖ Does not guarantee pointer-identical reuse of parsed results across calls

2. Extend ec.oci.blob to optionally return parsed JSON (mode flag)

• ➕ Single API surface for blob retrieval with an optional parsed form
• ➕ Could share parsing/caching implementation behind one builtin name
• ➖ More complex function contract and typing (string vs any)
• ➖ Higher risk of breaking existing callers or confusing return type semantics

3. Generic cached JSON parse builtin keyed by string (e.g., ec.json.parse_cached)

• ➕ Reusable caching for other JSON parsing hotspots beyond OCI blobs
• ➖ Still needs careful keying/lifecycle to avoid unbounded growth
• ➖ Doesn’t couple naturally to existing component-scoped OCI cache lifecycle

Recommendation: The current approach (a dedicated ec.oci.parsed_blob builtin with a per-component parsed-term cache) is a good fit: it keeps the existing ec.oci.blob contract intact, leverages existing blob fetch/digest verification, and scopes memory growth to component evaluation. The dedicated builtin also makes policy intent explicit ("this blob is JSON") and enables strong caching semantics (including pointer-identical reuse) validated by tests.

File Changes

Enhancement (1)

oci.go Register ec.oci.parsed_blob and cache parsed JSON AST per component +82/-4

Register ec.oci.parsed_blob and cache parsed JSON AST per component

• Adds a new OPA builtin (ec.oci.parsed_blob) that fetches an OCI blob via ociBlobInternal, parses it as JSON, converts it to an OPA AST value, and caches the resulting *ast.Term in a per-component sync.Map. Extends ComponentCache with parsedBlobCache and registers the builtin during init, while preserving the existing "never return error" builtin contract by returning nil on failures.

internal/rego/oci/oci.go

---

Tests (1)

oci_test.go Add unit coverage for OCI parsed-blob builtin behavior and registration +91/-0

Add unit coverage for OCI parsed-blob builtin behavior and registration

• Introduces TestOCIParsedBlob to validate successful parsing, invalid JSON handling (nil result), cache hit behavior (same pointer, single remote Layer call), unexpected input type, and remote errors. Updates TestFunctionsRegistered to assert the new builtin is registered.

internal/rego/oci/oci_test.go

---

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 1:39 PM UTC · Completed 1:51 PM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-review]

Review

Findings

Low

• [error-handling] internal/rego/oci/oci.go:654 — In the singleflight callback, ociBlobInternal returns (nil, nil) on all error paths (never a non-nil error). The if err != nil check after the ociBlobInternal call is therefore dead code. This is consistent with the existing ociBlob function pattern and is a pre-existing design choice rather than a new bug.

• [edge-case] internal/rego/oci/oci.go:648 — When ociBlobInternal succeeds but the blob content is not valid JSON (or ast.InterfaceToValue fails), the singleflight callback returns (nil, nil) without caching the failure. Subsequent calls will re-attempt the parse step. However, the raw blob is cached in cc.blobCache by ociBlobInternal, so only the lightweight parse step repeats — no network I/O is wasted. This is consistent with the existing caching strategy that avoids negative caching.

Info

• [test-inadequate] internal/rego/oci/oci_test.go — The test suite covers valid JSON, invalid JSON, caching identity, unexpected input type, and remote errors — a solid set of cases. One gap: there is no test for the singleflight deduplication path (concurrent callers for the same ref). The existing ociBlob tests also lack this, so this is a pre-existing gap.

• [resource-exhaustion] internal/rego/oci/oci.go — json.Unmarshal followed by ast.InterfaceToValue can amplify memory usage relative to the raw JSON string. This is the same risk profile as existing functions like ociBlobFiles, and the per-component cache scoping mitigates unbounded growth. Informational only.

• [sub-agent-failure] N/A — The style-conventions, intent-coherence, and docs-currency sub-agents did not return findings due to model unavailability. These are sonnet-tier dimensions; the opus-tier dimensions (correctness, security) completed successfully.

Previous run

Review

Findings

Low

• [error-handling] internal/rego/oci/oci.go:654 — In ociParsedBlob, when the singleflight callback returns (nil, nil) on failure (e.g. invalid JSON), the result is not cached in parsedBlobCache. Subsequent calls with the same ref to a non-JSON blob will re-execute the unmarshal path (though the raw blob fetch is fast due to ociBlobInternal caching). A negative cache entry could prevent repeated unmarshal attempts, but this is consistent with how ociBlobInternal handles failures and is not incorrect.

• [resource-exhaustion] internal/rego/oci/oci.go — json.Unmarshal is called on blob data without a size limit on the JSON input. However, ociBlobInternal (called with verifyDigest=true) already loads the entire blob into memory, so this does not expand the existing attack surface. The risk is consistent with the existing ociBlob function.

Info

• [test-adequacy] internal/rego/oci/oci_test.go — No concurrent access test for ociParsedBlob's singleflight+sync.Map pattern. However, the identical pattern is used by 5+ other functions in this file (ociBlob, ociBlobFiles, etc.) without concurrency tests, and the underlying primitives are concurrency-safe by design.

• [data-handling] internal/rego/oci/oci.go — The new function follows established security patterns: digest verification via ociBlobInternal(verifyDigest=true), component-scoped caching, singleflight deduplication, input type validation, and error swallowing consistent with the OPA builtin contract.

• [sub-agent-failure] The style-conventions and docs-currency sub-agents did not return findings due to model unavailability (claude-sonnet-4-5@20250929 not available on vertex deployment).

Previous run (2)

Review

Findings

High

• [missing-doc] docs/modules/ROOT/pages/ec_oci_parsed_blob.adoc — The PR adds xref:ec_oci_parsed_blob.adoc[ec.oci.parsed_blob] entries in both rego_builtins.adoc and rego_nav.adoc, but the target page docs/modules/ROOT/pages/ec_oci_parsed_blob.adoc does not exist in the repository. Every other builtin listed in those files has a corresponding .adoc page. This will produce a broken cross-reference link in published documentation.
  Remediation: Create docs/modules/ROOT/pages/ec_oci_parsed_blob.adoc with the builtin's description, usage, parameters, and return value, following the same structure as the existing ec_oci_blob.adoc page.

Medium

• [missing-singleflight] internal/rego/oci/oci.go — ociParsedBlob uses bare sync.Map Load/Store for its parsedBlobCache without a singleflight.Group. Every other cached operation in ComponentCache (blobCache/blobFlight, filesCache/filesFlight) pairs a sync.Map with a singleflight.Group to deduplicate concurrent requests. While the underlying ociBlobInternal call is protected by blobFlight, the json.Unmarshal and ast.InterfaceToValue steps are not deduplicated. For the stated use case (9x SBOM evaluations), concurrent calls for the same ref will each independently unmarshal and convert the same blob.
  Remediation: Add a parsedBlobFlight singleflight.Group to ComponentCache and wrap the fetch-parse-store logic in a singleflight.Do call.

Low

• [resource-exhaustion] internal/rego/oci/oci.go — ociParsedBlob calls json.Unmarshal on the full blob content without a size check, creating memory amplification (raw bytes → Go interface{} graph → OPA AST tree). However, this is consistent with the existing ociBlobInternal pattern which also reads entire blobs into memory without size limits. The maxTarEntrySize limit only applies to ociBlobFiles tar entries, not raw blob access. This is an existing architectural property rather than a regression.

• [error-handling] internal/rego/oci/oci.go — When ociBlobInternal returns a non-nil error, ociParsedBlob discards it and returns (nil, nil). This is consistent with the file-level convention (lines 17–19: "rego functions in this file never return an error") but means concurrent callers without singleflight independently encounter and discard the same error.

Previous run (3)

Review

Findings

Low

• [pattern-violation] internal/rego/oci/oci.go:638 — Unlike blobCache/filesCache which use singleflight.Group to deduplicate concurrent in-flight requests, parsedBlobCache has no corresponding singleflight. The underlying ociBlobInternal already deduplicates the fetch, so redundancy is limited to JSON unmarshal and AST conversion. sync.Map prevents data races, so correctness is maintained.
  Remediation: Add a parsedBlobFlight singleflight.Group field to ComponentCache and wrap the fetch-parse-store logic in a Do call, consistent with the existing blobFlight/filesFlight pattern.

• [error-handling] internal/rego/oci/oci.go:647 — When ociBlobInternal returns an error (only possible via singleflight panic recovery), ociParsedBlob silently returns (nil, nil) without logging. Normal errors are already logged within ociBlobInternal. Minor observability gap.

• [design-direction] internal/rego/oci/oci.go — Consider adding a brief comment explaining why per-component caching supplements OPA memoization (Memoize only deduplicates within a single Eval() call, not across policy namespace evaluations).

Info

• [naming-convention] internal/rego/oci/oci.go — Name ec.oci.parsed_blob could be more explicit (e.g., ec.oci.json_blob) to clarify only JSON parsing is supported. Current name leaves room for future format extension, which may be intentional.

[fullsend-ai-review]

🤖 Review · Started 1:58 PM UTC
Commit: 47d3320 · View workflow run →

[codecov]

Codecov Report

❌ Patch coverage is 85.71429% with 9 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/rego/oci/oci.go	85.71%	9 Missing ⚠️

Flag	Coverage Δ
acceptance	53.43% <30.15%> (-0.12%)	⬇️
generative	16.79% <0.00%> (-0.16%)	⬇️
integration	27.66% <0.00%> (-0.26%)	⬇️
unit	69.13% <85.71%> (+0.10%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
internal/rego/oci/oci.go	90.29% <85.71%> (-0.29%)	⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[fullsend-ai-review]

See the review comment for full details.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 1:58 PM UTC · Completed 2:09 PM UTC
Commit: 47d3320 · View workflow run →

[robnester-rh]

Fixed in 20d275f — added docs/modules/ROOT/pages/ec_oci_parsed_blob.adoc with the builtin's description, usage, parameters, and return value.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 2:38 PM UTC · Completed 2:47 PM UTC
Commit: 47d3320 · View workflow run →

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 3:07 PM UTC · Completed 3:15 PM UTC
Commit: 47d3320 · View workflow run →