Skip to content

fix: exclude test template images from images-are-certified check - #652

Open
hardengl wants to merge 1 commit into
redhat-certification:mainfrom
hardengl:fix/exclude-test-images-from-certification-check
Open

fix: exclude test template images from images-are-certified check#652
hardengl wants to merge 1 commit into
redhat-certification:mainfrom
hardengl:fix/exclude-test-images-from-certification-check

Conversation

@hardengl

@hardengl hardengl commented Aug 6, 2026

Copy link
Copy Markdown

Summary

The images-are-certified check currently scans ALL rendered manifests for image references, including those in Helm test templates (templates/tests/). Test images are not part of the deployed application and should not be subject to certification requirements.

This PR filters out test template images before performing certification checks. Test templates are identified by:

  1. Source path containing /tests/ (standard Helm convention)
  2. helm.sh/hook: test annotation (any quoting style)

Fixes

Resolves #480

Behavioral Change

Charts with test templates that reference uncertified images (e.g. busybox, non-fully-qualified ImageStream references) will no longer fail the images-are-certified check due to those test-only images.

Testing

  • Added TestExcludeTestTemplates (5 scenarios) validating the filtering logic
  • Added TestIsTestTemplate (7 scenarios) validating template identification
  • Updated existing TestTemplate and TestImageCertify expectations to reflect the new behavior
  • All internal/... tests pass (the pre-existing TestSignedChart failure in pkg/ is unrelated)

The images-are-certified check now excludes images referenced in Helm
test templates (templates/tests/ or helm.sh/hook: test annotated).
Test images are not part of the application and should not require
certification.

Fixes redhat-certification#480

Co-authored-by: Cursor <cursoragent@cursor.com>
@github-actions

github-actions Bot commented Aug 6, 2026

Copy link
Copy Markdown

Thanks for your pull request!

A maintainer will review this pull request and trigger functional testing by adding the ok-to-test label.

This comment was auto-generated by GitHub Actions.

@hardengl

hardengl commented Aug 6, 2026

Copy link
Copy Markdown
Author

Verification Summary

This fix addresses issue #480 which has been open since 2024.

Root Cause: The images-are-certified check scans ALL images referenced in rendered Helm templates, including images in templates/tests/ (Helm test hooks annotated with helm.sh/hook: test). These test images are only used during helm test — they never run in production — but they still trigger certification failures if not Red Hat certified.

Fix: Skip images that come from templates annotated with helm.sh/hook: test during the images-are-certified check. These are Helm test hooks, not production workload images.

Testing:

  • Unit tests pass (go test ./...)
  • Logic correctly identifies test hook templates via the helm.sh/hook: test annotation
  • Production images (non-test templates) are still fully validated
  • Charts without test templates are unaffected

Impact: Partners with Helm charts that include test hooks (a common best practice for chart validation) will no longer be blocked from certification due to uncertified test-only images like busybox or curl.

This has been a pain point for partners since 2024. Ready for review and merge. @komish @mgoerens

@hardengl

hardengl commented Aug 6, 2026

Copy link
Copy Markdown
Author

@komish @mgoerens Could you review this when you get a chance? This addresses issue #480 (open since 2024) — test template images causing false certification failures.

@hardengl

hardengl commented Aug 7, 2026

Copy link
Copy Markdown
Author

@komish @mgoerens — Friendly ping. This fixes #480 (open since 2024). Single-function change that filters test template images from the certification check. Could you add the ok-to-test label and review when you have a moment? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

images-are-certified flags non-certified images coming from a chart's tests

1 participant