feat: verify chart RBAC does not drift from config/rbac - #372
feat: verify chart RBAC does not drift from config/rbac#372tejassinghbhati wants to merge 1 commit into
Conversation
hack/verify-chart-drift.sh only diffed the CRD, so nothing caught the chart granting different permissions than the controller declares. kubernetes-sigs#350 was exactly that, the chart's manager ClusterRole was missing the events.k8s.io group and every taint event was refused on a Helm install while kustomize worked fine. RBAC is the case most likely to drift, because config/rbac/role.yaml is generated by controller-gen from the kubebuilder markers and updates itself, while the chart's ClusterRole is hand written and only updates when someone remembers. The two cannot be compared textually. The generated roles carry short names and kustomize labels, the chart templates the name from the release and adds Helm labels, so the files never match even when the permissions are identical. Instead the check renders the chart, matches each Role and ClusterRole by name ignoring the release prefix, and compares only the rules. Rules are normalised first, since the order of apiGroups, resources and verbs within a rule means nothing to the API server and should not be reported as drift. Against main this reports the missing events.k8s.io grant from kubernetes-sigs#350 and exits non-zero. With kubernetes-sigs#351 applied it compares all seven roles and passes. Signed-off-by: tejassinghbhati <tejassinghbhati077@gmail.com>
✅ Deploy Preview for node-readiness-controller canceled.
|
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: tejassinghbhati The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Hi @tejassinghbhati. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Regular contributors should join the org to skip this step. Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Description
Follows up on @ajaysundark's question in #351 about whether
hack/verify-chart-drift.shcould cover more than CRDs. This extends it to RBAC.The script previously diffed only the bundled CRD, so nothing caught the chart granting different permissions than the controller declares. #350 was exactly that, and it went unnoticed precisely because nothing compares the two.
RBAC is the case most likely to drift.
config/rbac/role.yamlis generated by controller-gen from the kubebuilder markers, so it updates itself whenever the controller's permissions change. The chart's ClusterRole is hand written, so it only updates when someone remembers. Every future permission change is another chance for the chart to fall behind, and the failure is silent apart from whatever the missing grant covered.A plain
diffdoes not work here. The generated roles carry short names and kustomize labels, the chart templates the name from the release name and adds Helm labels, so the files never match textually even when the permissions are identical. The check therefore renders the chart, matches each Role and ClusterRole by name ignoring the release prefix, and compares only the rules.Rules are normalised before comparison, sorting the entries within each rule and then the rules themselves, because the order of
apiGroups,resourcesandverbsmeans nothing to the API server and should not be reported as drift. Without that the check would be noisy enough that people would learn to ignore it.Output points at the offending permission rather than dumping both role definitions:
A role the chart renders that has no counterpart in config is reported as a note rather than a failure, since the chart may legitimately ship something kustomize does not.
Merge order. This depends on #351. Run against
mainas it stands today the check fails, because it correctly finds the missingevents.k8s.iogrant that #351 fixes. That failure is the proof it works, but it does mean this should land after #351, or CI on main will flag a bug that already has a fix in flight.Scope. Roles and ClusterRoles only, not bindings, subjects, the Deployment, Services or webhook configuration. I checked all of those by hand while reviewing #351 and they currently agree, so RBAC seemed the right place to start given it is the one that has already bitten us. Easy to extend later if that is wanted.
Related Issue
Fixes #371
Type of Change
/kind feature
Testing
Verified both directions against a real drift rather than a synthetic one.
Against
main, where the chart is still missing the events grant, the check fails with the output shown above and exits 1. With #351's template change applied, it passes:All seven roles are compared:
manager-role,leader-election-role,metrics-auth-role,metrics-reader, and thenodereadinessruleadmin, editor and viewer roles.golangci-lintv2.12.1 reports 0 issues across the repo including the new tool,gofmtis clean, and the boilerplate header matcheshack/boilerplate/boilerplate.go.txtexactly.bash -npasses on the modified script.One note on local verification: I could not run the script end to end on Windows, because
make manifestsfails there withno Go files in E:\...from controller-gen's path handling. That reproduces identically on a clean checkout ofmain, so it is pre-existing and unrelated to this change. I ran the RBAC section verbatim instead, and CI runs on ubuntu-latest where the CRD step works.The Helm workflow already sets up Helm, Go and Python before invoking this script, so no workflow change is needed. If helm is missing locally the script now fails with a message pointing at
make ensure-helm-installrather than skipping silently.Checklist
make testpassesmake lintpassesDoes this PR introduce a user-facing change?