Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion config/navigation.json
Original file line number Diff line number Diff line change
Expand Up @@ -514,7 +514,19 @@
{
"group": "Helm Charts",
"pages": [
"helm/k8s_reporter"
{
"group": "Kubernetes Reporter",
"pages": [
"helm/k8s_reporter/overview",
"helm/k8s_reporter/prerequisites",
"helm/k8s_reporter/installing",
"helm/k8s_reporter/upgrading",
"helm/k8s_reporter/uninstalling",
"helm/k8s_reporter/tls-proxy",
"helm/k8s_reporter/karpenter",
"helm/k8s_reporter/configuration"
]
}
]
}
]
Expand Down
4 changes: 4 additions & 0 deletions config/redirects.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
[
{
"source": "/helm/k8s_reporter",
"destination": "/helm/k8s_reporter/overview"
},
{
"source": "/tutorials/unauthorized_iac_changes",
"destination": "/tutorials/detecting_unexpected_statefile_changes"
Expand Down
397 changes: 0 additions & 397 deletions helm/k8s_reporter.mdx

This file was deleted.

186 changes: 186 additions & 0 deletions helm/k8s_reporter/configuration.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,186 @@
---
title: Configuration reference
description: All values.yaml options for the Kosli k8s-reporter Helm chart.
---

## General

<ParamField path="affinity" type="object" default="{}">
Affinity rules for scheduling the reporter pod. Supports nodeAffinity, podAffinity and podAntiAffinity.
</ParamField>

<ParamField path="concurrencyPolicy" type="string" default="Replace">
Specifies how to treat concurrent executions of a Job that is created by this CronJob.
</ParamField>

<ParamField path="cronSchedule" type="string" default="*/5 * * * *">
The cron schedule at which the reporter is triggered to report to Kosli.
</ParamField>

<ParamField path="failedJobsHistoryLimit" type="int" default="1">
Specifies the number of failed finished jobs to keep.
</ParamField>

<ParamField path="fullnameOverride" type="string" default="">
Overrides the fullname used for the created k8s resources. It has higher precedence than `nameOverride`.
</ParamField>

<ParamField path="nameOverride" type="string" default="">
Overrides the name used for the created k8s resources. If `fullnameOverride` is provided, it has higher precedence than this one.
</ParamField>

<ParamField path="nodeSelector" type="object" default="{}">
Node labels for scheduling the reporter pod. On EKS with Karpenter, use this to pin the reporter to a stable managed node group (e.g. `eks.amazonaws.com/nodegroup: <name>`) so it does not interfere with node consolidation. See [Running on EKS with Karpenter](/helm/k8s_reporter/karpenter).
</ParamField>

<ParamField path="podAnnotations" type="object" default="{}">
Annotations to add to the CronJob object itself. For pod-level annotations (added to each reporter pod), use `podTemplateAnnotations` instead.
</ParamField>

<ParamField path="podLabels" type="object" default="{}">
Custom labels to add to pods.
</ParamField>

<ParamField path="podTemplateAnnotations" type="object" default="{}">
Annotations to add to the reporter pod template (applied to each Job pod that the CronJob creates).
</ParamField>

<ParamField path="successfulJobsHistoryLimit" type="int" default="3">
Specifies the number of successful finished jobs to keep.
</ParamField>

<ParamField path="tolerations" type="list" default="[]">
Tolerations for scheduling the reporter pod, e.g. to run on a dedicated or tainted node group.
</ParamField>

## Image

<ParamField path="image.pullPolicy" type="string" default="IfNotPresent">
The kosli reporter image pull policy.
</ParamField>

<ParamField path="image.repository" type="string" default="ghcr.io/kosli-dev/cli">
The kosli reporter image repository.
</ParamField>

<ParamField path="image.tag" type="string" default="">
The kosli reporter image tag, overrides the image tag whose default is the chart appVersion.
</ParamField>

## Reporter configuration

<ParamField path="reporterConfig.dryRun" type="bool" default="false">
Whether the dry run mode is enabled or not. In dry run mode, the reporter logs the reports to stdout and does not send them to kosli.
</ParamField>

<ParamField path="reporterConfig.environments" type="list" default="[]">
List of Kosli environments to report to. Each entry has required 'name' and optional namespace selectors. Use one entry to report a single environment; use multiple entries to report to multiple environments with different selectors. Per entry: name (required), namespaces, namespacesRegex, excludeNamespaces, excludeNamespacesRegex (optional). Leave namespace fields unset for an entry to report the entire cluster to that environment.
</ParamField>

<ParamField path="reporterConfig.httpProxy" type="string" default="">
The http proxy url.
</ParamField>

<ParamField path="reporterConfig.kosliOrg" type="string" default="">
The name of the Kosli org.
</ParamField>

<ParamField path="reporterConfig.securityContext" type="object">
The security context for the reporter cronjob. Set to null or {} to disable security context entirely (not recommended). For OpenShift with SCC, explicitly set runAsUser to null to let OpenShift assign the UID from the allowed range. Simply omitting runAsUser from your values override will not work because Helm deep-merges with these defaults. Example OpenShift override: securityContext: allowPrivilegeEscalation: false runAsNonRoot: true runAsUser: null.
</ParamField>

<ParamField path="reporterConfig.securityContext.allowPrivilegeEscalation" type="bool" default="false">
Whether to allow privilege escalation.
</ParamField>

<ParamField path="reporterConfig.securityContext.runAsNonRoot" type="bool" default="true">
Whether to run as non root.
</ParamField>

<ParamField path="reporterConfig.securityContext.runAsUser" type="int" default="1000">
The user id to run as. For OpenShift environments with SCC, set to null (runAsUser: null) to allow automatic UID assignment. Simply omitting this field will not work due to Helm's deep merge with chart defaults.
</ParamField>

## Kosli API token

<ParamField path="kosliApiToken.secretKey" type="string" default="key">
The name of the key in the secret data which contains the Kosli API token.
</ParamField>

<ParamField path="kosliApiToken.secretName" type="string" default="kosli-api-token">
The name of the secret containing the kosli API token.
</ParamField>

## Environment variables

<ParamField path="env" type="object" default="{}">
Map of plain environment variables to inject into the reporter container. For a single-tenant Kosli instance, set `KOSLI_HOST` to `https://INSTANCE_NAME.kosli.com`.
</ParamField>

<ParamField path="extraEnvVars" type="list" default="[]">
Additional environment variables to inject into the reporter container. List of `{name, value}` or `{name, valueFrom}` entries, rendered verbatim into the container env. Supports plain values and valueFrom (`secretKeyRef` / `configMapKeyRef`). Note: entries here are appended after the chart's own env entries; on duplicate names the later entry wins.
</ParamField>

## Volumes

<ParamField path="extraVolumeMounts" type="list" default="[]">
Additional container-level volumeMounts for the reporter container. Rendered verbatim into the container spec alongside the chart's own mounts.
</ParamField>

<ParamField path="extraVolumes" type="list" default="[]">
Additional Pod-level volumes to attach to the reporter pod. Rendered verbatim into the Pod spec alongside the chart's own volumes. Use together with `extraVolumeMounts` to mount Secrets, ConfigMaps, or other volumes into the container.
</ParamField>

## Custom CA

<ParamField path="customCA" type="object">
Convenience wrapper for mounting a corporate / custom CA bundle. See [Running behind a TLS-inspecting proxy](/helm/k8s_reporter/tls-proxy) for usage.
</ParamField>

<ParamField path="customCA.enabled" type="bool" default="false">
Enable mounting a corporate/custom CA bundle into the trust store.
</ParamField>

<ParamField path="customCA.key" type="string" default="ca.crt">
Key within the Secret that holds the PEM-formatted CA certificate (single cert or multi-cert PEM bundle).
</ParamField>

<ParamField path="customCA.secretName" type="string" default="">
Name of an existing Secret in the same namespace containing the CA bundle.
</ParamField>

## Resources

<ParamField path="resources.limits.cpu" type="string" default="100m">
The cpu limit.
</ParamField>

<ParamField path="resources.limits.memory" type="string" default="256Mi">
The memory limit.
</ParamField>

<ParamField path="resources.requests.memory" type="string" default="64Mi">
The memory request.
</ParamField>

## Service account

<ParamField path="serviceAccount.annotations" type="object" default="{}">
Annotations to add to the service account.
</ParamField>

<ParamField path="serviceAccount.create" type="bool" default="true">
Specifies whether a service account should be created.
</ParamField>

<ParamField path="serviceAccount.name" type="string" default="">
The name of the service account to use. If not set and create is true, a name is generated using the fullname template.
</ParamField>

<ParamField path="serviceAccount.permissionScope" type="string" default="cluster">
Specifies whether to create a cluster-wide permissions for the service account or namespace-scoped permissions. allowed values are: \[cluster, namespace].
</ParamField>

***

Autogenerated from chart metadata using [helm-docs v1.14.2](https://github.com/norwoodj/helm-docs/releases/v1.14.2).
64 changes: 64 additions & 0 deletions helm/k8s_reporter/installing.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
---
title: Installing the chart
description: Install the Kosli k8s-reporter Helm chart via the Kosli Helm repository.
---

To install this chart via the Helm chart repository:

<Steps>
<Step title="Add the Kosli helm repo">
```shell
helm repo add kosli https://charts.kosli.com/ && helm repo update
```
</Step>

<Step title="Create a secret for the Kosli API token">
```shell
kubectl create secret generic kosli-api-token --from-literal=key=<your-api-key>
```
</Step>

<Step title="Install the helm chart">
Configure **reporterConfig.environments** (required). Each entry has required `name` and optional `namespaces`, `namespacesRegex`, `excludeNamespaces`, `excludeNamespacesRegex`. Omit namespace fields for an entry to report the entire cluster to that environment.

**One environment, entire cluster:**

```yaml
# values.yaml
reporterConfig:
kosliOrg: <your-org>
environments:
- name: <your-env-name>
```

**One environment, specific namespaces:**

```yaml
reporterConfig:
kosliOrg: <your-org>
environments:
- name: <your-env-name>
namespaces: [namespace1, namespace2]
```

**Multiple environments with different selectors:**

```yaml
reporterConfig:
kosliOrg: <your-org>
environments:
- name: prod-env
namespaces: [prod-ns1, prod-ns2]
- name: staging-env
namespacesRegex: ["^staging-.*"]
- name: infra-env
excludeNamespaces: [prod-ns1, prod-ns2, default]
```

```shell
helm install kosli-reporter kosli/k8s-reporter -f values.yaml
```
</Step>
</Steps>

<Info>See all available options in the [configuration reference](/helm/k8s_reporter/configuration).</Info>
53 changes: 53 additions & 0 deletions helm/k8s_reporter/karpenter.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
title: Running on EKS with Karpenter
description: Keep the k8s-reporter out of Karpenter's way so nodes can still consolidate.
---

By default the reporter runs as a CronJob every 5 minutes. On clusters that use [Karpenter](https://karpenter.sh) for node autoscaling, this frequent scheduling can prevent nodes from being **consolidated** (scaled down).

The cause is Karpenter's `consolidateAfter` timer: Karpenter only consolidates a node once it has seen no pod scheduling activity on it for the configured window. A reporter pod arriving every 5 minutes keeps resetting that timer, so any node whose `consolidateAfter` is longer than the reporter interval never becomes eligible for consolidation (see [karpenter#1921](https://github.com/kubernetes-sigs/karpenter/issues/1921)). This is Karpenter working as designed, not a reporter bug.

Frequent snapshots are what let Kosli surface drift or an unauthorized change quickly, so the best fix keeps the 5-minute cadence and moves the reporter out of Karpenter's way. Widening the interval trades away that detection speed and should be a last resort.

## 1. Pin the reporter to a stable node group (recommended)

If you run a stable managed node group that Karpenter does not manage, schedule the reporter there so it never disturbs Karpenter-managed nodes. Use `nodeSelector`, and `tolerations` if that node group is tainted:

```yaml
nodeSelector:
eks.amazonaws.com/nodegroup: system # your managed node group

tolerations:
- key: dedicated
operator: Equal
value: system
effect: NoSchedule
```

To steer the reporter away from Karpenter-managed nodes instead, use `affinity` (a plain `nodeSelector` cannot express "not on these nodes"):

```yaml
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: karpenter.sh/nodepool
operator: DoesNotExist
```

## 2. Run the reporter out of the cluster

For zero footprint on cluster nodes, run `kosli snapshot k8s` on a schedule outside the cluster (for example a CI cron job) with kubeconfig access, keeping your reporting cadence without placing a pod on the cluster's nodes. See the [Kubernetes environment reporting tutorial](/tutorials/report_k8s_envs).

## 3. Widen the report interval (last resort)

Only if you cannot pin the reporter or move it out of cluster: set `cronSchedule` longer than your NodePool's `consolidateAfter` so nodes get quiet windows long enough to consolidate. This works, but a longer interval widens the window in which a change can go unreported, so prefer the options above.

```yaml
cronSchedule: "*/15 * * * *"
```

<Warning>
`karpenter.sh/do-not-disrupt: "true"` is **not** a fix here. It prevents Karpenter from disrupting the pod, which protects a mid-run report from interruption but makes consolidation of that node *less* likely, not more. Likewise `cluster-autoscaler.kubernetes.io/safe-to-evict` only affects the Kubernetes Cluster Autoscaler and is ignored by Karpenter.
</Warning>
27 changes: 27 additions & 0 deletions helm/k8s_reporter/overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
title: Kubernetes Reporter Helm Chart
description: A Helm chart for installing the Kosli K8S reporter as a cronjob.
---

<Info>
This reference applies to **chart version 2.3.1**, which defaults to CLI **v2.12.0** via `appVersion`. Override with `image.tag`.
</Info>

A Helm chart for installing the Kosli K8S reporter as a CronJob.
The chart allows you to create a Kubernetes cronjob and all its necessary RBAC to report running images to Kosli at a given cron schedule.

Configuration is done via **reporterConfig.environments**: a list of Kosli environments to report to. Each entry has a required `name` and optional namespace selectors. Use one entry for a single environment, or multiple entries to report to different environments with different selectors.

<Note>Chart source can be found at [GitHub](https://github.com/kosli-dev/cli/tree/main/charts/k8s-reporter).</Note>

## In this section

<CardGroup cols={2}>
<Card title="Prerequisites" icon="clipboard-check" href="/helm/k8s_reporter/prerequisites" />
<Card title="Installing the chart" icon="download" href="/helm/k8s_reporter/installing" />
<Card title="Upgrading the chart" icon="arrow-up" href="/helm/k8s_reporter/upgrading" />
<Card title="Uninstalling the chart" icon="trash" href="/helm/k8s_reporter/uninstalling" />
<Card title="Running behind a TLS-inspecting proxy" icon="shield" href="/helm/k8s_reporter/tls-proxy" />
<Card title="Running on EKS with Karpenter" icon="server" href="/helm/k8s_reporter/karpenter" />
<Card title="Configuration reference" icon="gear" href="/helm/k8s_reporter/configuration" />
</CardGroup>
9 changes: 9 additions & 0 deletions helm/k8s_reporter/prerequisites.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
title: Prerequisites
description: Requirements for installing the Kosli k8s-reporter Helm chart.
---

* A Kubernetes cluster (minimum supported version is `v1.21`)
* Helm v3.0+
* If you want to report artifacts from just one namespace, you need to have permissions to `get` and `list` pods in that namespace.
* If you want to report artifacts from multiple namespaces or entire cluster, you need to have cluster-wide permissions to `get` and `list` pods.
Loading