feat(core): emit Kubernetes Events on Agent/ModelConfig/MCPServerTool/RemoteMCPServer reconciles

[braghettos]

What

Adds Kubernetes Events on reconcile to the Agent, ModelConfig, MCPServerTool, and RemoteMCPServer controllers, so users can see reconcile outcomes with kubectl describe / kubectl get events. This is the second of the three observability gaps proposed in #2148.

Today these four controllers emit no Events (unlike agentharness-substrate-controller, which already does), so a bad Agent/ModelConfig/MCPServer gives no signal in kubectl describe.

How

Each controller gains an optional Client + Recorder events.EventRecorder and a small nil-guarded recordEvent helper that re-fetches the reconciled object and emits an Event against it, mirroring the existing substrate-controller pattern.

Emitted events:

Controller	Success	Failure
Agent	Normal Accepted	Warning ReconcileFailed
ModelConfig	Normal Accepted	Warning ReconcileFailed
RemoteMCPServer	Normal Accepted	Warning ReconcileFailed
MCPServerTool	Normal ToolsDiscovered	Warning ValidationFailed (spec validation) / Warning ReconcileFailed (transient)

Recorders are wired from pkg/app/app.go via mgr.GetEventRecorder(...).

Additive / no behaviour change

• recordEvent is a no-op when either Recorder or Client is nil, so existing tests and any caller that doesn't wire them are unaffected.
• No new gate/env var — Events are a cheap, additive status surface.
• RBAC is already covered by the leader-election role (events: create,patch), so no RBAC changes are needed.
• No new dependencies (go.mod unchanged); k8s.io/client-go/tools/events and apierrors are already used in-tree.

Testing

• New mcp_server_tool_controller_events_test.go — asserts the correct event type/reason is emitted for each of the three MCPServerTool outcomes (ToolsDiscovered / ValidationFailed / ReconcileFailed) using a fake client + events.NewFakeRecorder, plus a no-Recorder no-panic case.
• go build ./core/..., go test -race -skip 'TestE2E.*' ./core/internal/controller/..., and golangci-lint run all pass.

Notes

Opening as draft / work-in-progress per CONTRIBUTING; happy to adjust event reasons/actions. The remaining gap from #2148 (controller log→OTLP bridge) will follow as a separate PR. Companion PR for gap 1 (token metrics): #2149.

Refs: #2148

[Copilot]

Pull request overview

Adds Kubernetes Event emission to several go/core controllers so reconcile outcomes are visible via kubectl describe / kubectl get events, wiring recorders from the controller manager.

Changes:

• Wire Client + EventRecorder into Agent/ModelConfig/MCPServerTool/RemoteMCPServer controllers from pkg/app/app.go.
• Emit Normal/Warning Kubernetes Events on reconcile success/failure for those controllers via a nil-guarded recordEvent helper.
• Add a unit test covering MCPServerTool event emission paths.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
go/core/pkg/app/app.go	Wires manager client + event recorders into the four controllers.
go/core/internal/controller/agent_controller.go	Emits Accepted/ReconcileFailed Events for Agent reconciles via recordEvent.
go/core/internal/controller/modelconfig_controller.go	Emits Accepted/ReconcileFailed Events for ModelConfig reconciles via recordEvent.
go/core/internal/controller/mcp_server_tool_controller.go	Emits ToolsDiscovered/ValidationFailed/ReconcileFailed Events for MCPServer tool discovery reconciles.
go/core/internal/controller/remote_mcp_server_controller.go	Emits Accepted/ReconcileFailed Events for RemoteMCPServer reconciles via recordEvent.
go/core/internal/controller/mcp_server_tool_controller_events_test.go	Adds tests validating MCPServerTool emits expected event type/reason.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[braghettos]

Addressed the Copilot feedback about Event flooding: the MCPServerTool and RemoteMCPServer controllers requeue every 60s, so they now emit the Normal success Event only on a meaningful transition — a pending spec change (generation != observedGeneration, captured before reconcile) — instead of on every periodic refresh. Warning Events (ReconcileFailed / ValidationFailed) still emit each time. Added a test asserting no success Event on a periodic no-op refresh. The Agent/ModelConfig controllers are unaffected since they don't requeue periodically.

[EItanya]

Overall the functionality makes sense. However I'm wondering if we should add this directly to the reconciler since that code already does the Get required to ensure that the object exists.