logger: record field sensitivity instead of destroying it

[paulwe]

Why

(logger.sensitivity) annotations are enforced by replacing the value with "<redacted>" (proto.go). That is right for console and JSON logs, but it also destroys data that a columnar sink could hold under a tag and filter per consumer. This is the protocol half of projecting sensitivity into the attribute wire format in backend-common/observability.

What

Adds SensitiveObjectEncoder / SensitiveArrayEncoder in logger/sensitive.go. An encoder that can record a value's sensitivity returns a sub-encoder for a level, or nil to refuse it. protoMarshaller asks via ObjectEncoderFor and redacts only what the sink refuses.

• No behaviour change for existing encoders. With no implementors ObjectEncoderFor always returns nil and the redaction path runs as before. proto_test.go exercises that path through zapcore.MapObjectEncoder and is unchanged apart from the UnredactedProto removals.
• The assertion is on the encoder, not a new marshaler interface. zapcore's Field.AddTo calls MarshalLogObject directly for InlineMarshalerType, so an encoder-side hook can never intercept a zap.Inline'd proto.
• Sensitivity propagates through the encoder, not the marshaller. A tagged encoder derives tagged children, so a sensitive message tags its whole subtree without protoMarshaller carrying state for it. A SECRET field nested under a PII one still redacts, because the sink refuses SECRET at any depth.
• marshalSummary restructured, not reordered. Maps, lists and nested messages continue before the sensitivity decision. They disclose no field contents, and hoisting the decision above them would replace a non-disclosing fooCount with a redaction.

logger/sensitive_test.go adds a path-recording encoder double (a flat dot-joined recorder mirroring how the columnar sink flattens nesting) covering: PII stored with the real value rather than its redact_format output, SECRET always redacted, level inherited through a map, SECRET-under-PII, and summary mode tagging scalars while leaving counts untagged.

Breaking

UnredactedProto is removed. Against a tagging sink it was identical to Proto; its only remaining behaviour was exposing PII in the clear to sinks that cannot tag. maxLevel goes with it, since every constructor now leaves it at UNSPECIFIED.

cloud has 30 call sites that need switching to Proto before it can take this bump. For the columnar sink that is a no-op; for console output those fields move from cleartext to <redacted>, which is the safe direction.

Note on rollout

A decoder that predates a sensitivity level cannot filter it — it ignores the tagged container tokens and so skips their key-depth adjustment. Every reader of an affected table has to be deployed before any writer starts tagging. That ordering is enforced by deploy sequencing, not by a flag.

[changeset-bot]

⚠️ No Changeset found

Latest commit: c319b50

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR