Skip to content

feat(stats)!: per-field cardinality limits - #2211

Open
Eldolfin wants to merge 40 commits into
mainfrom
oscarld/stats-per-key-cardinality-limit
Open

feat(stats)!: per-field cardinality limits#2211
Eldolfin wants to merge 40 commits into
mainfrom
oscarld/stats-per-key-cardinality-limit

Conversation

@Eldolfin

@Eldolfin Eldolfin commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Implement per-key cardinality limits to stats computation.

Also:

  • fix overflow key peer_tags: [] -> [tracer_blocked_value]
  • fix encode_grouped_stats meta with empty value: key: -> key (breaking change ?)

Motivation

CSS spec compliance.

Additional Notes

Todo:

How to test the change?

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

📚 Documentation Check Results

⚠️ 3454 documentation warning(s) found

📦 libdd-data-pipeline-ffi - 1336 warning(s)

📦 libdd-data-pipeline - 1225 warning(s)

📦 libdd-trace-stats - 893 warning(s)


Updated: 2026-07-28 14:23:12 UTC | Commit: a14f379 | missing-docs job results

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Jul 8, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 97.27%
Overall Coverage: 74.72% (+0.13%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 37ce354 | Docs | Datadog PR Page | Give us feedback!

@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

🔒 Cargo Deny Results

⚠️ 8 issue(s) found, showing only errors (advisories, bans, sources)

📦 libdd-data-pipeline-ffi - 3 error(s)

Show output
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:78:1
   │
78 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   │   └── libdd-data-pipeline-ffi v38.0.0
                     │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │   │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── (dev) libdd-data-pipeline-ffi v38.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-common-ffi v38.0.0
                     │   │   └── libdd-data-pipeline-ffi v38.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:226:1
    │
226 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   │   └── libdd-data-pipeline-ffi v38.0.0
      │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-data-pipeline-ffi v38.0.0 (*)
      │   │   │   ├── libdd-dogstatsd-client v4.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── (dev) libdd-data-pipeline-ffi v38.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-common-ffi v38.0.0
      │   │   └── libdd-data-pipeline-ffi v38.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── libdd-data-pipeline-ffi v38.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:296:1
    │
296 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0
                  └── libdd-data-pipeline-ffi v38.0.0

advisories FAILED, bans ok, sources ok

📦 libdd-data-pipeline - 3 error(s)

Show output
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:75:1
   │
75 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-data-pipeline v7.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-data-pipeline v7.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-data-pipeline v7.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   ├── libdd-data-pipeline v7.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:218:1
    │
218 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-data-pipeline v7.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-dogstatsd-client v4.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   │       └── libdd-data-pipeline v7.0.0 (*)
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   ├── libdd-data-pipeline v7.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-data-pipeline v7.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-data-pipeline v7.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   ├── libdd-data-pipeline v7.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── libdd-data-pipeline v7.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:287:1
    │
287 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v7.0.0

advisories FAILED, bans ok, sources ok

📦 libdd-trace-stats - 2 error(s)

Show output
error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:66:1
   │
66 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.1.0
                     │   ├── libdd-capabilities-impl v3.0.0
                     │   │   ├── libdd-shared-runtime v2.0.0
                     │   │   │   ├── libdd-telemetry v6.0.0
                     │   │   │   │   └── libdd-trace-stats v6.0.0
                     │   │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
                     │   │   ├── libdd-trace-stats v6.0.0 (*)
                     │   │   └── libdd-trace-utils v9.0.0
                     │   │       ├── libdd-trace-obfuscation v5.0.0
                     │   │       │   └── libdd-trace-stats v6.0.0 (*)
                     │   │       ├── libdd-trace-stats v6.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
                     │   ├── libdd-dogstatsd-client v4.0.0
                     │   │   └── libdd-trace-stats v6.0.0 (*)
                     │   ├── libdd-shared-runtime v2.0.0 (*)
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   ├── libdd-trace-obfuscation v5.0.0 (*)
                     │   ├── libdd-trace-stats v6.0.0 (*)
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.0
                     │   ├── libdd-telemetry v6.0.0 (*)
                     │   └── libdd-trace-stats v6.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.0
                     │   └── libdd-trace-utils v9.0.0 (*)
                     ├── (dev) libdd-trace-obfuscation v5.0.0 (*)
                     ├── (dev) libdd-trace-stats v6.0.0 (*)
                     └── (dev) libdd-trace-utils v9.0.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:198:1
    │
198 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.1.0
      │   ├── libdd-capabilities-impl v3.0.0
      │   │   ├── libdd-shared-runtime v2.0.0
      │   │   │   ├── libdd-telemetry v6.0.0
      │   │   │   │   └── libdd-trace-stats v6.0.0
      │   │   │   └── libdd-trace-stats v6.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v6.0.0 (*)
      │   │   ├── libdd-trace-stats v6.0.0 (*)
      │   │   └── libdd-trace-utils v9.0.0
      │   │       ├── libdd-trace-obfuscation v5.0.0
      │   │       │   └── libdd-trace-stats v6.0.0 (*)
      │   │       ├── libdd-trace-stats v6.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v9.0.0 (*)
      │   ├── libdd-dogstatsd-client v4.0.0
      │   │   └── libdd-trace-stats v6.0.0 (*)
      │   ├── libdd-shared-runtime v2.0.0 (*)
      │   ├── libdd-telemetry v6.0.0 (*)
      │   ├── libdd-trace-obfuscation v5.0.0 (*)
      │   ├── libdd-trace-stats v6.0.0 (*)
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.0
      │   ├── libdd-telemetry v6.0.0 (*)
      │   └── libdd-trace-stats v6.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.0
      │   └── libdd-trace-utils v9.0.0 (*)
      ├── (dev) libdd-trace-stats v6.0.0 (*)
      ├── libdd-trace-utils v9.0.0 (*)
      └── proptest v1.5.0
          └── (dev) libdd-tinybytes v1.1.1
              ├── (dev) libdd-tinybytes v1.1.1 (*)
              └── libdd-trace-utils v9.0.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-07-28 14:24:39 UTC | Commit: a14f379 | dependency-check job results

@Eldolfin

Eldolfin commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from a5a4786 to 2845fa8 Compare July 8, 2026 12:50
@Eldolfin
Eldolfin changed the base branch from main to oscarld/fix-css-obfuscation-logic July 8, 2026 12:51
@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from 2845fa8 to fca0f65 Compare July 8, 2026 12:52

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a5a4786925

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-data-pipeline/src/trace_exporter/builder.rs
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
@dd-octo-sts

dd-octo-sts Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 87.14 MB 87.25 MB +.12% (+111.73 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.01 MB 8.01 MB +0% (+48 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.77 MB 10.78 MB +.03% (+3.46 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 98.41 MB 98.53 MB +.11% (+117.05 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 26.00 MB 26.03 MB +.10% (+28.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.60 KB +.47% (+430 B) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 187.93 MB 188.06 MB +.07% (+136.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 979.20 MB 979.40 MB +.02% (+204.33 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.47 MB 8.48 MB +.10% (+9.50 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 89.18 KB 89.60 KB +.47% (+430 B) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 25.05 MB 25.08 MB +.12% (+32.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 49.85 MB 49.90 MB +.10% (+55.02 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 22.64 MB 22.66 MB +.09% (+23.00 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 90.58 KB 91.00 KB +.46% (+434 B) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 192.63 MB 192.78 MB +.07% (+152.00 KB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 968.15 MB 967.88 MB --.02% (-275.59 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.54 MB 6.55 MB +.14% (+10.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 90.58 KB 91.00 KB +.46% (+434 B) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.91 MB 26.94 MB +.11% (+32.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 47.44 MB 47.49 MB +.11% (+54.09 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 77.74 MB 77.84 MB +.12% (+102.44 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.91 MB 8.93 MB +.17% (+16.06 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 93.24 MB 93.34 MB +.11% (+106.30 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.85 MB 10.86 MB +.09% (+10.61 KB) 🔍

@Eldolfin
Eldolfin marked this pull request as ready for review July 8, 2026 15:15
@Eldolfin
Eldolfin requested review from a team as code owners July 8, 2026 15:15

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d82ebfd172

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-data-pipeline-ffi/src/trace_exporter.rs
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs
Comment thread libdd-trace-stats/src/span_concentrator/mod.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/aggregation.rs Outdated
Comment thread libdd-trace-stats/src/span_concentrator/mod.rs
@Eldolfin
Eldolfin force-pushed the oscarld/stats-per-key-cardinality-limit branch from efe280f to 83043a4 Compare July 10, 2026 12:14
@pr-commenter

pr-commenter Bot commented Jul 10, 2026

Copy link
Copy Markdown

Benchmarks

Comparison

Benchmark execution time: 2026-07-28 14:32:39

Comparing candidate commit 37ce354 in PR branch oscarld/stats-per-key-cardinality-limit with baseline commit 4a0e5bc in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 22 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:trace_buffer/4_senders/no_delay

  • 🟥 execution_time [+105.507µs; +135.268µs] or [+4.501%; +5.771%]
  • 🟥 throughput [-85444.113op/s; -66526.998op/s] or [-5.556%; -4.326%]

Benchmark execution time: 2026-07-28 14:30:22

Comparing candidate commit 37ce354 in PR branch oscarld/stats-per-key-cardinality-limit with baseline commit 4a0e5bc in branch main.

Found 0 performance improvements and 1 performance regressions! Performance is the same for 2 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:concentrator/add_spans_to_concentrator

  • 🟥 execution_time [+1.031ms; +1.038ms] or [+11.951%; +12.031%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 37ce354 1785248417 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 56.510ms 56.900ms ± 0.224ms 56.868ms ± 0.143ms 57.022ms 57.324ms 57.542ms 57.666ms 1.40% 0.672 0.474 0.39% 0.016ms 1 200
trace_buffer/1_senders/10us_delay throughput 15607.052op/s 15817.552op/s ± 62.235op/s 15826.107op/s ± 39.887op/s 15862.524op/s 15903.116op/s 15919.922op/s 15926.354op/s 0.63% -0.649 0.419 0.39% 4.401op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.028ms 50.083ms ± 0.029ms 50.082ms ± 0.012ms 50.092ms 50.138ms 50.176ms 50.190ms 0.22% 1.063 2.058 0.06% 0.002ms 1 200
trace_buffer/1_senders/1us_delay throughput 17931.709op/s 17970.075op/s ± 10.242op/s 17970.581op/s ± 4.187op/s 17975.256op/s 17984.947op/s 17989.085op/s 17989.886op/s 0.11% -1.058 2.046 0.06% 0.724op/s 1 200
trace_buffer/1_senders/no_delay execution_time 338.212µs 344.604µs ± 5.270µs 343.444µs ± 2.442µs 346.282µs 351.469µs 360.810µs 392.322µs 14.23% 4.332 33.050 1.53% 0.373µs 1 200
trace_buffer/1_senders/no_delay throughput 2294032.033op/s 2612266.345op/s ± 37656.707op/s 2620517.508op/s ± 18637.674op/s 2634209.951op/s 2651728.816op/s 2656302.241op/s 2661048.998op/s 1.55% -3.670 25.088 1.44% 2662.731op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 57.180ms 57.557ms ± 0.075ms 57.563ms ± 0.043ms 57.606ms 57.659ms 57.686ms 57.742ms 0.31% -1.482 4.785 0.13% 0.005ms 1 200
trace_buffer/2_senders/10us_delay throughput 31173.128op/s 31273.583op/s ± 40.690op/s 31270.204op/s ± 23.321op/s 31290.732op/s 31341.052op/s 31423.070op/s 31479.465op/s 0.67% 1.500 4.866 0.13% 2.877op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.063ms 50.118ms ± 0.036ms 50.110ms ± 0.020ms 50.131ms 50.195ms 50.221ms 50.229ms 0.24% 1.093 0.823 0.07% 0.003ms 1 200
trace_buffer/2_senders/1us_delay throughput 35835.790op/s 35915.588op/s ± 25.768op/s 35920.649op/s ± 14.306op/s 35934.347op/s 35946.421op/s 35952.590op/s 35954.704op/s 0.09% -1.090 0.815 0.07% 1.822op/s 1 200
trace_buffer/2_senders/no_delay execution_time 861.007µs 910.775µs ± 18.809µs 913.562µs ± 11.296µs 923.061µs 936.576µs 944.814µs 1004.105µs 9.91% 0.097 2.275 2.06% 1.330µs 1 200
trace_buffer/2_senders/no_delay throughput 1792642.100op/s 1977176.773op/s ± 40853.959op/s 1970310.644op/s ± 24267.560op/s 2000403.476op/s 2055547.747op/s 2066251.739op/s 2090576.147op/s 6.10% 0.141 1.510 2.06% 2888.811op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 57.537ms 57.704ms ± 0.082ms 57.689ms ± 0.050ms 57.748ms 57.869ms 57.943ms 57.966ms 0.48% 0.842 0.409 0.14% 0.006ms 1 200
trace_buffer/4_senders/10us_delay throughput 62105.896op/s 62387.359op/s ± 88.466op/s 62403.990op/s ± 53.701op/s 62454.500op/s 62497.328op/s 62530.372op/s 62568.355op/s 0.26% -0.835 0.393 0.14% 6.255op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 50.112ms 50.173ms ± 0.035ms 50.167ms ± 0.018ms 50.186ms 50.244ms 50.281ms 50.311ms 0.29% 1.260 1.840 0.07% 0.002ms 1 200
trace_buffer/4_senders/1us_delay throughput 71555.102op/s 71751.545op/s ± 49.666op/s 71760.821op/s ± 25.261op/s 71782.841op/s 71809.418op/s 71826.364op/s 71838.703op/s 0.11% -1.255 1.824 0.07% 3.512op/s 1 200
trace_buffer/4_senders/no_delay execution_time 2.281ms 2.464ms ± 0.066ms 2.472ms ± 0.046ms 2.514ms 2.556ms 2.575ms 2.672ms 8.10% -0.334 -0.096 2.66% 0.005ms 1 200
trace_buffer/4_senders/no_delay throughput 1347122.738op/s 1461874.145op/s ± 39451.940op/s 1456223.117op/s ± 26903.929op/s 1489988.673op/s 1532549.425op/s 1561488.373op/s 1578149.790op/s 8.37% 0.474 -0.051 2.69% 2789.673op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 57.868ms 58.128ms ± 0.154ms 58.103ms ± 0.095ms 58.213ms 58.437ms 58.577ms 58.603ms 0.86% 0.961 0.656 0.26% 0.011ms 1 200
trace_buffer/8_senders/10us_delay throughput 122860.355op/s 123865.738op/s ± 328.005op/s 123918.262op/s ± 203.095op/s 124113.454op/s 124290.713op/s 124366.603op/s 124420.331op/s 0.41% -0.947 0.623 0.26% 23.193op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 50.209ms 50.295ms ± 0.067ms 50.276ms ± 0.026ms 50.313ms 50.420ms 50.501ms 50.771ms 0.98% 2.942 13.877 0.13% 0.005ms 1 200
trace_buffer/8_senders/1us_delay throughput 141814.011op/s 143154.904op/s ± 190.099op/s 143209.885op/s ± 74.470op/s 143267.895op/s 143325.900op/s 143379.116op/s 143401.211op/s 0.13% -2.913 13.608 0.13% 13.442op/s 1 200
trace_buffer/8_senders/no_delay execution_time 5.760ms 6.339ms ± 0.176ms 6.389ms ± 0.089ms 6.468ms 6.531ms 6.562ms 6.666ms 4.34% -1.169 0.784 2.77% 0.012ms 1 200
trace_buffer/8_senders/no_delay throughput 1080148.466op/s 1136701.178op/s ± 32730.877op/s 1127004.496op/s ± 15533.745op/s 1150605.377op/s 1210978.113op/s 1230577.317op/s 1250000.362op/s 10.91% 1.291 1.128 2.87% 2314.423op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [56.869ms; 56.931ms] or [-0.055%; +0.055%] None None None
trace_buffer/1_senders/10us_delay throughput [15808.927op/s; 15826.177op/s] or [-0.055%; +0.055%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.079ms; 50.087ms] or [-0.008%; +0.008%] None None None
trace_buffer/1_senders/1us_delay throughput [17968.656op/s; 17971.494op/s] or [-0.008%; +0.008%] None None None
trace_buffer/1_senders/no_delay execution_time [343.874µs; 345.334µs] or [-0.212%; +0.212%] None None None
trace_buffer/1_senders/no_delay throughput [2607047.488op/s; 2617485.202op/s] or [-0.200%; +0.200%] None None None
trace_buffer/2_senders/10us_delay execution_time [57.546ms; 57.567ms] or [-0.018%; +0.018%] None None None
trace_buffer/2_senders/10us_delay throughput [31267.943op/s; 31279.222op/s] or [-0.018%; +0.018%] None None None
trace_buffer/2_senders/1us_delay execution_time [50.113ms; 50.123ms] or [-0.010%; +0.010%] None None None
trace_buffer/2_senders/1us_delay throughput [35912.016op/s; 35919.159op/s] or [-0.010%; +0.010%] None None None
trace_buffer/2_senders/no_delay execution_time [908.169µs; 913.382µs] or [-0.286%; +0.286%] None None None
trace_buffer/2_senders/no_delay throughput [1971514.807op/s; 1982838.738op/s] or [-0.286%; +0.286%] None None None
trace_buffer/4_senders/10us_delay execution_time [57.693ms; 57.715ms] or [-0.020%; +0.020%] None None None
trace_buffer/4_senders/10us_delay throughput [62375.098op/s; 62399.619op/s] or [-0.020%; +0.020%] None None None
trace_buffer/4_senders/1us_delay execution_time [50.168ms; 50.178ms] or [-0.010%; +0.010%] None None None
trace_buffer/4_senders/1us_delay throughput [71744.661op/s; 71758.428op/s] or [-0.010%; +0.010%] None None None
trace_buffer/4_senders/no_delay execution_time [2.455ms; 2.473ms] or [-0.370%; +0.370%] None None None
trace_buffer/4_senders/no_delay throughput [1456406.486op/s; 1467341.805op/s] or [-0.374%; +0.374%] None None None
trace_buffer/8_senders/10us_delay execution_time [58.106ms; 58.149ms] or [-0.037%; +0.037%] None None None
trace_buffer/8_senders/10us_delay throughput [123820.280op/s; 123911.196op/s] or [-0.037%; +0.037%] None None None
trace_buffer/8_senders/1us_delay execution_time [50.286ms; 50.305ms] or [-0.018%; +0.018%] None None None
trace_buffer/8_senders/1us_delay throughput [143128.558op/s; 143181.250op/s] or [-0.018%; +0.018%] None None None
trace_buffer/8_senders/no_delay execution_time [6.315ms; 6.364ms] or [-0.385%; +0.385%] None None None
trace_buffer/8_senders/no_delay throughput [1132164.993op/s; 1141237.362op/s] or [-0.399%; +0.399%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 37ce354 1785248417 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 15.672µs 16.075µs ± 0.162µs 16.058µs ± 0.093µs 16.158µs 16.394µs 16.537µs 16.657µs 3.73% 0.684 1.118 1.01% 0.011µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [16.053µs; 16.098µs] or [-0.140%; +0.140%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 37ce354 1785248417 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 9.634ms 9.661ms ± 0.016ms 9.658ms ± 0.012ms 9.673ms 9.692ms 9.703ms 9.711ms 0.56% 0.748 -0.077 0.17% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [9.659ms; 9.663ms] or [-0.023%; +0.023%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 37ce354 1785248417 oscarld/stats-per-key-cardinality-limit
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.057µs 1.073µs ± 0.007µs 1.074µs ± 0.005µs 1.078µs 1.082µs 1.083µs 1.089µs 1.40% -0.428 -0.614 0.63% 0.000µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [1.072µs; 1.074µs] or [-0.088%; +0.088%] None None None

Baseline

Baseline benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 4a0e5bc 1785248590 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
trace_buffer/1_senders/10us_delay execution_time 56.413ms 56.872ms ± 0.208ms 56.847ms ± 0.153ms 57.021ms 57.203ms 57.399ms 57.518ms 1.18% 0.407 -0.070 0.37% 0.015ms 1 200
trace_buffer/1_senders/10us_delay throughput 15647.161op/s 15825.122op/s ± 57.875op/s 15832.071op/s ± 42.628op/s 15862.701op/s 15914.848op/s 15932.941op/s 15953.867op/s 0.77% -0.388 -0.096 0.36% 4.092op/s 1 200
trace_buffer/1_senders/1us_delay execution_time 50.026ms 50.079ms ± 0.032ms 50.080ms ± 0.014ms 50.089ms 50.142ms 50.151ms 50.197ms 0.23% 0.651 0.466 0.06% 0.002ms 1 200
trace_buffer/1_senders/1us_delay throughput 17929.523op/s 17971.611op/s ± 11.602op/s 17971.151op/s ± 5.113op/s 17977.496op/s 17988.678op/s 17989.908op/s 17990.766op/s 0.11% -0.648 0.458 0.06% 0.820op/s 1 200
trace_buffer/1_senders/no_delay execution_time 335.141µs 343.720µs ± 5.870µs 343.274µs ± 1.757µs 344.910µs 348.066µs 353.887µs 413.787µs 20.54% 8.639 99.938 1.70% 0.415µs 1 200
trace_buffer/1_senders/no_delay throughput 2175029.869op/s 2619079.152op/s ± 39384.851op/s 2621813.523op/s ± 13487.717op/s 2635399.735op/s 2654602.053op/s 2670379.670op/s 2685436.665op/s 2.43% -7.313 79.250 1.50% 2784.930op/s 1 200
trace_buffer/2_senders/10us_delay execution_time 57.288ms 57.551ms ± 0.071ms 57.557ms ± 0.046ms 57.586ms 57.666ms 57.711ms 57.714ms 0.27% -0.532 1.335 0.12% 0.005ms 1 200
trace_buffer/2_senders/10us_delay throughput 31188.393op/s 31276.925op/s ± 38.569op/s 31273.398op/s ± 24.801op/s 31301.242op/s 31344.285op/s 31389.743op/s 31420.407op/s 0.47% 0.543 1.356 0.12% 2.727op/s 1 200
trace_buffer/2_senders/1us_delay execution_time 50.060ms 50.122ms ± 0.039ms 50.116ms ± 0.025ms 50.141ms 50.198ms 50.225ms 50.239ms 0.24% 0.833 0.178 0.08% 0.003ms 1 200
trace_buffer/2_senders/1us_delay throughput 35829.054op/s 35912.265op/s ± 27.819op/s 35916.670op/s ± 18.050op/s 35933.784op/s 35947.049op/s 35952.238op/s 35956.664op/s 0.11% -0.830 0.170 0.08% 1.967op/s 1 200
trace_buffer/2_senders/no_delay execution_time 840.067µs 891.933µs ± 19.100µs 892.972µs ± 13.754µs 905.125µs 918.972µs 939.071µs 966.394µs 8.22% 0.119 0.801 2.14% 1.351µs 1 200
trace_buffer/2_senders/no_delay throughput 1862594.548op/s 2019009.819op/s ± 43205.868op/s 2015741.586op/s ± 30780.680op/s 2050634.891op/s 2090525.380op/s 2121624.528op/s 2142686.507op/s 6.30% 0.053 0.610 2.13% 3055.116op/s 1 200
trace_buffer/4_senders/10us_delay execution_time 57.544ms 57.696ms ± 0.074ms 57.690ms ± 0.046ms 57.737ms 57.840ms 57.946ms 57.953ms 0.46% 0.768 1.095 0.13% 0.005ms 1 200
trace_buffer/4_senders/10us_delay throughput 62119.146op/s 62396.098op/s ± 79.998op/s 62402.222op/s ± 50.163op/s 62450.186op/s 62515.465op/s 62543.620op/s 62560.925op/s 0.25% -0.759 1.071 0.13% 5.657op/s 1 200
trace_buffer/4_senders/1us_delay execution_time 50.109ms 50.168ms ± 0.034ms 50.163ms ± 0.020ms 50.182ms 50.235ms 50.272ms 50.278ms 0.23% 1.138 1.240 0.07% 0.002ms 1 200
trace_buffer/4_senders/1us_delay throughput 71602.207op/s 71758.715op/s ± 48.147op/s 71765.868op/s ± 28.237op/s 71794.343op/s 71816.419op/s 71827.431op/s 71842.860op/s 0.11% -1.134 1.229 0.07% 3.405op/s 1 200
trace_buffer/4_senders/no_delay execution_time 2.147ms 2.344ms ± 0.085ms 2.338ms ± 0.059ms 2.411ms 2.487ms 2.525ms 2.540ms 8.64% 0.069 -0.558 3.61% 0.006ms 1 200
trace_buffer/4_senders/no_delay throughput 1417441.699op/s 1537859.701op/s ± 55689.988op/s 1539849.881op/s ± 39790.330op/s 1576762.162op/s 1625384.457op/s 1675218.209op/s 1677076.185op/s 8.91% 0.090 -0.503 3.61% 3937.877op/s 1 200
trace_buffer/8_senders/10us_delay execution_time 57.883ms 58.158ms ± 0.128ms 58.132ms ± 0.073ms 58.245ms 58.410ms 58.461ms 58.480ms 0.60% 0.531 -0.260 0.22% 0.009ms 1 200
trace_buffer/8_senders/10us_delay throughput 123118.306op/s 123801.602op/s ± 271.590op/s 123856.775op/s ± 154.676op/s 123978.958op/s 124205.275op/s 124273.047op/s 124388.112op/s 0.43% -0.521 -0.269 0.22% 19.204op/s 1 200
trace_buffer/8_senders/1us_delay execution_time 50.210ms 50.298ms ± 0.062ms 50.280ms ± 0.034ms 50.332ms 50.418ms 50.500ms 50.660ms 0.76% 1.778 5.621 0.12% 0.004ms 1 200
trace_buffer/8_senders/1us_delay throughput 142123.288op/s 143148.057op/s ± 176.897op/s 143196.961op/s ± 95.661op/s 143273.831op/s 143345.406op/s 143358.356op/s 143398.109op/s 0.14% -1.762 5.508 0.12% 12.509op/s 1 200
trace_buffer/8_senders/no_delay execution_time 6.313ms 6.577ms ± 0.085ms 6.589ms ± 0.057ms 6.638ms 6.700ms 6.736ms 6.750ms 2.43% -0.508 -0.110 1.29% 0.006ms 1 200
trace_buffer/8_senders/no_delay throughput 1066743.578op/s 1094910.891op/s ± 14231.048op/s 1092683.460op/s ± 9396.050op/s 1104809.054op/s 1122147.836op/s 1128419.950op/s 1140536.182op/s 4.38% 0.572 -0.013 1.30% 1006.287op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
trace_buffer/1_senders/10us_delay execution_time [56.843ms; 56.901ms] or [-0.051%; +0.051%] None None None
trace_buffer/1_senders/10us_delay throughput [15817.101op/s; 15833.143op/s] or [-0.051%; +0.051%] None None None
trace_buffer/1_senders/1us_delay execution_time [50.075ms; 50.083ms] or [-0.009%; +0.009%] None None None
trace_buffer/1_senders/1us_delay throughput [17970.003op/s; 17973.219op/s] or [-0.009%; +0.009%] None None None
trace_buffer/1_senders/no_delay execution_time [342.906µs; 344.533µs] or [-0.237%; +0.237%] None None None
trace_buffer/1_senders/no_delay throughput [2613620.790op/s; 2624537.514op/s] or [-0.208%; +0.208%] None None None
trace_buffer/2_senders/10us_delay execution_time [57.541ms; 57.560ms] or [-0.017%; +0.017%] None None None
trace_buffer/2_senders/10us_delay throughput [31271.579op/s; 31282.270op/s] or [-0.017%; +0.017%] None None None
trace_buffer/2_senders/1us_delay execution_time [50.117ms; 50.128ms] or [-0.011%; +0.011%] None None None
trace_buffer/2_senders/1us_delay throughput [35908.410op/s; 35916.121op/s] or [-0.011%; +0.011%] None None None
trace_buffer/2_senders/no_delay execution_time [889.285µs; 894.580µs] or [-0.297%; +0.297%] None None None
trace_buffer/2_senders/no_delay throughput [2013021.901op/s; 2024997.737op/s] or [-0.297%; +0.297%] None None None
trace_buffer/4_senders/10us_delay execution_time [57.686ms; 57.706ms] or [-0.018%; +0.018%] None None None
trace_buffer/4_senders/10us_delay throughput [62385.011op/s; 62407.184op/s] or [-0.018%; +0.018%] None None None
trace_buffer/4_senders/1us_delay execution_time [50.163ms; 50.173ms] or [-0.009%; +0.009%] None None None
trace_buffer/4_senders/1us_delay throughput [71752.042op/s; 71765.387op/s] or [-0.009%; +0.009%] None None None
trace_buffer/4_senders/no_delay execution_time [2.332ms; 2.356ms] or [-0.502%; +0.502%] None None None
trace_buffer/4_senders/no_delay throughput [1530141.604op/s; 1545577.797op/s] or [-0.502%; +0.502%] None None None
trace_buffer/8_senders/10us_delay execution_time [58.140ms; 58.176ms] or [-0.030%; +0.030%] None None None
trace_buffer/8_senders/10us_delay throughput [123763.962op/s; 123839.242op/s] or [-0.030%; +0.030%] None None None
trace_buffer/8_senders/1us_delay execution_time [50.289ms; 50.306ms] or [-0.017%; +0.017%] None None None
trace_buffer/8_senders/1us_delay throughput [143123.541op/s; 143172.574op/s] or [-0.017%; +0.017%] None None None
trace_buffer/8_senders/no_delay execution_time [6.565ms; 6.589ms] or [-0.179%; +0.179%] None None None
trace_buffer/8_senders/no_delay throughput [1092938.605op/s; 1096883.178op/s] or [-0.180%; +0.180%] None None None

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 4a0e5bc 1785248590 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 15.057µs 15.522µs ± 0.163µs 15.515µs ± 0.099µs 15.611µs 15.767µs 15.971µs 16.442µs 5.98% 0.967 4.549 1.05% 0.012µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [15.500µs; 15.545µs] or [-0.145%; +0.145%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 4a0e5bc 1785248590 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 8.597ms 8.627ms ± 0.019ms 8.622ms ± 0.013ms 8.638ms 8.662ms 8.677ms 8.683ms 0.70% 0.833 0.122 0.22% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [8.624ms; 8.629ms] or [-0.030%; +0.030%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 4a0e5bc 1785248590 main
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.081µs 1.094µs ± 0.007µs 1.093µs ± 0.005µs 1.098µs 1.105µs 1.106µs 1.116µs 2.11% 0.398 -0.388 0.59% 0.000µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [1.093µs; 1.095µs] or [-0.083%; +0.083%] None None None

Base automatically changed from oscarld/fix-css-obfuscation-logic to main July 10, 2026 16:26
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot requested review from a team as code owners July 10, 2026 16:26
@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot requested review from mabdinur and removed request for a team July 10, 2026 16:26

@VianneyRuhlmann VianneyRuhlmann left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, although I think we should make sure that this is shipped in the same release as the telemetry PR

Comment thread libdd-data-pipeline-ffi/src/trace_exporter.rs
@gh-worker-ownership-write-b05516
gh-worker-ownership-write-b05516 Bot removed the request for review from a team July 21, 2026 09:55
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Jul 21, 2026
…2170)

# What does this PR do?

- Add additional metric tags to the stats aggregation key. For each configured key, the matching span meta tag is included in the key and populates `ClientGroupedStats.additional_metric_tags`.
- Limit the number of keys for `additional_metric_tags` to 4, not configurable.
- Limit the maximum length of individual additional_metric_tags values to 200 characters, not configurable. Values exceeding this are replaced with `tracer_blocked_value`.

# Motivation

- Allow Serverless Compatibility Layer to send additional metric tags on trace stats. See [RFC](https://datadoghq.atlassian.net/wiki/spaces/APM/pages/6482919540/PENDING+Span-Derived+Primary+Tags+-+V1).
- DataDog/serverless-components#140 to follow.
- https://datadoghq.atlassian.net/browse/SVLS-8787

# Additional Notes

Additional metric tags still need to be implemented in libdd-data-pipeline. Omitted from this PR since the goal of this PR is to support additional metric tags tags for Serverless without affecting other teams.

https://github.com/DataDog/libdatadog/blob/42e3296aa3fa76749caf5d96b5bbf1ff2b1913b6/libdd-data-pipeline/src/trace_exporter/stats.rs#L146


Per-field cardinality limit for additional metric tags will be implemented in a following PR, #2211.
- Limit the number of distinct aggregation entries per bucket that carry additional metric tags to 100 by default, configurable via `additional_metric_tags_max_entries`. Once exceeded, new entries have their tag values masked to `tracer_blocked_value`.

---

Implementing additional metric tags instead of span derived primary tags. See previous PR for span derived primary tags - #1815.

# How to test the change?

- Unit tests
- Deploy to Azure Functions with the following environment variables and validate that additional tags appear on trace metrics
  * `DD_TAGS: custom.primary:val,workload:azure_functions`
  * `DD_TRACE_EXPERIMENTAL_FEATURES_ENABLED: true`
  * `DD_TRACE_STATS_ADDITIONAL_TAGS: custom.primary,workload`

<img width="685" height="411" alt="Screenshot 2026-07-06 at 3 39 47 PM" src="https://github.com/user-attachments/assets/b91495b6-11aa-4051-922f-63655666c4e5" />


Co-authored-by: duncan.harvey <duncan.harvey@datadoghq.com>
@Eldolfin

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2906b45c52

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread libdd-trace-stats/src/span_concentrator/mod.rs
@Eldolfin Eldolfin changed the title feat(stats)!: per-key cardinality limits feat(stats)!: per-field cardinality limits Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants