Skip to content

DO NOT MERGE - test the converting-agent PR against all system-tests - #7339

Draft
ajgajg1134 wants to merge 8 commits into
mainfrom
andrew.glaude/TestConvertingAgent
Draft

DO NOT MERGE - test the converting-agent PR against all system-tests#7339
ajgajg1134 wants to merge 8 commits into
mainfrom
andrew.glaude/TestConvertingAgent

Conversation

@ajgajg1134

Copy link
Copy Markdown
Contributor

Motivation

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

CODEOWNERS have been resolved as:

binaries/agent-image                                                    @DataDog/system-tests-core
run-all-jobs                                                            @DataDog/system-tests-core
tests/debugger/utils/approvals/php/1.18.0/exception_replay_async_spans_expected_v1.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/approvals/php/1.18.0/exception_replay_inner_spans_expected_v1.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/approvals/php/1.18.0/exception_replay_multiframe_spans_expected_v1.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/approvals/php/1.18.0/exception_replay_recursion_3_spans_expected_v1.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/approvals/php/1.18.0/exception_replay_recursion_5_spans_expected_v1.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/approvals/php/1.18.0/exception_replay_rockpaperscissors_spans_expected_v1.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils/approvals/php/1.18.0/exception_replay_simple_spans_expected_v1.json  @DataDog/debugger @DataDog/system-tests-core
tests/debugger/test_debugger_exception_replay.py                        @DataDog/debugger @DataDog/system-tests-core
tests/debugger/utils.py                                                 @DataDog/debugger @DataDog/system-tests-core

@datadog-official

datadog-official Bot commented Jul 17, 2026

Copy link
Copy Markdown

Pipelines  Tests

Unblock PR with BitsAI

⚠️ Warnings

🚦 23 Pipeline jobs failed

Testing the test | System Tests (java, prod) / End-to-end #1 / play 1   View in Datadog   GitHub Actions

🧪 1 Test failed

tests.integrations.test_inferred_proxy.Test_AWS_API_Gateway_Inferred_Span_Creation_v2.test_api_gateway_http_inferred_span_creation_optional_tags[play] from system_tests_suite   View in Datadog
ValueError: No span validates this test

self = <tests.integrations.test_inferred_proxy.Test_AWS_API_Gateway_Inferred_Span_Creation_v2 object at 0x7f400bde2b10>

    def test_api_gateway_http_inferred_span_creation_optional_tags(self):
        assert self.r.text == "ok"
    
>       interfaces.library.validate_one_span(
            self.r,
            validator=mandatory_tags_validator_factory(
...

Testing the test | System Tests (php, dev) / End-to-end #2 / apache-mod-7.0 2   View in Datadog   GitHub Actions

🧪 1 Test failed

tests.test_telemetry.Test_Telemetry.test_app_heartbeats_delays[apache-mod-7.0] from system_tests_suite   View in Datadog
AssertionError: Heartbeat sent too fast: 1.4746582105263157s. It should be sent every 2s
assert 1.4746582105263157 > 1.5

self = <tests.test_telemetry.Test_Telemetry object at 0x7fa27509cbf0>

    @features.telemetry_heart_beat_collected
    def test_app_heartbeats_delays(self):
        """Check for telemetry heartbeat are not sent too fast/slow, regarding DD_TELEMETRY_HEARTBEAT_INTERVAL
        There are a lot of reason for individual heartbeats to be sent too slow/fast, and the subsequent ones
        to be sent too fast/slow so the RFC says that it must not drift. So we will check the average delay
...

Testing the test | System Tests (php, dev) / End-to-end #2 / apache-mod-7.0-zts 2   View in Datadog   GitHub Actions

🧪 1 Test failed

tests.test_telemetry.Test_Telemetry.test_app_heartbeats_delays[apache-mod-7.0-zts] from system_tests_suite   View in Datadog
AssertionError: Heartbeat sent too fast: 1.4745683684210527s. It should be sent every 2s
assert 1.4745683684210527 > 1.5

self = <tests.test_telemetry.Test_Telemetry object at 0x7f158a068290>

    @features.telemetry_heart_beat_collected
    def test_app_heartbeats_delays(self):
        """Check for telemetry heartbeat are not sent too fast/slow, regarding DD_TELEMETRY_HEARTBEAT_INTERVAL
        There are a lot of reason for individual heartbeats to be sent too slow/fast, and the subsequent ones
        to be sent too fast/slow so the RFC says that it must not drift. So we will check the average delay
...

View all 23 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8de6179 | Docs | Datadog PR Page | Give us feedback!

ajgajg1134 and others added 7 commits July 21, 2026 08:50
…de origin

The trace agent's v1 conversion feature made three debugger test groups fail,
all because the tests still assumed the legacy agent payload shape.

Exception replay (java): a converted span carries a `_dd.convertedv1`
provenance marker, and v0.4 `meta` values stay strings where a natively
emitted v1 payload keeps real booleans. Drop the marker and canonicalize
booleans on both sides of the approval comparison, so a single set of
approval files stays valid whether or not conversion is enabled. Normalizing
is preferred over re-recording the baselines, which would bake the current
conversion path into them.

Code origin (python): `_wait_for_code_origin_span` indexed the raw payload
with legacy-only keys (`tracerPayloads`, `resource`, `type`, `meta`), so
under v1 it never matched a span and timed out into a misleading "code origin
not enabled" assertion. Iterate `interfaces.agent.get_spans()` instead, whose
typed wrappers resolve those fields for both formats.

Exception replay (php): the `*_spans_expected_v1.json` approvals were never
generated, so the tests raised FileNotFoundError. Recorded them with
DI_OVERRIDE_APPROVALS so they land in the existing 1.18.0 folder alongside
the legacy files, rather than in a new version folder.

Validated locally per language with the converting agent and with the default
agent, confirming the payload format in each run via the captured agent
interface files:

  java/spring-boot   DEBUGGER_EXCEPTION_REPLAY       8 passed both ways
  python/flask-poc   DEBUGGER_INPRODUCT_ENABLEMENT   5 passed both ways
  php/apache-mod-8.0 DEBUGGER_EXCEPTION_REPLAY       9 passed both ways

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Re-run CI after widespread SSI matrix and PHP e2e job failures.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant