Add scenario duplicates for `dd-trace-java` with forced v1 protocol by AlexeyKuznetsov-DD · Pull Request #7439 · DataDog/system-tests

AlexeyKuznetsov-DD · 2026-08-03T17:00:49Z

Motivation

Validate the v1 trace intake protocol in dd-trace-java before it becomes the tracer default, while retaining the existing v0.x jobs as a baseline. The coverage should exercise the complete scenario set for a selected weblog instead.

Changes

Add a generic _duplicate_jobs utility that selects weblogs by name and duplicates every generated job/shard for them, preserving each job's complete scenario list.
Configure Java prod to duplicate only spring-boot-jetty jobs, suffix their instances with _v1, and set DD_TRACE_AGENT_PROTOCOL_VERSION=1.0 on the weblog container.
Keep the original spring-boot-jetty jobs unchanged so the same coverage continues to run with v0.x.
Pass generic per-job weblog environment overrides through the reusable end-to-end workflow and apply them to the library container.
Make the duplication mechanism reusable for other tracers, weblogs, or protocol versions, including future legacy-protocol coverage.
Remove the obsolete DD_APM_ENABLE_V1_TRACE_ENDPOINT setting from APM_TRACING_EFFICIENT_PAYLOAD; current Agents expose the v1 endpoint by default.

Impact

The additional v1 coverage is limited to Java prod jobs for spring-boot-jetty. Java dev, other Java weblogs, and all other tracers keep their existing matrices unchanged. Because selection happens by weblog name, every current or future spring-boot-jetty shard is duplicated automatically.

Validation

All GitHub Actions checks passed, including the original and _v1 Java prod spring-boot-jetty jobs.
All GitLab/DDCI checks passed.
./run.sh TEST_THE_TEST tests/test_the_test/test_ci_orchestrator.py tests/test_the_test/test_docker_scenario.py::test_load_environment_overrides tests/test_the_test/test_build_pipeline.py tests/test_the_test/test_gitlab_pipeline_structure.py — 23 passed.
Focused Ruff and mypy checks passed.
The generated Java prod matrix was verified to pair every selected spring-boot-jetty job with an equivalent _v1 duplicate carrying the protocol override.

Workflow

⚠️ Create your PR as draft ⚠️
Work on you PR until the CI passes
Mark it as ready for review
- Test logic is modified? -> Get a review from RFC owner.
- Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
A docker base image is modified?
- the relevant build-XXX-image label is present
A scenario is added, removed or renamed?
- Get a review from R&P team

github-actions · 2026-08-03T17:01:34Z

CODEOWNERS have been resolved as:

.github/workflows/run-end-to-end.yml                                    @DataDog/system-tests-core
.github/workflows/system-tests.yml                                      @DataDog/system-tests-core
tests/test_the_test/test_ci_orchestrator.py                             @DataDog/system-tests-core
tests/test_the_test/test_docker_scenario.py                             @DataDog/system-tests-core
tests/test_the_test/test_gitlab_pipeline_structure.py                   @DataDog/system-tests-core
utils/_context/_scenarios/__init__.py                                   @DataDog/system-tests-core
utils/_context/_scenarios/endtoend.py                                   @DataDog/system-tests-core
utils/ci/gitlab/build_pipeline.py                                       @DataDog/system-tests-core
utils/ci/gitlab/system-tests.yml.j2                                     @DataDog/system-tests-core
utils/scripts/ci_orchestrators/workflow_data.py                         @DataDog/system-tests-core

datadog-prod-us1-3 · 2026-08-03T17:59:01Z

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 1 passed on retry

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 5c7252b | Docs | Datadog PR Page | Give us feedback!}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7381053080

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-11T21:14:53Z

+            _duplicate_jobs(
+                jobs,
+                weblog_names=("spring-boot-jetty",),
+                name_suffix="_v1",
+                weblog_env={"DD_TRACE_AGENT_PROTOCOL_VERSION": "1.0"},


Preserve duplicated jobs in GitLab rendering

For any GitLab Java prod run containing spring-boot-jetty, utils/ci/gitlab/build_pipeline.py flattens these jobs to (weblog, scenario, build_required) and discards both weblog_instance and weblog_env. The original and _v1 copies therefore produce identical top-level keys in system-tests.yml.j2 (for example, system_tests_run_java_DEFAULT_spring-boot-jetty twice), while neither job receives DD_TRACE_AGENT_PROTOCOL_VERSION; the generated pipeline may reject or collapse the duplicate definitions and cannot provide the intended v1 coverage. Preserve the suffix and environment in the GitLab renderer, or avoid generating these copies for that backend.

Useful? React with 👍 / 👎.

Fixed in 5c7252b

chatgpt-codex-connector · 2026-08-11T21:14:53Z

+    if library == "java" and ci_environment == "prod":
+        jobs.extend(
+            _duplicate_jobs(


Reserve matrix capacity before adding v1 jobs

When splitting already consumes maximum_parallel_jobs, extending the list afterward violates that limit. With the current Java end-to-end scenarios, desired_execution_time=1 and maximum_parallel_jobs=256 return 273 serialized jobs because 17 Jetty shards are duplicated after capping; .github/workflows/system-tests.yml feeds the entire list directly into one matrix, exceeding its configured capacity. Reserve slots for duplicates before splitting or reapply the cap after duplication.

Useful? React with 👍 / 👎.

Fixed in 5c7252b

cbeauchesne

I've made few RC, but the biggest one is on how the duplication is performed.

For maintainability and complexity reasons, we think that the duplication should be handled inside the CI orchestrator, but directly in workflow call.

It'll be too complex to explain in the PR, let's discuss all of that tomorrow in our meeting with @nccatoni.

cbeauchesne · 2026-08-12T10:02:48Z

        default: false
        required: false
        type: boolean
+      weblog_env:


Rename this into force_v1_traces (type boolean, default false).

This option should be populated threw different system-tests layers, and handled into WeblogInfra.configure

cbeauchesne · 2026-08-12T10:03:22Z

      SYSTEM_TESTS_FORCE_EXECUTE: ${{ inputs.force_execute }}
      SYSTEM_TESTS_DEV_MODE: ${{ inputs._system_tests_dev_mode }}
      SYSTEM_TESTS_WEBLOG:  ${{ inputs.weblog }}
+      SYSTEM_TESTS_WEBLOG_ENV: ${{ inputs.weblog_env }}


Suggested change

SYSTEM_TESTS_WEBLOG_ENV: ${{ inputs.weblog_env }}

SYSTEM_TESTS_FORCE_v1_TRACES: ${{ inputs.force_v1_traces }}

cbeauchesne · 2026-08-12T10:04:59Z

 {% endfor %}
-{% for variant, scenario, build_required in scenario_pairs %}
-system_tests_run_{{library}}_{{scenario}}_{{variant}}:
+{% for job in scenario_jobs %}


You can remove any gitlab stuff, we'll handle it.

Add DEFAULT_V1 scenario for Java and Go

57f6a99

AlexeyKuznetsov-DD self-assigned this Aug 3, 2026

Simplification, keep only tracers that supports v1, but not by default.

27473ad

AlexeyKuznetsov-DD changed the title ~~Add DEFAULT_V1 scenario for Java and Go~~ Add DEFAULT_V1 scenario for tracers that support v1 protocol, but not yet use it by default Aug 3, 2026

AlexeyKuznetsov-DD added 6 commits August 4, 2026 09:32

Merge branch 'main' into alexeyk/default-v1-protocol-tests

b2be730

Merge branch 'main' into alexeyk/default-v1-protocol-tests

0a1fae1

Merge branch 'main' into alexeyk/default-v1-protocol-tests

060fdff

Merge branch 'main' into alexeyk/default-v1-protocol-tests

5b8f96f

Merge branch 'main' into alexeyk/default-v1-protocol-tests

2893b8e

Refine Java v1 protocol test coverage

ef40c24

AlexeyKuznetsov-DD force-pushed the alexeyk/default-v1-protocol-tests branch from 3b4ecdc to ef40c24 Compare August 5, 2026 19:55

AlexeyKuznetsov-DD changed the title ~~Add DEFAULT_V1 scenario for tracers that support v1 protocol, but not yet use it by default~~ Add scenarios duplicates for dd-trace-java with forced v1 protocol. Aug 5, 2026

AlexeyKuznetsov-DD changed the title ~~Add scenarios duplicates for dd-trace-java with forced v1 protocol.~~ Add scenario duplicates for dd-trace-java with forced v1 protocol Aug 5, 2026

AlexeyKuznetsov-DD added 8 commits August 5, 2026 16:29

Refactor protocol job duplication by weblog

4e67249

Merge branch 'main' into alexeyk/default-v1-protocol-tests

ae5661d

Merge branch 'main' into alexeyk/default-v1-protocol-tests

00145cc

Merge branch 'main' into alexeyk/default-v1-protocol-tests

d2b1af3

Merge branch 'main' into alexeyk/default-v1-protocol-tests

8663bb8

Merge branch 'main' into alexeyk/default-v1-protocol-tests

5393bda

Merge branch 'main' into alexeyk/default-v1-protocol-tests

b9c90ce

Fixed issue after merge.

7381053

AlexeyKuznetsov-DD marked this pull request as ready for review August 11, 2026 21:09

AlexeyKuznetsov-DD requested a review from a team as a code owner August 11, 2026 21:09

chatgpt-codex-connector Bot reviewed Aug 11, 2026

View reviewed changes

Fixed review comments.

5c7252b

cbeauchesne requested changes Aug 12, 2026

View reviewed changes

Navigation Menu

Uh oh!

Add scenario duplicates for `dd-trace-java` with forced v1 protocol - #7439

Add scenario duplicates for `dd-trace-java` with forced v1 protocol#7439
AlexeyKuznetsov-DD wants to merge 17 commits into
mainfrom
alexeyk/default-v1-protocol-tests

AlexeyKuznetsov-DD commented Aug 3, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 3, 2026 •

edited

Loading

Uh oh!

datadog-prod-us1-3 Bot commented Aug 3, 2026 •

edited by datadog-prod-us1-5 Bot

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Aug 11, 2026

Uh oh!

AlexeyKuznetsov-DD Aug 11, 2026

Uh oh!

Uh oh!

chatgpt-codex-connector Bot Aug 11, 2026

Uh oh!

AlexeyKuznetsov-DD Aug 11, 2026

Uh oh!

cbeauchesne left a comment

Uh oh!

cbeauchesne Aug 12, 2026

Uh oh!

cbeauchesne Aug 12, 2026

Uh oh!

cbeauchesne Aug 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	SYSTEM_TESTS_WEBLOG_ENV: ${{ inputs.weblog_env }}
	SYSTEM_TESTS_FORCE_v1_TRACES: ${{ inputs.force_v1_traces }}

Uh oh!

Conversation

AlexeyKuznetsov-DD commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Impact

Validation

Workflow

Reviewer checklist

Uh oh!

github-actions Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-prod-us1-3 Bot commented Aug 3, 2026 • edited by datadog-prod-us1-5 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DD Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

AlexeyKuznetsov-DD Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

cbeauchesne left a comment

Choose a reason for hiding this comment

Uh oh!

cbeauchesne Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

cbeauchesne Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

cbeauchesne Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AlexeyKuznetsov-DD commented Aug 3, 2026 •

edited

Loading

github-actions Bot commented Aug 3, 2026 •

edited

Loading

datadog-prod-us1-3 Bot commented Aug 3, 2026 •

edited by datadog-prod-us1-5 Bot

Loading