Skip to content

Persist skipped test visibility for solution runs - #11010

Open
Amaury Levé (Evangelink) wants to merge 1 commit into
mainfrom
evangelink-fix-skipped-test-output
Open

Persist skipped test visibility for solution runs#11010
Amaury Levé (Evangelink) wants to merge 1 commit into
mainfrom
evangelink-fix-skipped-test-output

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Summary

Repository solution and module test runs launch built test hosts directly, so the existing TestingPlatformCommandLineArguments setting did not reach them and skipped-test details remained visible.

  • persist show-test-results=failed in each test host as a passive default
  • make the terminal reporter consume passive defaults while preserving explicit command-line precedence
  • cover passive-default resolution and explicit override behavior

Validation

  • Microsoft.Testing.Platform.UnitTests: 2515 passed
  • generated test-host configuration contains show-test-results: failed
  • default execution hides per-test result blocks, while explicit --show-test-results all restores them

Fixes #11009

Store the repository's failed-only result visibility as a passive test-host default so solution and module test runs honor it. Teach the terminal reporter to consume passive defaults while preserving explicit command-line precedence.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 3, 2026 12:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The reviewed changes have no unresolved approval-blocking issues.

Review tier: Balanced
Findings: None

What changed in this PR

Persists failed-only test-result visibility for solution and module test-host runs while preserving explicit command-line overrides.

Changes:

  • Adds a passive show-test-results=failed default.
  • Applies passive defaults with explicit CLI precedence.
  • Adds coverage for default resolution and overrides.
File Description
test/​UnitTests/​Microsoft.Testing.Platform.UnitTests/​OutputDevice/​Terminal/​TerminalTestReporterCommandLineOptionsProviderTests.cs Tests passive-default resolution and explicit overrides.
test/​UnitTests/​Microsoft.Testing.Platform.UnitTests/​Helpers/​TestCommandLineOptions.cs Supports passive defaults in tests.
test/​Directory.Build.targets Persists the failed-only test-host default.
src/​Platform/​Microsoft.Testing.Platform/​OutputDevice/​TerminalOutputDevice.Initialization.cs Documents updated visibility precedence.
src/​Platform/​Microsoft.Testing.Platform/​OutputDevice/​Terminal/​TerminalTestReporterCommandLineOptionsProvider.cs Resolves explicit or passive-default visibility.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

🧵 Parallel-safety audit — PR #11010

Parallelization — assembly audited (owns the two changed test files):

Test assembly Scope Workers Analyzer coverage
Microsoft.Testing.Platform.UnitTests MethodLevel 0 (CPU count) attribute-based ([assembly: Parallelize(Scope = ExecutionScope.MethodLevel, Workers = 0)] in Program.cs) — coverable once the parallel-safety analyzers ship

Findings: A (global-state) 0 · B (paths) 0 · C (declaration) 0 · D (over-serialization) 0 — by severity: Critical 0 · High 0 · Warning 0 · Info 0.

Reviewed the PR's diff directly (the workflow's pre-extraction step outputs were unavailable in this run, so files/regions were resolved from the PR file list instead):

  • test/UnitTests/Microsoft.Testing.Platform.UnitTests/Helpers/TestCommandLineOptions.cs — adds an ICommandLineOptionsWithDefaults implementation backed by a second, instance-local Dictionary<string, string[]> _defaults supplied per test instance. No static/process-global state, no filesystem or environment mutation.
  • test/UnitTests/Microsoft.Testing.Platform.UnitTests/OutputDevice/Terminal/TerminalTestReporterCommandLineOptionsProviderTests.cs — the two new tests (GetShowTestResultsVisibility_WhenPassiveDefaultPresent_ReturnsParsedFlags, GetShowTestResultsVisibility_WhenOptionAndPassiveDefaultPresent_OptionWins) each construct their own local TestCommandLineOptions and call the static, side-effect-free TerminalTestReporterCommandLineOptionsProvider.GetShowTestResultsVisibility(...). No env vars, CWD, console, shared paths, or [ResourceLock]/[DoNotParallelize] declarations touched.
  • src/Platform/.../TerminalTestReporterCommandLineOptionsProvider.cs and TerminalOutputDevice.Initialization.cs — pure production-code renames/logic changes (TryGetOptionArgumentListTryGetOptionArgumentListOrDefault), no new statics or shared mutable state introduced.
  • test/Directory.Build.targets — adds a TestingPlatformCommandLineOptionDefault MSBuild item for show-test-results; this configures CLI defaults, not parallelization scope/workers, so it is out of scope for this audit.

Nothing in this PR mutates process-global state, touches shared filesystem paths, or changes [ResourceLock]/[DoNotParallelize]/[Parallelize] declarations. No findings to report under the detected MethodLevel scope.

Advisory only — heuristic, non-blocking. Re-run with /parallel-audit. This audit answers "is it parallel-safe?"; for testability, smells, or flakiness see the detect-static-dependencies / test-smell-detection / test-anti-patterns analyses.

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 38.3 AIC · ⌖ 7.52 AIC · ⊞ 24.8K · [◷]( · )

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

✅ 22/22 dimensions clean — no findings.

@Evangelink Amaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Sep 3, 2026
@Evangelink

Copy link
Copy Markdown
Member Author

/backport to rel/4.4

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Started backporting to rel/4.4: https://github.com/microsoft/testfx/actions/runs/33773338646

Amaury Levé (Evangelink) added a commit that referenced this pull request Sep 3, 2026
…11010 (backport to rel/4.4) (#11017)

Co-authored-by: Copilot <amauryleve@microsoft.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

state/needs-review Awaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skipped test details are shown in repository solution runs

2 participants