Skip to content

Enable parallelism in test projects - #5697

Merged
danielmarbach merged 17 commits into
masterfrom
rhys/test-parallel
Aug 7, 2026
Merged

Enable parallelism in test projects#5697
danielmarbach merged 17 commits into
masterfrom
rhys/test-parallel

Conversation

@rbev

@rbev rbev commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

This pull request introduces several improvements to the test infrastructure to introduce parallelisation within the test projects to improve CI build times.

Test Infrastructure and Parallelization Enhancements:

  • Added global NUnit settings (FixtureLifeCycle, Parallelizable, and LevelOfParallelism) to both src/Particular.LicensingComponent.UnitTests/NUnitSettings.cs and src/ServiceControl.AcceptanceTests.RavenDB/NUnitSettings.cs to enable test parallelization and control concurrency level.
  • Introduced a SetupFixture in src/ServiceControl.AcceptanceTests.RavenDB/SetupFixture.cs to ensure required event log sources are created before tests run, preventing race conditions during test startup.
  • Added a static SemaphoreSlim-based lock and UseDatabaseLifecycleLock method in AcceptanceTestStorageConfiguration to serialize database lifecycle operations, preventing parallel access issues with the shared RavenDB server.

Test Safety and Reliability:

  • Marked tests that interact with shared resources (e.g., CORS, forwarded headers, critical errors, message import failures) as [NonParallelizable] to avoid race conditions and ensure reliable test execution.

Groups with full parallelism

Group Test Projects
AzureStorageQueues ServiceControl.Transports.ASQ.Tests
Default ServiceControl.Audit.AcceptanceTests, ServiceControl.Audit.AcceptanceTests.RavenDB (linked NUnitSettings.cs from Audit.AcceptanceTests), ServiceControl.Audit.Persistence.Tests, ServiceControl.Audit.Persistence.Tests.RavenDB (linked NUnitSettings.cs from Audit.Persistence.Tests), ServiceControl.Audit.UnitTests, ServiceControl.Config.Tests, ServiceControl.Infrastructure.Tests, ServiceControlInstaller.Engine.UnitTests, ServiceControlInstaller.Packaging.UnitTests, ServiceControl.Monitoring.AcceptanceTests, ServiceControl.Monitoring.UnitTests, ServiceControl.MultiInstance.AcceptanceTests, ServiceControl.Persistence.Tests.InMemory, ServiceControl.Transports.Tests, ServiceControl.UnitTests, Particular.LicensingComponent.UnitTests
IBMMQ ServiceControl.Transports.IBMMQ.Tests
PostgreSQL ServiceControl.Transports.PostgreSql.Tests
PostgreSQLPersistence ServiceControl.Persistence.Tests.PostgreSql
PrimaryRavenAcceptance ServiceControl.AcceptanceTests.RavenDB
PrimaryRavenPersistence ServiceControl.Persistence.Tests.RavenDB
SqlServerPersistence ServiceControl.Persistence.Tests.SqlServer

Groups not updated

Group Test Projects
AzureServiceBus ServiceControl.Transports.ASBS.Tests
MSMQ ServiceControl.Transports.Msmq.Tests
RabbitMQ ServiceControl.Transports.RabbitMQClassicConventionalRouting.Tests, ServiceControl.Transports.RabbitMQClassicDirectRouting.Tests, ServiceControl.Transports.RabbitMQQuorumConventionalRouting.Tests, ServiceControl.Transports.RabbitMQQuorumDirectRouting.Tests
SQS ServiceControl.Transports.SQS.Tests
SqlServer ServiceControl.Transports.SqlServer.Tests

@rbev
rbev force-pushed the rhys/test-parallel branch from 7bac798 to f747755 Compare August 4, 2026 08:03
@rbev
rbev marked this pull request as draft August 4, 2026 08:32
@rbev
rbev force-pushed the rhys/test-parallel branch from 28b6e7d to 20f562d Compare August 5, 2026 06:08
@rbev
rbev marked this pull request as ready for review August 5, 2026 08:04
@johnsimons

Copy link
Copy Markdown
Member

Should we just use the default for LevelOfParallelism https://docs.nunit.org/articles/nunit/writing-tests/attributes/levelofparallelism.html, which mean not using the attribute?

@rbev

rbev commented Aug 6, 2026

Copy link
Copy Markdown
Contributor Author

Should we just use the default for LevelOfParallelism https://docs.nunit.org/articles/nunit/writing-tests/attributes/levelofparallelism.html, which mean not using the attribute?

I removed the explicit setting and it ran much slower, the build agents don't have 4 cores and benefit from having higher parallelism.

Comment thread src/ServiceControl.Persistence.Tests/NUnitParallelRunnerSettings.cs
Comment thread src/ServiceControl.Transports.Tests/Learning/TransportTestsConfiguration.cs Outdated
@danielmarbach

Copy link
Copy Markdown
Contributor

Curious did you already compare before and after? How much gain do we get?

@rbev

rbev commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Curious did you already compare before and after? How much gain do we get?

Here's the groupings that have changed, omitting some because they were already parallel from a previous pr, it's based on a single run so take it with a grain of salt.

Group Parallelism Before-Linux Before-Windows After-Linux After-Windows
AzureStorageQueues ✅ Enabled 4m 30s 6m 27s 4m 33s 6m 58s
Default ✅ Enabled 10m 47s 13m 40s 7m 47s 11m 17s
IBMMQ ✅ Enabled 3m 56s 3m 38s
PostgreSQL ✅ Enabled 2m 52s 7m 45s 3m 47s 7m 23s
PrimaryRavenAcceptance ✅ Enabled 10m 51s 14m 17s 6m 31s 9m 49s
PrimaryRavenPersistence ✅ Enabled 4m 0s 8m 23s 5m 7s 7m 34s

@danielmarbach danielmarbach left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@danielmarbach
danielmarbach merged commit 6cbabe2 into master Aug 7, 2026
71 of 72 checks passed
@danielmarbach
danielmarbach deleted the rhys/test-parallel branch August 7, 2026 04:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants