feat: Add the FDv2 data system orchestrator by beekld · Pull Request #190 · launchdarkly/rust-server-sdk

beekld · 2026-08-10T17:15:49Z

Summary

Adds the FDv2 orchestrator, the DataSystem implementation that owns the
in-memory store and keeps it populated. It runs an initializer phase to obtain
a basis, then a synchronizer phase for ongoing changes.

The initializer phase tries each initializer in order until one produces a
basis. The synchronizer phase then runs one synchronizer at a time and rotates
between them on three triggers:

A fallback timer advances to the next synchronizer after a sustained
interruption.
A recovery timer returns to the primary synchronizer after the recovery
interval.
A terminal error blocks a synchronizer and advances past it.

Initialization is signaled as complete on the first basis and as failed once
every source is exhausted without one, because the client's initialization path
already resolves to success or failure. Synchronizer status transitions are
logged, with interruptions at info (not repeated while ongoing) and terminal
errors at warn.

Note

Overview
Introduces FDv2DataSystem, the FDv2 DataSystem implementation that owns an in-memory store and keeps it updated via a two-phase async run loop spawned from start.

Initializer phase walks factory-built initializers until one returns a basis (full changeset with a selector), applying each non-None changeset and signaling init_complete(true) once any full payload lands (after the initializer chain when needed). Synchronizer phase drives one synchronizer at a time through SourceManager: cyclic rotation, blocking on terminal errors, fallback after sustained Interrupted, and recovery back to the prime when a backup source was active. Selectors are passed into next and updated per changeset rules (None kind preserves selector; selectorless partial clears it).

Wires the module in fdv2/mod.rs and drops dead_code allowances on ChangeSet::selector and TransactionalDataStore::apply now that the orchestrator consumes them. Adds a large #[cfg(test)] suite covering init/failure, rotation, timers, and selector edge cases.

^{Reviewed by Cursor Bugbot for commit f8ccaf3. Bugbot is set up for automated code reviews on this repo. Configure here.}

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit f3e5a99. Configure here.}

keelerm84

A few questions and suggestions on the orchestrator, mostly around the initializer chain and test coverage.

keelerm84 · 2026-08-17T16:16:38Z

+            _ = shutdown => return,
+            event = initializer.run().fuse() => {
+                if let FDv2SourceResult::ChangeSet(change_set) = event.result {
+                    if !matches!(change_set.kind, ChangeSetKind::None) {


Initializers can arrive with a full change set but no basis (e.g. file based initialization). So applying a change set and determining if the init chain is done are two different things.

Ah, yeah, I guess I was assuming any initializer that finished would set a basis, but that's not a good assumption. Fixed. And test added.

Okay, so I've made it so that init_complete is called as soon as there's a full payload, which is consistent with Java and C++ (and FDv1). I don't think calling init_complete should require a selector, nor does it necessarily have to happen from an initializer. We may not get a full payload until we get to a synchronizer, if the initializers fail. And if we're on FDv1 fallback, we may get usable data without a selector.

And I've fixed it so that we don't exit the initializer loop until we get a selector. I think we're all on the same page about that.

If that doesn't sound right to you, then we should probably have a quick chat about it.

(or at least, that will be the case after i finish pushing)

Okay, pushed.

keelerm84 · 2026-08-17T16:16:38Z

+        }
+    }
+
+    struct MockInitializer {


Might be worth implementing the test data source for FDv2 so future tests can rely on that if needed.

I'm not convinced that would actually make these tests any simpler, because we're mocking out the initializers and synchronizers to make them behave in non-standard ways to exercise the orchestrator paths.

But I do have the test data source in my road map, and I intend to do it in a later PR.

keelerm84 · 2026-08-17T16:16:38Z

+    }
+
+    #[tokio::test]
+    async fn initializer_basis_signals_once_and_propagates_selector() {


Based on my earlier comment, you will also want a test that gets a payload from the first source, doesn't have a basis, and then continues onto the second one.

Test added.

keelerm84 · 2026-08-20T16:50:35Z

+                        if !initialized {
+                            init_complete(true);
+                            initialized = true;
+                        }


We should only mark this as initialized if we received the basis.

When you say "basis" here, do you mean (a) a full payload, or (b) a full payload with a selector?

As mentioned above, based on my interpretation of the spec, and the C++ and Java implementations, I think we want to mark as initialized as soon as we get a full payload, regardless of selector, because that's when the data store becomes usable. But we don't advance to the synchronizers until we get the selector.

keelerm84 · 2026-08-20T16:51:50Z

                    FDv2SourceResult::ChangeSet(change_set) => {
                        if !matches!(change_set.kind, ChangeSetKind::None) {
-                            selector = change_set.selector.clone();
+                            if change_set.selector.is_some() {


Same deal here.

Agreed that this should be consistent with the other spot.

keelerm84 · 2026-08-20T16:53:44Z

+        .await;
+
+        // The request after the selectorless change still carries "s1".
+        let seen = selectors_seen.lock().unwrap();


I don't think this is right.

If we maintain a basis even after a subsequent change, then we can't safely pick back up where we left off because we have a different state than the basis suggests to FD.

You're right, and this is an existing bug in C++ as well. Fixed, and added a test.

…they are consumed

…ta changes

keelerm84 · 2026-08-26T12:39:40Z

+
+/// Produces a fresh initializer each time the orchestrator starts a run.
+pub(crate) trait InitializerFactory: Send + Sync {
+    fn create(&self) -> Box<dyn Initializer>;


Didn't notice this the first round through, but I think the convention within this repository tends to be more "Builder / build" than "Factory / create". Do you think we should align so it's more consistent for the customer? (Not going to block over this of course)

Do you think we should align so it's more consistent for the customer?

This is not a public API. It's an internal factory for creating initializer instances. The public config API comes in a later PR and does use the "Builder / build" terminology to build the config.

keelerm84 · 2026-08-26T12:50:37Z

+            event = initializer.run().fuse() => {
+                match event.result {
+                    FDv2SourceResult::ChangeSet(change_set) => {
+                        let is_full = matches!(change_set.kind, ChangeSetKind::Full);


I think there is still a difference between this and the Java implementation. The java implementation does not require a full change set. If ANY change set is received, it considers that initialized. So a partial change set with a basis would behave differently between the two. I think the Java one will also trigger if it receives an "up-to-date" without any payload data as well.

I'm happy to make this work like Java, but I'm a little confused.

Is it even possible to get a partial result from a request that didn't already have a basis?

Is it possible to get a partial result before any full result?

I was assuming that partial results are always diffs from an existing basis, but these cases where Rust and Java diverge are only cases where there's a partial result before any basis.

beekld force-pushed the bklimt/SDK-2753/fdv2-orchestrator branch from 4ea2eb8 to 775e611 Compare August 10, 2026 18:41

beekld force-pushed the bklimt/SDK-2753/fdv2-orchestrator branch from 775e611 to 2fa17de Compare August 10, 2026 18:46

beekld force-pushed the bklimt/SDK-2753/fdv2-orchestrator branch from 2fa17de to a523fa8 Compare August 12, 2026 21:45

beekld force-pushed the bklimt/SDK-2753/fdv2-orchestrator branch from a523fa8 to 3e77e84 Compare August 12, 2026 22:10

Base automatically changed from bklimt/SDK-2742/fdv2-streaming to main August 13, 2026 21:29

beekld force-pushed the bklimt/SDK-2753/fdv2-orchestrator branch from 3e77e84 to f3e5a99 Compare August 13, 2026 21:29

beekld marked this pull request as ready for review August 13, 2026 21:35

beekld requested a review from a team as a code owner August 13, 2026 21:35

cursor Bot reviewed Aug 13, 2026

View reviewed changes

Comment thread launchdarkly-server-sdk/src/fdv2/data_system.rs

beekld force-pushed the bklimt/SDK-2753/fdv2-orchestrator branch from f3e5a99 to 58c2a46 Compare August 14, 2026 20:53

keelerm84 reviewed Aug 17, 2026

View reviewed changes

beekld requested a review from keelerm84 August 20, 2026 01:34

keelerm84 requested changes Aug 20, 2026

View reviewed changes

beekld requested a review from keelerm84 August 24, 2026 18:07

beekld added 12 commits August 25, 2026 15:52

feat: Add FDv2 data system orchestrator

1455dc1

feat: Add FDv2 synchronizer rotation and fallback/recovery timers

7e7ef4c

feat: Log FDv2 synchronizer status transitions

c822ce2

refactor: Rename RecoveringPrimeFactory to DownThenDataFactory

f1c940e

feat: Make FDv2 fallback and recovery timeouts configurable

49c1f3c

chore: Remove the FDv2 transactional-store dead-code allows now that …

de3aef7

…they are consumed

chore: Include the source name in FDv2 orchestrator log messages

c30482c

fix: Ignore no-change changesets in the FDv2 orchestrator

1909d4b

fix: Treat an FDv2 change set as a basis only when it has a selector

e302830

refactor: Simplify the FDv2 is_prime check

bfd5b91

refactor: Build FDv2 initializers lazily as the phase iterates

abf0463

fix: Signal FDv2 init on a full payload and update the selector on da…

9f1f16c

…ta changes

fix: Signal FDv2 init on a basis and defer a selector-less full

f8ccaf3

beekld force-pushed the bklimt/SDK-2753/fdv2-orchestrator branch from 6165759 to f8ccaf3 Compare August 25, 2026 22:54

keelerm84 reviewed Aug 26, 2026

View reviewed changes

Uh oh!

Conversation

beekld commented Aug 10, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

keelerm84 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

beekld commented Aug 10, 2026 •

edited by cursor Bot

Loading