Reload cluster discovery settings from remote_servers without restart. by ianton-ru · Pull Request #2197 · Altinity/ClickHouse

ianton-ru · 2026-08-10T15:13:41Z

Changelog category (leave one):

Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Reload cluster discovery settings from remote_servers without restart.

Documentation entry for user-facing changes

Previously ClusterDiscovery read user/password/path and related fields only at startup, so XML changes had no effect until restart. Apply discovery config diffs in place on reload, including add/remove of discovery and multicluster paths.

CI/CD Options

Exclude tests:

Regression jobs to run:

Previously ClusterDiscovery read user/password/path and related fields only at startup, so XML changes had no effect until restart. Apply discovery config diffs in place on reload, including add/remove of discovery and multicluster paths. Co-authored-by: Cursor <cursoragent@cursor.com>

Serialize start()/ensureWorkerStarted() so concurrent config reload cannot double-assign the worker thread, and remove the ephemeral ZK node when a participant is reloaded as an observer. Co-authored-by: Cursor <cursoragent@cursor.com>

Invisible-only config updates now schedule an upsert, and invisible upserts clear the published cluster so visibility toggles take effect immediately. Co-authored-by: Cursor <cursoragent@cursor.com>

When a multicluster-discovered cluster already occupies a name, adding a static <path> entry previously left watches/callbacks inconsistent. Remove the dynamic entry first so static config wins cleanly. Co-authored-by: Cursor <cursoragent@cursor.com>

ianton-ru · 2026-08-10T15:14:40Z

@codex review

github-actions · 2026-08-10T15:14:55Z

Workflow [PR], commit [71dc0e1]

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 90d6e81a0b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-10T15:46:31Z

        if (finished)
            break;

+        consumePendingConfigUpdate();


Consume pending config before retrying initialization

When the initial update repeatedly fails against a Keeper path that the reload removes or corrects, the worker calls initialUpdate before reaching this pending-config consumer, so every retry continues using the obsolete configuration and the corrective reload is never applied. Process the pending update before retrying initialization so the advertised restart-free recovery works even while the old path remains unavailable.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-10T15:46:31Z

+    if (registration_changed || invisible_changed)
+        clusters_to_update->set(info.name);
+    else
+        rebuildClusterObject(info);


Refresh cached node payloads after registration changes

When a reload changes <my_hostname> or <shard> without changing the node UUID set, this only schedules upsertCluster: registerInZk updates the existing znode data, but needUpdate compares only UUIDs and the subsequent rebuild uses the old nodes_info payload. The local cluster therefore retains the old address or shard, and peers are not notified because setting child data does not trigger their children watches; invalidate and reread the payloads or otherwise notify all observers.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-08-10T15:46:31Z

+    catch (...)
+    {
+        tryLogCurrentException(log, "Error while unregistering node from cluster '" + info.name + "'");


Propagate failed discovery unregistration

If Keeper is temporarily unavailable while a participant cluster is removed or its path is replaced, swallowing this exception lets removeStaticCluster erase all local tracking and treat the reload as applied even though the old ephemeral registration may remain for the live session. Other servers can consequently keep routing queries or writes to a node that was removed from that cluster, with no later retry; fail the update and retry unregistration instead.

AGENTS.md reference: AGENTS.md:L153-L153

Useful? React with 👍 / 👎.

Registration changes only update ephemeral payload data, which does not fire children watches, and needUpdate skipped getNodes when UUIDs were unchanged. Clear nodes_info on registration change, recreate the ephemeral when payload differs so peers are notified, and always refresh payloads in upsertCluster. Co-authored-by: Cursor <cursoragent@cursor.com>

Previously, the first ClusterDiscovery constructed on config reload skipped updateFromConfig and never called start(), so enabling discovery post-startup left registration and watches inactive until restart. Start the worker when the server is already up, including the allow-flag-only reload path where remote_servers is unchanged. Co-authored-by: Cursor <cursoragent@cursor.com>

Invalid discovery XML (e.g. both password and secret) used to update clusters_config first and only then fail in updateFromConfig, leaving Clusters and discovery out of sync. Parse/validate discovery before mutating shared cluster state so SYSTEM RELOAD CONFIG fails closed. Co-authored-by: Cursor <cursoragent@cursor.com>

Unrelated config reloads were always calling notifyHostIDsUpdated after the early-return path was removed for allow-flag discovery start. Notify only when clusters changed, discovery was updated, or discovery was just created. Co-authored-by: Cursor <cursoragent@cursor.com>

An existing ClusterDiscovery is still updated when allow_experimental_cluster_discovery is disabled, so invalid discovery XML could commit Clusters and then fail in updateFromConfig. Validate whenever remote_servers changes and discovery is enabled or already constructed. Co-authored-by: Cursor <cursoragent@cursor.com>

If initialUpdate keeps failing, the worker never reached consumePendingConfigUpdate in the main loop, so a corrective remote_servers reload stayed stuck. Drain the pending update before each init attempt (and before startImpl's first init). Co-authored-by: Cursor <cursoragent@cursor.com>

Keep applying the local cluster removal when Keeper cleanup fails, queue the ephemeral path for worker retry, and cover it with a failpoint integration test instead of failing the whole reload. Co-authored-by: Cursor <cursoragent@cursor.com>

Drop queued ZK cleanup for a path on participant re-add/register, and skip retries while an active participant owns that path, so a delayed unregister cannot remove a freshly re-registered ephemeral. Co-authored-by: Cursor <cursoragent@cursor.com>

Drop redundant cancel helpers and fold unregister into one function; keep skipping retries when a participant owns the path again, and merge the failpoint coverage into a single test. Co-authored-by: Cursor <cursoragent@cursor.com>

Also stop marking Flags::cv mutable; only the mutex needs mutable for isStopped. Co-authored-by: Cursor <cursoragent@cursor.com>

ianton-ru · 2026-08-11T07:49:58Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cde1440d06

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-11T08:09:40Z

+    clusters_to_update->remove(name);
+    get_nodes_callbacks.erase(name);
+    clusters_info.erase(it);


Rescan roots after removing a shadowing static cluster

When a static observer cluster has replaced a same-named cluster discovered through a still-configured multicluster_root_path, removing that static entry erases the name here without marking the root's need_update. The generic wake from updateFromConfig is insufficient because findDynamicClusters skips roots whose flag is false, and the worker uses an untimed wait, so with no Keeper child event the dynamic cluster can remain absent indefinitely. Mark the remaining multicluster roots for rescan when removing a static shadow.

Useful? React with 👍 / 👎.

When a static cluster that shadowed a same-named dynamic entry is removed, mark remaining multicluster roots for update so findDynamicClusters rediscovers it without waiting for the force-refresh interval. Co-authored-by: Cursor <cursoragent@cursor.com>

ianton-ru · 2026-08-11T08:45:31Z

@codex review

chatgpt-codex-connector · 2026-08-11T09:14:36Z

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 0a3163a9fe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…ry_update

Erase impl when a name enters discovery and drop automatic_clusters when it returns to static or is removed, so Context no longer prefers a stale static Cluster after ownership transitions. Co-authored-by: Cursor <cursoragent@cursor.com>

Flags: :wait and multicluster/register bits were cleared before Keeper I/O, so a one-shot throw left the worker blocked until an unrelated event; restore consumed signals and re-arm wakeup for retry. Co-authored-by: Cursor <cursoragent@cursor.com>

…isabled. Reloading the allow flag from 1 to 0 left the worker registered and clusters published; tear down ClusterDiscovery with synchronous unregister so reload matches a restart with discovery off. Co-authored-by: Cursor <cursoragent@cursor.com>

Participant aliases on the same Keeper path share one ephemeral; skip unregister (and observer tryRemove) while another non-observer still owns that path so peers do not temporarily lose the retained replica. Co-authored-by: Cursor <cursoragent@cursor.com>

Watch callbacks used Flags::set, which could reinsert a removed cluster name and cause perpetual Unknown cluster scans; use setIfPresent and drop unknown keys in the worker loop. Co-authored-by: Cursor <cursoragent@cursor.com>

mkmkme

lgtm in general, left three comments

mkmkme · 2026-08-12T09:47:49Z

Not sure if this test actually makes sense to add. IIUC you changed the behaviour of repeating start() / updateFromConfig() so it doesn't abort anymore. Does this really need to be always checked in the unit test suite?

mkmkme · 2026-08-13T09:10:43Z

+
+        auto & info = it->second;
        auto zk = context->getDefaultOrAuxiliaryZooKeeper(info.zk_name);
        registerInZk(zk, info);


It seems that the first thing upsertCluster does is registerInZk so the lines 1061-1062 seem to be redundant

mkmkme · 2026-08-13T09:29:35Z

I'm also not sure about this unit test tbh. It has a any_need_update field which is not used (but I assume it's just taken from ClusterDiscovery::Flags it's mirroring).

But if we have a look at the rest of it, this test:

Has its own class to test and doesn't test any of the actual production code, so any change that would break the class it mirrors would not be caught by the test

It basically just tests the underlying std::unordered_map which we can assume to work just fine since it's part of the standard C++ library.

mkmkme · 2026-08-17T19:08:32Z

@ianton-ru could you resolve conflicts please?

…ry_update

Selfeer · 2026-08-20T11:16:29Z

The new tests (test_cluster_discovery/test_config_reload.py, test_enable_after_startup.py, test_enable_allow_only.py, and unit tests gtest_cluster_discovery_flags, gtest_cluster_discovery_start) never executed.

@ianton-ru can you please trigger a run so these are included? Once these tests are done I can verify the PR.

…ry_update

ianton-ru · 2026-08-20T14:20:59Z

Looks like need to uncheck checkboxes first :(

…ry_update

ianton-ru · 2026-08-21T06:47:23Z

@blau-ai

blau-ai · 2026-08-21T06:54:35Z

CI triage — PR #2197 (cluster discovery reload)

Verdict: CI is broadly red, but I could not attribute any failure to this PR. Every failure I looked at is pre-existing, flaky, or infra. Head SHA analyzed: 71dc0e1.

Three facts frame this:

Builds are green (all Build (...) jobs "skipping" = reused from a successful cached build) → the code compiles.
The PR's own tests are not in the failure list — neither the new unit tests (gtest_cluster_discovery_*) nor the new integration tests (test_cluster_discovery/*) appear among failures, so they passed.
Servers start and run normally everywhere: each stateless shard passed ~2000 tests and each integration shard ~1800; the failures are isolated modules or late run-level events, not a systemic startup/cluster regression. The stateless server log for the "Server died" shard has zero ClusterDiscovery lines (that code path isn't even exercised there).

Per-failure breakdown

Failure	Classification	Evidence
TSan data race (Stateless `amd_tsan, 2/2`)	Not PR — pre-existing	Race is in `QueryStatus::releaseWorkloadResources()` / `getMemoryReservation()` / `MemoryReservation` (`ProcessList.cpp`, `PipelineExecutor.cpp`, `BlockIO.cpp`). This PR touches none of those files.
"Server died" (Stateless `amd_asan_ubsan, distributed plan, 2/2` and `amd_tsan, 2/2`)	Not PR — infra/OOM	No fatal/crash stack in `clickhouse-server.err.log` (only normal test-expected exceptions); 2192 tests passed before a late server death, 0 asserted failures, 4 "broken" (couldn't run). Consistent with memory pressure under asan/tsan.
`test_s3_cluster` — all 30 tests (Integration `amd_asan_ubsan, db disk, old analyzer, 3/8`)	Not PR — infra flaky	All fail in the `started_cluster` fixture: `Timed out while waiting for instance 'c2.s0_0_1' … to start`. A single container failed to become ready; the other ~1800 tests in the same shard passed, so servers start fine in general. `test_s3_cluster` doesn't use cluster discovery.
`test_storage_iceberg_with_spark::test_schema_inference` — 12 params (Integration `4/8`)	Not PR — infra flaky	Needs an external Spark instance; environment-dependent, unrelated to cluster discovery.
`test_storage_s3_queue::test_tag_after_processing[AzureQueue]` (Integration `arm_binary, 1/4`)	Not PR — flaky	Single assert in the AzureQueue path; known-flaky Azure queue test, unrelated.
`04327_png_terminal_mode` (Stateless `amd_msan, 2/4`)	Not PR — flaky	Randomized-settings diagnosis; PNG terminal output, unrelated to clusters.
`00974_query_profiler` (Stateless `amd_tsan, s3, 2/2`)	Not PR — flaky	Timing-sensitive query-profiler test; flaky under tsan+s3, unrelated.
Stress test (`arm_debug`, `arm_tsan`)	Not PR — pre-existing	"Lost connection to server". The base branch `antalya-26.6` tip has the same two stress checks failing — pre-existing on the target branch.
*Regression aarch64/release (`cas_`, `settings`, `tiered_storage`)**	Not PR — infra	Near-identical "2 failed / 26 retried" pattern across every `cas_` module on both arches = systemic environment/setup issue, plus 1 flaky scenario in `settings` (1 of 1704). This PR adds no settings* (no `M(...)`/`DECLARE` in the diff) and doesn't touch aggregate/alter/atomic-insert/tiered-storage code.
Grype Scan (`clickhouse-server:…-alpine`)	Not PR — base image	1 high/critical CVE in the built server image, from base-OS packages — independent of this C++/tests PR.

Suggested next steps

Re-run the flaky suites (test_s3_cluster, iceberg_with_spark, s3_queue AzureQueue, 04327_png_terminal_mode, 00974_query_profiler, and the "Server died" stateless shards). No code change is needed for these.
Stress (arm_debug, arm_tsan) and Grype are pre-existing on antalya-26.6 — track/fix on the base branch; they shouldn't block this PR.
The TSan data race in QueryStatus/MemoryReservation is a genuine (but unrelated) pre-existing bug worth a separate issue if one doesn't already exist.

Health check: from a CI standpoint the PR looks healthy — it compiles, its own unit + integration tests pass, and none of the red checks trace back to the cluster-discovery-reload changes.

Note: I could not extract the c2 container's server log (the log tarball extraction is blocked in my sandbox), but the structural evidence — every other module in that shard passing — is conclusive that this is a per-container start timeout, not a startup regression. I cannot build or run ClickHouse locally, so final confirmation of the flaky suites comes from a CI re-run.

_{— @blau-ai (automated CI triage)}

Selfeer · 2026-08-21T10:03:38Z

PR #2197 CI Triage — Reload cluster discovery settings without restart (run 32414727312)

PR: Altinity/ClickHouse#2197
Head SHA: 71dc0e144d768837742ac49b6f67b73e577af634
Build version under test: 26.6.2.20000.altinityantalya (antalya-26.6)
CI run: 32414727312
Report: ci_run_report.html

Bottom line

No CI failure is caused by this PR. The PR only touches cluster-discovery code
(ClusterDiscovery.cpp/.h, Cluster.cpp, Context.cpp, a new fail point). Every failure is in
an unrelated feature/subsystem, is a sanitizer/infra artifact, or is pre-existing.
✅ The tests created by this PR ran and passed — unit gtests and all three new integration
test files, across multiple builds, with zero failures.

✅ Tests added by this PR — all ran and passed

Integration tests (`test_cluster_discovery/`)

CI detected them as changed_integration_tests and ran the whole suite. Confirmed 23 PASSED / 0
failed independently in amd_msan (4/10), amd_tsan (2/6), and arm_binary (2/4):

New test file	Passed
`test_config_reload.py`	13
`test_enable_after_startup.py`	2
`test_enable_allow_only.py`	2

(The other 6 passing cases are the pre-existing test.py, test_dynamic_clusters.py,
test_auxiliary_keeper.py, test_password.py.)

Unit tests (gtests)

Unit tests (asan_ubsan / msan / tsan) all success (fail: 0, passed: 12004 / 12000 / 12003).
The new gtests executed and passed in every build:

ClusterDiscoveryFlags.SetIfPresentDoesNotResurrectRemovedKey — from gtest_cluster_discovery_flags.cpp
ClusterDiscoveryFlags.SetIfPresentUpdatesExistingKey — from gtest_cluster_discovery_flags.cpp
ClusterDiscovery.ConcurrentStartDoesNotAbort — from gtest_cluster_discovery_start.cpp

Additional positive signal: the swarms regression suite (dynamic cluster membership, the closest
feature to cluster discovery) passed in the prior run, and no cluster-discovery test appears in any
failure tab here.

Summary of failures (none related to the PR)

Category	Count	Tests / Jobs	Verdict
Iceberg schema inference	12	`test_storage_iceberg_with_spark/test_schema_inference.py` (asan_ubsan 4/8)	Unrelated
S3Queue / Azure	1	`test_storage_s3_queue/test_0.py::test_tag_after_processing[AzureQueue]`	Unrelated / flaky
TSAN data race	1	`MemoryReservation` unique_ptr reset (tsan)	Unrelated sanitizer flaky
"Server died" (cascade)	2	stateless asan_ubsan + tsan	Cascade from sanitizer abort
Stateless flaky	2	`00974_query_profiler`, `04327_png_terminal_mode`	Unrelated / flaky
Stress "Unknown error"	2	`Stress test (arm_debug)`, `(arm_tsan)`	Infra / pre-existing
Stateless job error	1	`Stateless tests (arm_asan_ubsan, targeted)` (Traceback)	Infra
Regression: settings snapshot drift	2	`settings/.../export_merge_tree_part_schema_mismatch_mode`	Unrelated (export-part setting)
Regression: CAS / tiered storage	9	`Cas_*`, `Tiered_Storage_Cas`, `Tiered_Storage_S3Gcs`, `Tiered_Storage_Minio`	Infra (build lacks `cas` storage type)
Docker image CVEs	2	image scan (not a test)	Unrelated
Known-flaky (BROKEN)	28	S3-unreachable / debug-timeout stateless	Pre-existing

Detailed analysis

New Fails in PR (2) — unrelated

test_storage_s3_queue/test_0.py::test_tag_after_processing[AzureQueue] (arm_binary 1/4) —
S3Queue over AzureQueue; nothing to do with cluster discovery. Flaky (passes on other builds/shards).
00974_query_profiler (amd_tsan, s3 storage) — the query profiler stateless test; unrelated,
timing-sensitive under TSAN.

TSAN data race + "Server died" — unrelated sanitizer artifacts

The amd_tsan stateless job reported:

SUMMARY: ThreadSanitizer: data race ... in
std::unique_ptr<DB::MemoryReservation, ...>::reset(DB::MemoryReservation*)

The race is in the memory-reservation subsystem, not in ClusterDiscovery/Cluster/Context.
TSAN aborts the process on a race, which is why the same job also reports "Server died" (a
cascade indicator, not a separate bug). The amd_asan_ubsan "Server died" is the analogous
sanitizer cascade. None of these touch cluster-discovery code paths.

Integration `test_schema_inference` (12) & S3Queue (1) — unrelated

Iceberg-with-Spark schema inference (all azure/local/s3 param variants) failed only in the
amd_asan_ubsan, db disk, old analyzer 4/8 shard — an Iceberg/Spark fixture problem unrelated to
this PR. The other integration shards (msan, tsan, arm) were green (Failures: 0/…).

Stress `arm_debug` / `arm_tsan` "Unknown error" — infra

Reported as generic "Unknown error" at the job level; not a cluster-discovery code path. Treat as
infra/pre-existing stress harness noise.

Regression `Settings` (2) — snapshot drift, unrelated

settings/default values/export_merge_tree_part_schema_mismatch_mode fails with
SnapshotNotFoundError (server default strict, not present in the >=26.6_antalya snapshot).
This is an export-part setting, not part of this PR. Pre-existing snapshot drift.

Regression `CAS` / `Tiered_Storage` (9) — infra (build lacks `cas`)

The CAS and tiered-storage-with-CAS jobs use a cas metadata storage type that this build does not
recognize (server fails to start / cluster bring-up fails), matching the same branch-wide infra
issue seen on other antalya-26.6 PRs. Unrelated to cluster discovery.

Checks Known Fails (28) — pre-existing flaky

DeltaLake "S3 sometimes unreachable", 04070_url_base_setting debug timeouts, etc. Pre-existing.

Selfeer · 2026-08-21T11:30:12Z

@ianton-ru can you please check this last audit review comment

AI audit note: This review comment was generated by AI (gpt-5.3-codex).

Audit findings for PR #2197:

Medium: `clusters_version` is not bumped for allow-flag-only discovery transitions

Impact: StorageDistributed may keep using stale DistributedAsyncInsertDirectoryQueue cluster endpoints after allow_experimental_cluster_discovery is toggled (with unchanged remote_servers), so runtime routing can diverge from current discovery state.
Anchor: src/Interpreters/Context.cpp in Context::setClustersConfig, and src/Storages/StorageDistributed.cpp in StorageDistributed::getDirectoryQueue.
Trigger:
1. Have an existing distributed directory queue cached for a discovery-backed cluster.
2. Run SYSTEM RELOAD CONFIG changing only allow_experimental_cluster_discovery (no remote_servers diff).
3. Continue using the same queue key in StorageDistributed.
Why defect: Context::setClustersConfig now performs discovery create/disable on allow-flag-only reloads, but shared->clusters_version is incremented only inside if (!remote_servers_unchanged). StorageDistributed refreshes cached queue endpoints only when node_data.clusters_version < getClustersVersion(), so these discovery-state transitions are invisible to that refresh gate.
Fix direction (short): bump shared->clusters_version when discovery state changes (create, disable, or existing discovery config update), even if remote_servers is unchanged.
Regression test direction (short): add an integration/unit test that toggles only allow_experimental_cluster_discovery and verifies StorageDistributed refreshes/rebuilds queue endpoints (or fails closed) instead of reusing stale cached addresses.

// src/Interpreters/Context.cpp
const bool remote_servers_unchanged
    = shared->clusters && shared->clusters_config
    && isSameConfiguration(*config, *shared->clusters_config, config_name);

...
if (!remote_servers_unchanged)
{
    ...
    ++shared->clusters_version;
    clusters_changed = true;
}

// src/Storages/StorageDistributed.cpp
if (!node_data.directory_queue
    || (node_data.clusters_version < getContext()->getClustersVersion() && node_data.addresses != parseAddresses(name)))
{
    node_data.addresses = parseAddresses(name);
    node_data.clusters_version = getContext()->getClustersVersion();
    ...
}

ianton-ru and others added 4 commits August 10, 2026 14:02

Fix discovery cluster not appearing after invisible-to-visible reload.

f851336

Invisible-only config updates now schedule an upsert, and invisible upserts clear the published cluster so visibility toggles take effect immediately. Co-authored-by: Cursor <cursoragent@cursor.com>

ianton-ru changed the title ~~Feature/antalya 26.6/cluster discovery update~~ Reload cluster discovery settings from remote_servers without restart. Aug 10, 2026

chatgpt-codex-connector Bot reviewed Aug 10, 2026

View reviewed changes

ianton-ru and others added 10 commits August 10, 2026 18:57

Simplify pending discovery unregister retry path.

e7b9dc6

Drop redundant cancel helpers and fold unregister into one function; keep skipping retries when a participant owns the path again, and merge the failpoint coverage into a single test. Co-authored-by: Cursor <cursoragent@cursor.com>

Drop unused Flags constructor and duplicate discovery reload docs.

cde1440

Also stop marking Flags::cv mutable; only the mutex needs mutable for isStopped. Co-authored-by: Cursor <cursoragent@cursor.com>

chatgpt-codex-connector Bot reviewed Aug 11, 2026

View reviewed changes

ianton-ru marked this pull request as ready for review August 11, 2026 09:58

ianton-ru added antalya antalya-26.6 labels Aug 11, 2026

ianton-ru and others added 4 commits August 11, 2026 22:00

Merge branch 'antalya-26.6' into feature/antalya-26.6/cluster_discove…

270b45a

…ry_update

Merge branch 'antalya-26.6' into feature/antalya-26.6/cluster_discove…

3bd021e

…ry_update

ianton-ru and others added 3 commits August 12, 2026 19:12

Ignore late Keeper watches for removed discovery clusters.

49db0ee

Watch callbacks used Flags::set, which could reinsert a removed cluster name and cause perpetual Unknown cluster scans; use setIfPresent and drop unknown keys in the worker loop. Co-authored-by: Cursor <cursoragent@cursor.com>

blau-ai mentioned this pull request Aug 12, 2026

Antalya 26.6: Implement TRUNCATE TABLE for Iceberg Engine (REST … #2125

Merged

27 tasks

mkmkme previously approved these changes Aug 13, 2026

View reviewed changes

Merge branch 'antalya-26.6' into feature/antalya-26.6/cluster_discove…

a93c923

…ry_update

ianton-ru dismissed mkmkme’s stale review via a93c923 August 18, 2026 07:23

Fix build after merge

b6f7512

Merge branch 'antalya-26.6' into feature/antalya-26.6/cluster_discove…

7c47c34

…ry_update

Merge branch 'antalya-26.6' into feature/antalya-26.6/cluster_discove…

71dc0e1

…ry_update

Uh oh!

Conversation

ianton-ru commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

CI/CD Options

Exclude tests:

Regression jobs to run:

Uh oh!

ianton-ru commented Aug 10, 2026

Uh oh!

github-actions Bot commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 10, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 10, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Aug 10, 2026

Choose a reason for hiding this comment

Uh oh!

ianton-ru commented Aug 11, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

ianton-ru commented Aug 11, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 11, 2026

Uh oh!

mkmkme left a comment

Choose a reason for hiding this comment

Uh oh!

mkmkme Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

mkmkme Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

mkmkme Aug 13, 2026

Choose a reason for hiding this comment

Uh oh!

mkmkme commented Aug 17, 2026

Uh oh!

Selfeer commented Aug 20, 2026

Uh oh!

ianton-ru commented Aug 20, 2026

Uh oh!

ianton-ru commented Aug 21, 2026

Uh oh!

blau-ai commented Aug 21, 2026

CI triage — PR #2197 (cluster discovery reload)

Per-failure breakdown

Suggested next steps

Uh oh!

Selfeer commented Aug 21, 2026

PR #2197 CI Triage — Reload cluster discovery settings without restart (run 32414727312)

Bottom line

✅ Tests added by this PR — all ran and passed

Integration tests (test_cluster_discovery/)

Unit tests (gtests)

Summary of failures (none related to the PR)

Detailed analysis

New Fails in PR (2) — unrelated

TSAN data race + "Server died" — unrelated sanitizer artifacts

Integration test_schema_inference (12) & S3Queue (1) — unrelated

Stress arm_debug / arm_tsan "Unknown error" — infra

Regression Settings (2) — snapshot drift, unrelated

ianton-ru commented Aug 10, 2026 •

edited

Loading

github-actions Bot commented Aug 10, 2026 •

edited

Loading

Integration tests (`test_cluster_discovery/`)

Integration `test_schema_inference` (12) & S3Queue (1) — unrelated

Stress `arm_debug` / `arm_tsan` "Unknown error" — infra

Regression `Settings` (2) — snapshot drift, unrelated

Regression `CAS` / `Tiered_Storage` (9) — infra (build lacks `cas`)

Medium: `clusters_version` is not bumped for allow-flag-only discovery transitions