fix: harden xDS stream handling against reconnects and faulty nodes

[dergraf]

What

Fixes a bug where a reconnecting Envoy could crash its own ADS stream, plus nine
other defects found in the same audit. Details in CHANGELOG.md.

The reconnect bug

ExControlPlane.Stream classified discovery requests by doing arithmetic on the
version_info the node reports. That value is opaque to the node and survives
reconnects: after a control plane redeploy a node replays a version the new
instance never issued, while its per-stream counter restarts at 0. The case
had no clause for that:

case state.version do
  ^version when is_nil(error) -> ...                                  # ACK
  nil when is_nil(error) -> ...                                       # "new node"
  _ when is_nil(error) and waiting_ack != nil and waiting_ack > version -> ...
  _ when not is_nil(error) -> ...                                     # NACK
end

Envoy last applied "7"; we push v1; Envoy re-sends still reporting "7"
(normal for RDS/EDS/SDS when the subscription set changes) → state.version = 1,
version = 7, waiting_ack = 1, and 1 > 7 is false → CaseClauseError.
Since Stream.event/4 dispatches with GenServer.call, the crash exits the
caller — the gRPC stream handler — so the ADS session drops and Envoy
reconnects into the same trap.

Requests are now classified by response_nonce, as the xDS protocol defines
(the field was decoded and thrown away before): empty nonce = initial request,
new or reconnected; our latest nonce = ACK/NACK; anything else = superseded.
version_info is never parsed.

Why it matters

Envoy keeps serving its last-good config when xDS drops, so this is silent until
config actually needs to change — at which point the fleet is frozen, including
SDS certificate rotation. It is triggered by exactly the events that should
be safe: rolling redeploy, scale-down, pod restart.

Other fixes

• One slow or dead node could crash or block config distribution for every
  cluster — pushes ran serially inside the ConfigCache GenServer and wrote to
  the gRPC stream on that critical path. Now concurrent, fault-isolated, and the
  write happens off the caller's path.
• load_events/4 returned :ok when config generation raised.
• One malformed resource from the adapter killed the stream.
• One bad snapshot could prevent start-up permanently (crash → restart → re-read
  the same object).
• Snapshot's disabled-state clause was dead code and returned an invalid
  GenServer reply; force_persist/0 raised with snapshots disabled.
• Non-exhaustive error handling in ensure_registred/3 and event/4.
• The node identifier is now remembered per stream — xDS only requires it on the
  first request. Envoy sends it every time, so this is hardening for other
  spec-conformant xDS clients rather than a live bug.

Behaviour changes

• Stream.event/4 takes a map instead of a {version, error} tuple and never
  raises.
• Stream children are :temporary, not :transient.
• load_events/4 no longer blocks the ConfigCache GenServer during its sync
  wait, and returns {:error, ...} on generation failure.
• New Stream.sync_status/1 — in_sync/1 reports true when nothing is
  connected, which it cannot distinguish from "in sync". load_events/4 still
  returns :ok in that case for compatibility, but logs a warning.
• New config: :stream_push_timeout (10s), :max_concurrent_streams
  (:infinity).

How to test

mix test --exclude integration     # 99 tests
mix test --only integration        # needs an envoy binary on PATH

Every fix has a test that fails against the previous code. The reconnect bug was
reproduced first: {:case_clause, 1} at stream.ex:111, surfacing as an
{:exit, ...} out of GenServer.call — i.e. it takes the handler with it.

• Unit: 99 tests, 0 failures (verified across 5 seeds — three of these tests were
  flaky when first written and were fixed to synchronise on the now-async push)
• Integration vs. real Envoy 1.34.2: 0 failures
• mix format --check-formatted and mix compile --warnings-as-errors clean

Not addressed

Detecting that a node is gone is a transport concern and takes ~50s (measured
against Envoy 1.34.2). A node that dies mid-push leaves an out-of-sync stream for
that long, during which load_events/4 for its cluster returns
{:error, :no_sync_state_reached}. Papering over that with a grace period would
weaken the sync guarantee, so it is left as a documented limitation.

Note

chore(dev) removes export MIX_ENV=dev from the flake's shell hook — it is
Mix's default anyway and made mix test run in the dev environment where
test-only deps are unavailable.