Multi-model LLM load balancer (local + frontier + optional Nova)

[kochj23]

Summary

Fans out the shared multi-model LLM load balancer to MLX Code. When enabled, each chat is spread across all models available to the machine — all installed local models (native MLX + a local Ollama if present), all frontier models via OpenRouter, and an optional Nova Gateway — health-gated and load-balanced — instead of a single pinned MLX model.

Off by default. When every toggle is off, behavior is exactly as before (single pinned MLX model).

Nova is never a hard requirement. With zero Nova present the balancer still works over local MLX/Ollama and/or OpenRouter. The Nova Gateway is one optional entry, health-probed via GET <url>/v1/models; if it fails it drops from the pool and everything else keeps working. No hard dependency on Nova / PG / the gateway anywhere.

What changed

• Ported pure, network-free building blocks from AIStudio (verbatim logic): ModelRegistry (discovery + assemblePool toggle composition), LoadBalancer (round-robin / least-busy + health gating), OpenRouterProvider / OpenAICompatibleRequest, KeychainStore.
• New LLMBalancer service mirroring AIStudio's LLMBackendManager balancer surface (discoverEnabledPool / healthMap / chatCompletionBalanced / dispatchBalanced). Local MLX runs in-process via MLXService.chatCompletion (true token streaming); Ollama / OpenRouter / Nova ride the generic OpenAI-compatible HTTP path.
• MLX discovery adaptation: unlike AIStudio (HF hub cache), MLX Code discovers SafeTensors models via MLXService.discoverModels() over its configured model paths; ModelRegistry.mlxModels(fromLocalNames:) maps them into the pool.
• Wired into ChatViewModel: balanced dispatch is used when any toggle is on, with a clean fallback to the single pinned MLX model when the pool is empty.
• Three persisted toggles in AppSettings + a new Settings → Balancer tab (gateway URL, Keychain-stored OpenRouter key).
• Tests: network-free LoadBalancerTests (24 tests) covering parsing, pool composition, and selection policies.
• Docs: README "Multi-model load balancing" section with a Mermaid discovery→pool→balancer diagram; CHANGELOG entry.

Adaptations for MLX Code

• MLX model discovery source: MLXService.discoverModels() (MLX Code's own configured paths), not the HF hub cache.
• Gateway health probe: GET <novaGatewayURL>/v1/models.
• Sandbox: the main MLX Code app target is not sandboxed (no com.apple.security.app-sandbox in its entitlements) — nothing to disable. Only the Xcode Source Editor Extension target carries the sandbox entitlement (required by Apple for editor extensions); left untouched. Hardened Runtime / notarization intact.
• pbxproj: explicit build-file + file-ref + group + Sources-phase entries added with stable hex IDs (the main app target uses explicit refs; only the extension is a synchronized folder).

Verification

• xcodebuild build (Debug, macOS arm64, signing off): BUILD SUCCEEDED
• xcodebuild build (Release, macOS arm64, signing off): BUILD SUCCEEDED
• xcodebuild test: 546 tests, 0 failures (LoadBalancerTests: 24, 0 failures)

🤖 Generated with Claude Code