AI Agent node: per-node/workflow model configuration with optional direct-provider routing

[tbrandenburg]

Idea

The ai-studio/ai-agent node currently has a single, worker-wide model configuration: apps/execution-worker/src/engines/temporal/worker.ts builds one OpenRouter client and one model (env.AI_MODEL) at process startup, and every ai-agent node in every workflow uses it. There is no way to pick a different model per node, per workflow, or to point a node at a different provider.

This request is to make the model and provider configurable per agent node, with a sensible fallback chain down to today's worker-wide default, without expanding the required/documented env-var surface, and without any one provider (including OpenRouter) getting privileged/default status in the design.

Decisions

1. Model becomes a per-node config field, not just a worker-wide constant.

   • Add an optional model string field to the ai-agent node's schema/uischema (apps/ai-studio/src/nodes/ai-agent/{schema,uischema}.ts) and to AiAgentNodeConfig (apps/execution-worker/src/domain/ai-studio-nodes.ts).
   • Left unset by default — an empty value means "inherit," which must stay visibly distinct in the UI (placeholder text), not silently coerced to a fixed default.

2. Fallback chain, resolved per node execution, not once at boot:

   node.config.model  →  workflow.defaultAiModel (optional)  →  env.AI_MODEL
   

   Same shape for provider:

   node.config.provider  →  workflow.defaultAiProvider (optional)  →  env.AI_PROVIDER (or a documented default)
   

   This mirrors the pattern already used for apps/execution-worker's per-workflow data (e.g. globalVariables in packages/sdk/src/store/slices/diagram-slice.ts), so a workflow-level default lives in the diagram JSON via the existing "Workflow Settings" modal (packages/sdk/src/features/variables/modals/), not a new backend concept.

3. provider is a flat, explicit field — every AI SDK provider with a documented, standard API-key env var is an equal, named option, none privileged as the implicit default path.

   • Earlier drafts of this idea folded provider selection into a routeVia: 'auto' | 'openrouter' | 'direct' enum, where 'direct' bundled OpenAI and Anthropic together and OpenRouter silently absorbed "everything else." That's too opinionated: it treats OpenRouter as the default umbrella provider and OpenAI/Anthropic as a single undifferentiated escape hatch, when a user may deliberately want any of them explicitly.
   • Instead, provider is a single field listing every AI SDK provider we have direct evidence for (a documented @ai-sdk/<x> package whose factory reads one flat, conventional API-key env var with no explicit apiKey passed — confirmed against https://ai-sdk.dev/providers), plus 'auto' as just one more option (not a wrapper around the others):

     provider?:
       | 'auto'
       | 'openrouter'
       | 'openai'
       | 'anthropic'
       | 'google'        // @ai-sdk/google        — GOOGLE_GENERATIVE_AI_API_KEY
       | 'xai'            // @ai-sdk/xai            — XAI_API_KEY
       | 'mistral'        // @ai-sdk/mistral        — MISTRAL_API_KEY
       | 'cohere'         // @ai-sdk/cohere         — COHERE_API_KEY
       | 'deepseek'       // @ai-sdk/deepseek       — DEEPSEEK_API_KEY
       | 'moonshotai'     // @ai-sdk/moonshotai     — MOONSHOT_API_KEY
       | 'groq'           // @ai-sdk/groq           — GROQ_API_KEY
       | 'togetherai'     // @ai-sdk/togetherai     — TOGETHER_API_KEY
       | 'fireworks'      // @ai-sdk/fireworks      — FIREWORKS_API_KEY
       | 'perplexity'     // @ai-sdk/perplexity     — PERPLEXITY_API_KEY
       | 'cerebras'       // @ai-sdk/cerebras       — CEREBRAS_API_KEY
       | 'deepinfra'      // @ai-sdk/deepinfra      — DEEPINFRA_API_KEY
       | (string & {});

     Deliberately excluded despite being AI SDK providers: Amazon Bedrock, Google Vertex AI, and Azure OpenAI. Each authenticates via a credential chain (AWS credential chain, Google ADC, Azure resource+key pair) rather than one flat, presence-checkable API-key env var, so they don't fit the "just check Boolean(process.env.X)" pattern this design relies on — adding them is a separate follow-up with its own credential-presence logic, not a case branch like the others.
   • UI: preferred as a dropdown (Select) populated from this list, so the common case is a couple of clicks. The field also accepts free text (or an "other…" option revealing a text input) for a provider added later, or a custom OpenAI-compatible endpoint, without requiring a schema change to unlock it. This keeps the dropdown as the primary, discoverable UX while not hard-locking the field to only the providers wired today.
   • 'auto' keeps today's zero-config default behavior: infer from the model-id prefix (openai/…, deepseek/…, etc.) when a matching API key is present, otherwise fall back to OpenRouter, which can serve nearly all of the same underlying models under one key. Choosing 'auto' is an explicit, visible choice in the dropdown like any other value — not an invisible default state that happens whenever the user hasn't thought about it.

4. One uniform dispatch mechanism for every provider — all provider-specific behavior lives inside Vercel's own @ai-sdk/<x> packages, never in our code.

   • resolveModel() treats every table entry identically: look up a factory, call it, get back a LanguageModel, hand it to the same generateText() call the executor already makes. Auth, request/response translation, and streaming quirks are entirely @ai-sdk/<x>'s concern — our code never branches on provider behavior beyond "which factory to call."
   • This is a testability decision as much as a design one: with 14+ providers there is no realistic way to integration-test all of them (that would mean provisioning 14+ live API keys just for this repo's CI, and would mostly be re-verifying Vercel's SDKs rather than our own logic). Because the mechanism is uniform, proving it correct for one provider (OpenRouter, already in production use) proves it correct for all of them structurally — see Tests below.
   • @ai-sdk/openai's createOpenAI() and @ai-sdk/anthropic's createAnthropic() (and the other twelve factories) all auto-read their own conventionally-named env var when constructed with no explicit apiKey — the same convention the official provider SDKs use elsewhere.
   • We never declare, document, or require these vars in apps/execution-worker/src/env.ts — we only do a Boolean(process.env.<X>_API_KEY)-style presence check to decide whether a given provider selection is actually usable. The key value itself is never read, stored, or logged by our code; each SDK reads it directly.
   • This keeps the existing, documented env surface (OPENROUTER_API_KEY, AI_MODEL) completely unchanged, while letting deployments that already export these standard vars (for other tooling, or intentionally) unlock direct-provider routing. Everyone else still has explicit 'openrouter' available for any model OpenRouter serves.
   • Selecting a provider explicitly with its matching key absent must fail loudly at execution time (not silently fall back to another provider) — a silent fallback would defeat the purpose of an explicit choice.

Implementation plan

Node config / UI

• apps/ai-studio/src/nodes/ai-agent/schema.ts — add optional model: string and optional provider: string (validated against the known-provider list plus free text at the UI layer; kept as a plain string in the schema so a new provider value doesn't require a schema migration).
• apps/ai-studio/src/nodes/ai-agent/uischema.ts — text field for model (placeholder "Inherit workflow/deployment default"); provider as a Select populated from ['auto', 'openrouter', 'openai', 'anthropic'] plus an "Other…" option that reveals a free-text input.
• apps/ai-studio/src/nodes/ai-agent/default-properties-data.ts — leave model unset, provider defaults to 'auto'.

Workflow-level default (optional, can ship as a follow-up)

• packages/sdk/src/store/slices/diagram-slice.ts — add defaultAiModel/defaultAiProvider to DiagramState, mirroring layoutDirection/globalVariables.
• packages/sdk/src/store/slices/diagram-slice/actions.ts — load them in setDiagramModel, same as globalVariables.
• packages/sdk/src/types/integration.ts + packages/sdk/src/features/integration/utils/validate-integration-data.ts + integration-variants/*.tsx — round-trip the new fields through save/load.
• packages/sdk/src/features/variables/modals/tab-general/tab-general.tsx (or a new SETTINGS_TABS tab) — UI fields for the workflow-level defaults, same dropdown+free-text pattern as the node panel.
• No backend/DB migration needed — apps/backend's workflows table stores the whole diagram as opaque draftJson/publishedJson, so the new fields ride through unchanged.

Execution worker

• New apps/execution-worker/src/model-provider.ts. A table drives both the auto-detect prefix map and the per-provider client construction, so adding a 14th provider is one table row, not a new if/case:

  type KnownProvider =
    | 'openai' | 'anthropic' | 'google' | 'xai' | 'mistral' | 'cohere'
    | 'deepseek' | 'moonshotai' | 'groq' | 'togetherai' | 'fireworks'
    | 'perplexity' | 'cerebras' | 'deepinfra';
  
  const PROVIDER_TABLE: Record<KnownProvider, {
    envVar: string;
    prefix: string; // matches OpenRouter's own model-id namespacing for that vendor
    load: (modelId: string) => Promise<LanguageModel>;
  }> = {
    openai:     { envVar: 'OPENAI_API_KEY',                 prefix: 'openai/',     load: async id => (await import('@ai-sdk/openai')).createOpenAI()(id) },
    anthropic:  { envVar: 'ANTHROPIC_API_KEY',               prefix: 'anthropic/',  load: async id => (await import('@ai-sdk/anthropic')).createAnthropic()(id) },
    google:     { envVar: 'GOOGLE_GENERATIVE_AI_API_KEY',    prefix: 'google/',     load: async id => (await import('@ai-sdk/google')).createGoogleGenerativeAI()(id) },
    xai:        { envVar: 'XAI_API_KEY',                     prefix: 'xai/',        load: async id => (await import('@ai-sdk/xai')).createXai()(id) },
    mistral:    { envVar: 'MISTRAL_API_KEY',                 prefix: 'mistral/',    load: async id => (await import('@ai-sdk/mistral')).createMistral()(id) },
    cohere:     { envVar: 'COHERE_API_KEY',                  prefix: 'cohere/',     load: async id => (await import('@ai-sdk/cohere')).createCohere()(id) },
    deepseek:   { envVar: 'DEEPSEEK_API_KEY',                prefix: 'deepseek/',   load: async id => (await import('@ai-sdk/deepseek')).createDeepSeek()(id) },
    moonshotai: { envVar: 'MOONSHOT_API_KEY',                prefix: 'moonshotai/', load: async id => (await import('@ai-sdk/moonshotai')).createMoonshot()(id) },
    groq:       { envVar: 'GROQ_API_KEY',                    prefix: 'groq/',       load: async id => (await import('@ai-sdk/groq')).createGroq()(id) },
    togetherai: { envVar: 'TOGETHER_API_KEY',                prefix: 'togetherai/', load: async id => (await import('@ai-sdk/togetherai')).createTogetherAI()(id) },
    fireworks:  { envVar: 'FIREWORKS_API_KEY',               prefix: 'fireworks/',  load: async id => (await import('@ai-sdk/fireworks')).createFireworks()(id) },
    perplexity: { envVar: 'PERPLEXITY_API_KEY',              prefix: 'perplexity/', load: async id => (await import('@ai-sdk/perplexity')).createPerplexity()(id) },
    cerebras:   { envVar: 'CEREBRAS_API_KEY',                prefix: 'cerebras/',   load: async id => (await import('@ai-sdk/cerebras')).createCerebras()(id) },
    deepinfra:  { envVar: 'DEEPINFRA_API_KEY',               prefix: 'deepinfra/',  load: async id => (await import('@ai-sdk/deepinfra')).createDeepInfra()(id) },
  };
  
  function inferAutoProvider(modelId: string): KnownProvider | 'openrouter' {
    for (const [id, entry] of Object.entries(PROVIDER_TABLE) as [KnownProvider, typeof PROVIDER_TABLE[KnownProvider]][]) {
      if (modelId.startsWith(entry.prefix) && process.env[entry.envVar]) return id;
    }
    return 'openrouter'; // serves nearly all the same models under one key
  }
  
  function stripPrefix(modelId: string, prefix: string): string {
    return modelId.startsWith(prefix) ? modelId.slice(prefix.length) : modelId;
  }
  
  export async function resolveModel(
    modelId: string,
    provider: string, // 'auto' | 'openrouter' | KnownProvider | free-text for a future/custom provider
    openrouter: OpenRouterClient,
  ) {
    const resolved = provider === 'auto' ? inferAutoProvider(modelId) : provider;
  
    if (resolved === 'openrouter') return openrouter.chat(modelId);
  
    const entry = PROVIDER_TABLE[resolved as KnownProvider];
    if (!entry) {
      throw new Error(`Unknown provider "${provider}" — no client registered for it yet`);
    }
    if (!process.env[entry.envVar]) {
      throw new Error(`provider: '${resolved}' requires ${entry.envVar} to be set`);
    }
    return entry.load(stripPrefix(modelId, entry.prefix));
  }

  Every provider package is dynamically imported only on the branch that needs it, so a deployment using only OpenRouter never pulls in the other thirteen. A free-text provider value that isn't a table key fails clearly rather than guessing — adding real support for it means adding a table row plus its dependency, not something the field silently pretends to handle today.
• apps/execution-worker/src/domain/ai-studio-nodes.ts — AiAgentNodeConfig gains model?: string and provider?: string.
• apps/execution-worker/src/activities/ai-agent.ts — AiAgentDeps.model (a fixed LanguageModel) becomes a resolver call: const model = await resolveModel(node.config.model ?? workflowDefault.model ?? env.AI_MODEL, node.config.provider ?? workflowDefault.provider ?? 'auto', openrouter).
• apps/execution-worker/src/engines/temporal/worker.ts — keep building the single openrouter client exactly as today; pass it (plus the resolver) into executeAiAgent instead of a pre-resolved model.
• apps/execution-worker/src/env.ts — no changes. None of the fourteen providers' key vars are declared here.
• apps/execution-worker/package.json — add @ai-sdk/openai, @ai-sdk/anthropic, @ai-sdk/google, @ai-sdk/xai, @ai-sdk/mistral, @ai-sdk/cohere, @ai-sdk/deepseek, @ai-sdk/moonshotai, @ai-sdk/groq, @ai-sdk/togetherai, @ai-sdk/fireworks, @ai-sdk/perplexity, @ai-sdk/cerebras, @ai-sdk/deepinfra (all small, official packages), each imported dynamically so there's no runtime cost for the providers a deployment doesn't use.
• apps/execution-worker/README.md — one line per provider documenting that its standard env var is recognized-if-present, not required, and is read directly by that provider's own AI SDK package; and that provider accepts free text for values not yet in the table (which currently fails at execution with a clear error until support is added).

Tests

• A real integration test can only ever exercise the provider(s) we have live credentials for in CI (today, OpenRouter). We do not attempt to integration-test all fourteen @ai-sdk/* packages — that would mean provisioning fourteen sets of API keys just for this repo's CI, and it would mostly be re-testing Vercel's own SDKs, not our code. Coverage instead comes from design, not enumeration:
  • The abstraction puts every provider-specific detail (auth, request/response shape, streaming) entirely inside @ai-sdk/<x>'s own create<X>() factory — PROVIDER_TABLE's load entries are one-line calls into that factory, nothing else. Once OpenRouter's entry (already in production use) round-trips correctly through resolveModel() → generateText(), every other table entry runs through the exact same call shape (resolveModel → provider factory → LanguageModel → generateText) with only the factory swapped. If our dispatch mechanism is correct for one entry, it's structurally correct for all of them — the risk surface is the shared dispatch code, not each individual provider's SDK internals, which are Vercel's tested surface, not ours.
  • Unit tests therefore target the mechanism, using one or two fake/mocked table entries rather than all fourteen real ones: 'auto' resolution (prefix match + key present/absent, falling through to 'openrouter'), explicit provider selection (found in the table vs. throwing when the env var is missing), an unknown free-text provider (throws clearly), and an unprefixed model id under 'auto'. These same cases, run against one representative entry, validate the dispatch logic that every real provider shares.
  • A lightweight non-mocked smoke check (in CI, gated on whichever key(s) are actually available — currently just OPENROUTER_API_KEY) confirms the end-to-end path still works for at least one real provider; adding a 15th provider does not require adding a 15th live smoke test.
• Unit test for the node → workflow → env fallback chain in the activity, for both model and provider independently — again mechanism-level, independent of which specific provider is selected.

Non-goals (explicitly out of scope for this request)

• Credential-chain-based providers (Amazon Bedrock, Google Vertex AI, Azure OpenAI) — no flat, single API-key env var to presence-check against; would need dedicated credential-resolution logic as a separate follow-up.
• A general provider registry/plugin system for arbitrary providers beyond the fourteen listed plus OpenRouter — the free-text field is intentionally forward-compatible in shape, not in behavior; wiring a new provider is still a follow-up code change (one table row + one dependency).
• Per-node credentials or a credential vault — API keys remain deployment-wide, server-side only.
• Any change to how tools (e.g. the existing web-search tool) are wired — orthogonal to model/provider selection.