[APPS-2792] Add: dev:verify mode-aware routing for local execution (build-plugins half) by tyffical · Pull Request #490 · DataDog/build-plugins

tyffical · 2026-08-21T16:34:06Z

Motivation

Part of APPS-2792 — local Node execution for App Builder backend functions, the npm run dev:verify CLI milestone described in the Kickoff doc.
Gap this PR closes:
- npm run dev:verify doesn't exist yet — pre-publish parity checking against the real cloud round trip currently requires manually curling /__dd/executeActionViaCloud.
- This PR makes the dev server mode-aware so dev:verify (added to the scaffold template in a separate web-ui PR) can route through the same /__dd/executeAction URL the frontend already calls, without the frontend needing to know which mode it's in.
Rejected alternative: making the client-side transport (dev-server-transport.ts) mode-aware via import.meta.env.MODE.
- Rejected because this repo's Jest setup (ts-jest) has no CommonJS equivalent for import.meta — introducing it breaks the transform for any file that imports it.
- Routing server-side, keyed off Vite's own resolved --mode, avoids this and keeps the client transport unchanged.

Architecture

npm run dev              (--mode development, default)     npm run dev:verify        (--mode dev-verify)
         │                                                            │
         └──────────────────────┬─────────────────────────────────────┘
                                 ▼
                  Browser: executeBackendFunction()
                  → devServerTransport → POST /__dd/executeAction
                  (unchanged either way — the client never knows the mode)
                                 │
                                 ▼
                  createDevServerMiddleware's /__dd/executeAction branch
                  checks `mode` (threaded from `server.config.mode`,
                  Vite's own resolved --mode, read in configureServer)
                                 │
                  ┌──────────────┴───────────────┐
                  ▼ mode !== 'dev-verify'          ▼ mode === 'dev-verify'
          handleExecuteAction                handleExecuteActionViaCloud
          (local, in-process, no bundling)    (bundle + real preview-async
                                                round trip — unchanged)

/__dd/executeActionViaCloud remains directly reachable — this only adds a second way to reach the same cloud behavior, gated by mode, at the URL the client already calls by default.

Changes

8 changes across 6 files

What changed	File
New `DEV_VERIFY_MODE` constant (`'dev-verify'`), the Vite `--mode` value `dev:verify` will use.	constants.ts
`createDevServerMiddleware` takes a new required `mode: string` parameter.	dev-server.ts
Extracted `routeToCloudHandler` so both cloud-bound routes share one auth/error path.	dev-server.ts
When `req.url === '/__dd/executeAction'` and `mode === DEV_VERIFY_MODE`, delegates to the same cloud-execution logic `/__dd/executeActionViaCloud` uses (including the existing "auth not configured" 400 guard) instead of running locally.	dev-server.ts
`configureServer(server)` passes `server.config.mode` (Vite's own resolved mode, read via the plugin API — no `import.meta.env` involved) through to `createDevServerMiddleware`.	vite/index.ts
New test: `mode: DEV_VERIFY_MODE` routes `/__dd/executeAction` through the real `preview-async` round trip (via `nock`) and never calls `loadModule`.	dev-server.test.ts
New `configureServer`-level test: confirms the cloud path is taken through the real plugin wiring, not just the middleware in isolation, when `config.mode` is `DEV_VERIFY_MODE`.	index.test.ts
`mode` is now a required `createDevServerMiddleware` parameter; existing call sites updated to pass an explicit mode.	dev-server.integration.test.ts

QA Instructions

yarn install

yarn test:unit packages/plugins/apps/src/vite/dev-server.test.ts
# Expected: Test Suites: 1 passed / Tests: 49 passed ✅ VERIFIED

yarn build:all && yarn test:unit
# Expected: Test Suites: 91 passed / Tests: 2241 passed, 1 skipped ✅ VERIFIED

yarn workspace @dd/apps-plugin run typecheck
# Expected: no output, clean exit ✅ VERIFIED

npx eslint 'packages/plugins/apps/**/*.ts' --quiet
# Expected: no output, clean exit ✅ VERIFIED

Manual QA against a real scaffolded app (invoking vite dev --mode dev-verify directly, ahead of the web-ui half's dev:verify script landing), per the Testing and QA Guide:

npx vite dev --mode dev-verify --port 5185 --strictPort
# VITE v7.3.6  dev-verify  ready in 185 ms ✅ VERIFIED (mode banner confirms it's active)
curl -X POST http://localhost:5185/__dd/executeAction -d '{"functionName":"...example","args":[99]}'
# {"success":true,"result":{"data":{"doubled":198,"tripled":297}}} ✅ VERIFIED

Confirmed it's genuinely routing to the cloud, not local in-process execution:

The customer function's own console.log did not print in the local terminal (unlike the same call under default npm run dev, where it does).
The server log shows the real round trip:
- Bundling backend function...
- Executing action via cloud...
- Calling Datadog API: https://api.datad0g.com/api/v2/app-builder/queries/preview-async
- Query execution started with receipt: ...
- Long-poll attempt 1/10...
- Long-poll response, done: true

Blast Radius

No behavior change for npm run dev (default mode) — mode defaults to undefined/'development', which never equals DEV_VERIFY_MODE, so the existing local-execution branch is unchanged.
/__dd/executeActionViaCloud is untouched.
Risk: low.
- Purely additive branch in the middleware.
- No new dependency, no client-side change.
- No new config surface exposed to customers — the mode comes from Vite's own --mode flag, which the web-ui template's dev:verify script will set.

Out of Scope / Follow-ups

3 items deferred

Item	Status	Next step
web-ui: add the `dev:verify` script (`vite dev --mode dev-verify`) and a minimal example backend function to the create-apps template	In progress	ddoghq/web-ui#4501
Update onboarding docs to reference the real `dev:verify` command	Not started	After both halves land
Nudge/enforce `dev:verify` in the publish flow before `datadog-apps publish`	Not started	Separate follow-up, likely in the `datadog-apps` CLI

Documentation

RFC: Confluence page 7012712651
Kickoff doc: Confluence page 7025394455 — npm run dev:verify CLI
QA guide: Confluence page 7100498075 — exact local/staging QA commands for this repo

Copilot

Pull request overview

Friend, this PR adds mode-aware cloud verification routing to the Vite development server.

Changes:

Adds the dev-verify mode constant.
Routes the standard execution endpoint through cloud execution in verification mode.
Adds coverage for the new routing behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
`constants.ts`	Defines the verification mode.
`dev-server.ts`	Implements mode-aware routing.
`dev-server.test.ts`	Tests cloud routing in verification mode.
`index.ts`	Passes Vite’s resolved mode to the middleware.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

datadog-prod-us1-4 · 2026-08-25T18:43:32Z

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: dbea903 | Docs | View more details | Give us feedback!}

tyffical · 2026-09-01T18:10:45Z

@codex review

chatgpt-codex-connector · 2026-09-01T18:11:10Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-01T18:14:23.382067Z	`3b24011`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3b2401177f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

tyffical · 2026-09-01T18:15:25Z

@copilot review

Copilot

Pull request overview

Copilot reviewed 33 out of 35 changed files in this pull request and generated 3 comments.

/__dd/executeAction now routes to executeScriptLocally directly (no bundling on this path at all) via a new loadModule parameter threaded from vite/index.ts's server.ssrLoadModule, giving the local path the same TS-transform/resolve rules and HMR-aware module cache a real request gets. /__dd/debugBundle and the cloud round trip (/__dd/executeActionViaCloud) are unchanged and still bundle. Auth is required upfront for the whole executeAction endpoint, matching production's auth-before-execution ordering, instead of checking lazily inside a $.Actions call. $.Actions calls thread connectionId end-to-end through ExecuteAction -> makeExecuteActionRemotely -> the single-action preview-async query spec. A second, independent absolute ceiling bounds one execution's total wall-clock time regardless of in-flight $.Actions calls. getAllowedConnectionIds's module-graph collector fails closed instead of falling back to a static import's raw specifier text when Vite's resolveId can't resolve it, and scopes suffixed-subgraph tracking (used to propagate the local-execution marker through nested backend imports) to one execution via AsyncLocalStorage instead of a dev-server-lifetime Set. Config() hook adds ssr.noExternal for @datadog/apps-backend and @datadog/action-catalog, which ship ESM-only and would otherwise crash under Vite's default node_modules externalization. Concurrency: executeColdActionLocally wraps priming, connection-ID collection, and execution in a single enqueue() call, since priming a cold function's module (which runs its top-level code) previously happened outside the execution queue's lock, allowing two concurrent cold-function requests to interleave their top-level evaluation. bundle()'s external option is now a matcher function instead of a plain string array, so a dependency's subpath imports (e.g. rollup/parseAst) are externalized the same as its bare specifier -- affects every published package's build. Extracted handleHttpError/sendSuccess/ guardAuthenticated to dedupe repeated response/auth-gating logic across the three endpoint handlers.

…ng-polling budget The absolute action-call and total-execution ceilings were fixed constants (10min/6min) shorter than the worst-case long-poll retry budget the caller's own longPolling config can produce (up to 400s by default), so a legitimate slow retry sequence could be killed by the ceiling before it finished. Both ceilings now derive from longPolling.maxRetries * longPolling.timeoutMs.

…t after A queried id (e.g. ./helper.ts?raw) and its plain counterpart (./helper.ts) normalize to the same moduleId for the visited-set dedup. If the plain form was visited first, the dedup check silently skipped the query'd form's rejection before it ever reached the semantic-query check below it, instead of failing closed as intended.

…l execution The dev server has no build-time moduleParsed hook (that's Rollup-only), so nothing re-ran the production bundle's static checks (banned Node built-ins, restricted globals) against a backend function's transitively imported helper modules during local execution. A helper containing a top-level `fs.readFileSync` or similar would run fine locally and only get rejected once the real build checked it at publish time. collectModuleGraphFromServer now runs runBackendStaticChecks against every app-local module record it collects, reusing the AST and scope analysis already computed for connection-ID extraction, matching what createBackendStaticChecksPlugin's moduleParsed hook already does for the production bundling path.

…e split Both doc comments restated the function name/signature instead of explaining why parseAndLookupFunction exists as its own function (so the no-bundling local-execution path can reuse it without a bundle).

Several doc comments and inline explanations had grown to 10-21 lines (or, for single-line comments, 200-600+ characters) by stacking every contributing justification instead of keeping the one that matters. Trimmed each to its tightest form; no logic changed.

Runs the customer function's own call (not loadModule or the action-catalog/ apps-backend registrations, which need real network/fs access) with global net/fetch/child_process access blocked, hardened against every bypass a review pass found: a malicious result's toJSON()/getter still running under the block, and $.Actions/action-catalog calls made from inside the function exempted via runAllowed so the guard doesn't also block trusted API calls.

…ppers Node's native exec/execFile carry a util.promisify.custom implementation resolving {stdout, stderr}; wrapping them into a plain function silently drops that symbol, so promisify() falls back to its generic single-value behavior instead. This broke @dd/tools' own execute() helper — used by, among other things, the injection plugin's test suite to run built bundles and assert on their captured stdout — whenever it ran in the same Jest worker as a test that imports network-guard.ts, since the property patch is process-wide. Bisected via git bisect against a CI failure entirely outside this package (packages/plugins/injection), confirmed by reproducing the exact symptom in isolation before landing the fix. Reimplements the {stdout, stderr} contract through the guarded callback path rather than reusing Node's original custom implementation, which would call straight into the native binding and bypass the block guard. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

Neither goes through net.Socket or fetch, so both fully escaped the existing guard — a backend function could exfiltrate data over UDP or open a raw WebSocket undetected. Also drops a comment's dangling cross-reference to env-guard.ts, which doesn't exist on this branch.

…f, sync ESM child_process bindings installGuardedProperty's setter treated any incoming value as a new real implementation, including a previously-read guard object handed back by the common "capture original, mock, restore" pattern -- corrupting real into a guard that calls itself forever on the next unblocked invocation. Now tracks which real value was active when each guard was built, and restores that value when a tracked guard is written back instead of the guard itself. Also calls syncBuiltinESMExports() after installing the child_process guards, since Node keeps ESM named bindings (e.g. `import { spawn } from 'node:child_process'`) as separate references that stay bound to the original native functions otherwise -- verified with a standalone `node --input-type=module` script since Jest's CJS transform can't reproduce the real ESM-binding divergence this fixes. Corrected a misleading comment: forceReset() invalidates the epoch so a stale runAllowed call becomes a no-op, it does not restore real network access -- the block itself stays enforced via AsyncLocalStorage regardless.

…DNS resolvers net.Server.prototype.listen and dgram.Socket.prototype.bind were unguarded inbound entry points, letting a dependency open a real listening socket during blocked execution; both are now guarded the same way the existing outbound dgram send/connect methods are, via the shared guardNetworkMethod wrapper (renamed from guardDgramMethod, since it was always generic). worker_threads.Worker construction is blocked via a Proxy construct trap mirroring guardWebSocket's shape -- a Worker gets a fresh V8 realm that doesn't inherit any of this file's monkeypatches or the AsyncLocalStorage block context, so guarding its internals from the parent thread isn't possible; blocking construction itself is the only enforceable boundary. dns.resolve*()/dns.promises.resolve*() (and their Resolver-class equivalents, across all 4 surfaces: dns, dns.promises, dns.Resolver.prototype, dns.promises.Resolver.prototype) bypass net.Socket/dgram.Socket entirely via Node's native c-ares channel, so neither existing guard ever observed those calls. The promise-returning surfaces (dns.promises.*, dns.promises.Resolver .prototype.*) reject rather than throw synchronously when blocked, matching guardFetch's existing reject-not-throw contract and the native dns.promises API shape -- a caller chaining .catch() outside an async wrapper would otherwise see an uncaught synchronous throw instead of a normal rejection. dns.lookup remains intentionally unguarded, per the existing Out of Scope decision.

…a shared mutable variable The direct-reassignment restore idiom (const original = x; x = mock; ...; x = original) was already handled by tracking which real value was active when each guard was built. A wrapper closure over the previous guard (x = (...a) => previous(...a)) is a distinct pattern some mocking approaches use instead, and it still recursed: the previous guard's own getReal() read the same shared mutable variable the new guard had just set to the wrapper, so unblocked calls looped between the two guards until the stack overflowed. Each guard now closes over its own snapshot of the real delegate taken at build time, so a stale guard reference always resolves to what was real when it was built, regardless of what the property currently holds.

…e essential WHY Several doc comments and single-line // comments in this stack had grown to 280-1100+ characters or 8+ lines by stacking every contributing justification instead of keeping the one that matters. Also extracts a repeated `$.Actions.${pathParts.join('.')}` computation in makeActionsProxy's apply trap into a single local, since it was now computed three times.

… mode Adds `npm run dev:verify`, which starts the dev server with Vite's mode set to `dev-verify` so `/__dd/executeAction` calls route through the same queue + Deno subprocess round trip as production instead of the local in-process path, for pre-publish parity checks.

tyffical requested a balanced review from Copilot August 24, 2026 08:08

Copilot started reviewing on behalf of tyffical August 24, 2026 08:08 View session

Copilot AI reviewed Aug 24, 2026

View reviewed changes

Comment thread packages/plugins/apps/src/vite/dev-server.ts Outdated

Comment thread packages/plugins/apps/src/vite/dev-server.ts Outdated

This comment was marked as resolved.

Sign in to view

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from 6822e55 to c37ab18 Compare August 24, 2026 16:57

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch 2 times, most recently from e979af2 to e30cf71 Compare August 24, 2026 19:16

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from e30cf71 to 2626d97 Compare August 24, 2026 19:29

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from 2626d97 to 7ec3cae Compare August 24, 2026 20:13

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from 7ec3cae to 44b7d0c Compare August 25, 2026 00:30

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from 44b7d0c to 6f4c565 Compare August 25, 2026 01:17

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from 6f4c565 to cb7626e Compare August 25, 2026 15:30

tyffical requested a balanced review from Copilot August 25, 2026 15:45

Copilot started reviewing on behalf of tyffical August 25, 2026 15:46 View session

This comment was marked as resolved.

Sign in to view

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from cb7626e to 5196d11 Compare August 25, 2026 18:39

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from 5196d11 to 5ec8c6e Compare August 26, 2026 01:47

DataDog deleted a comment from chatgpt-codex-connector Bot Aug 26, 2026

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from ac8855c to 1089c4c Compare August 28, 2026 05:49

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from 1089c4c to cca298b Compare August 28, 2026 06:20

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from cca298b to 68bb17f Compare August 28, 2026 06:45

tyffical force-pushed the tiffany.trinh/apps-2792-dev-verify-cli branch from 68bb17f to 9c45d6e Compare August 28, 2026 07:24

tyffical mentioned this pull request Aug 31, 2026

[APPS-2792] Add: wire local execution into the real dev server #481

Merged

chatgpt-codex-connector Bot reviewed Sep 1, 2026

View reviewed changes

Comment thread packages/plugins/apps/src/vite/dev-server-module-graph.ts

Copilot AI reviewed Sep 1, 2026

View reviewed changes

Comment thread packages/plugins/apps/src/vite/network-guard.ts

Comment thread packages/plugins/apps/src/vite/network-guard.ts

Comment thread packages/plugins/apps/src/vite/dev-server-module-graph.ts

tyffical mentioned this pull request Sep 2, 2026

[APPS-2792] Add: local-execution resilience tests #492

Closed

tyffical and others added 14 commits September 2, 2026 14:29

Navigation Menu

Uh oh!

[APPS-2792] Add: dev:verify mode-aware routing for local execution (build-plugins half) - #490

[APPS-2792] Add: dev:verify mode-aware routing for local execution (build-plugins half)#490
tyffical wants to merge 14 commits into
tiffany.trinh/apps-2792-runtime-network-guardfrom
tiffany.trinh/apps-2792-dev-verify-cli

tyffical commented Aug 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

Uh oh!

datadog-prod-us1-4 Bot commented Aug 25, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

Uh oh!

tyffical commented Sep 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 1, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

tyffical commented Sep 1, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

tyffical commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Architecture

Changes

QA Instructions

Blast Radius

Out of Scope / Follow-ups

Documentation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

This comment was marked as resolved.

This comment was marked as resolved.

Uh oh!

datadog-prod-us1-4 Bot commented Aug 25, 2026 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tyffical commented Sep 1, 2026

Uh oh!

chatgpt-codex-connector Bot commented Sep 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

tyffical commented Sep 1, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tyffical commented Aug 21, 2026 •

edited

Loading

datadog-prod-us1-4 Bot commented Aug 25, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading

chatgpt-codex-connector Bot commented Sep 1, 2026 •

edited

Loading