feat(kilo-mcp): add server-side PostHog analytics to the MCP worker (part 1/1)

[iscekic]

Changelog for users

• The remote MCP worker records PostHog events for session starts, tool calls, searches, rejected calls, and OAuth sign-in outcomes.
• Each tool call records a published tool name or unknown, the resolved catalog path, success or failure, the error class, and the latency.
• Each search records the hit count, whether the result was empty, and the query shape. It never records the query text.
• Each rejected call records the reason: an auth failure, an unknown catalog path, or input that fails the published schema.
• OAuth sign-in records the started, succeeded, and failed phases. A denied pairing is recorded once, not on every status poll.
• Authenticated events bind to the user and the organization. Anonymous events create no PostHog person profile.
• Analytics is best-effort. A capture that fails, rejects, or never resolves leaves the MCP response unchanged.

Changelog for maintainers

• services/kilo-mcp/src/index.ts:331 — accepted: tools/call accepts any string as name, so the recorded value was caller-controlled. Analytics now records only the published names (search, call) and reports every other value as unknown; the path-is-a-real-catalog-key guard is unchanged.
• services/kilo-mcp/src/oauth-pages/authorize-page.ts:179 — accepted: the terminal denied branch re-emitted the sign-in failure on every poll. denyCode only transitions pending → denied at the branch that already emits once, so polling a denied record now answers denied without re-emitting.
• The event builder and transport are new in services/kilo-mcp/src/analytics.ts: a direct capture POST with a 5 second timeout, no SDK, and no new dependency.
• Every emit logs [kilo-mcp] analytics <event> <json>, gates on NEXT_PUBLIC_POSTHOG_KEY, and schedules through waitUntil; the emitter cannot throw.
• OAuth phases come from auth/authorize.ts (started, failed), auth/token.ts (succeeded and invalid_grant), and oauth-pages/authorize-page.ts (denied, expired).
• wrangler.jsonc adds NEXT_PUBLIC_POSTHOG_KEY as a worker var; it is absent from env.dev, so local runs do not send. worker-configuration.d.ts is regenerated with unrelated workerd runtime-type churn — review the generated diff.
• vitest.config.ts now uses the verbose reporter. Confirm this is intended.
• Review hint: this diff adds no account-deletion suppression. Confirm the existing deletion flow stops or anonymises these events.

E2E proof

Stack-wide proof from the stack tip. The injection-name and denial-poll checks replace the earlier claims that a tool call records the raw caller name and that every poll of a denied pairing emits a failure.

• Best-effort guarantee: a PostHog capture that rejects or never resolves leaves the JSON-RPC response unchanged and raises no unhandled rejection.
• Happy path tool call: a resolved call emits one kilo_mcp_tool_called with the published tool name, success=true, the resolved path, and a latency.
• Empty search state: a search with no hits emits kilo_mcp_search_performed hit_count=0 empty=true; a search with hits emits hit_count>0 empty=false.
• Rejected call: an unknown catalog path and schema-invalid input each emit kilo_mcp_tool_called success=false with error_class unknown_path or schema_invalid, plus a rejection event.
• Auth failure: a POST /mcp without an Authorization header answers 401 and emits kilo_mcp_call_rejected reason=auth_failure with distinct_id kilo-mcp-anonymous and no userId.
• Retryable upstream failure: an unreachable Kilo API emits kilo_mcp_tool_called success=false error_class=upstream_unreachable while the MCP response is unchanged.
• Session start: an initialize request emits kilo_mcp_session_started with protocol_version and client_name.
• OAuth sign-in started: a GET /authorize with a registered client emits kilo_mcp_oauth_sign_in_started with client_id and no user identity.
• OAuth sign-in succeeded: a completed authorization_code token exchange emits kilo_mcp_oauth_sign_in_succeeded bound to userId and organizationId.
• OAuth sign-in failed: an expired pairing and an invalid_grant exchange each emit kilo_mcp_oauth_sign_in_failed with reason expired or invalid_grant; a denied pairing emits once at the transition, as the denial-poll check below shows.
• Injection-shaped tool name: an authenticated tools/call whose params.name is an injection-shaped string logs kilo_mcp_tool_called with tool=unknown and errorClass=unknown_tool; the raw name is absent from the log line and any captured payload.
• Denial polled three times: after a denial, three polls of GET /authorize/status each answer {"status":"denied"} and exactly one kilo_mcp_oauth_sign_in_failed with reason=denied is logged across all polls.
• Tool-name regression: an authenticated tools/call for search and for call still logs tool=search and tool=call in kilo_mcp_tool_called.
• Event-builder unit proof: the analytics suite runs green with the property-allowlist, secret-absence, and non-throwing failure-path checks.
• Local worker run: the worker logs [kilo-mcp] analytics kilo_mcp_call_rejected for a POST /mcp without Authorization.

Owner request for the stack

Add PostHog analytics to the kilo-MCP worker

Surface: the kilo-MCP Cloudflare Worker (services/kilo-mcp), the server side
of the remote MCP product. Base this work on PR 6030 (branch
kwf/kilo-remote-mcp-c204-l1). Do not add analytics to apps/web or the mobile
app in this item.

Add server-side PostHog analytics to the worker. Track what is useful to
understand how the remote MCP is used and where it fails. Cover at least:

• an MCP session or connection starts;
• every tool call: the tool name, the catalog path it resolves, success or
  failure, the error class, and the latency;
• a search call: the hit count, whether the result was empty, and the query
  shape only (never raw user text if it can carry personal data);
• a rejected call: auth failure, unknown catalog path, or input that fails
  the published schema;
• OAuth sign-in start, success, and failure when that slice exists.

Rules:

• Use the existing PostHog client and the repository conventions. Do not add
  a second analytics dependency.
• Send through a server-side client only. Never expose the PostHog key to a
  client bundle.
• Never send message content, prompts, tokens, cookies, or any credential.
  Send identifiers, counts, durations, and error classes only.
• Bind each event to the user and the organization when the request is
  authenticated. Do not send anonymous events for authenticated actions.
• Keep analytics best-effort: it must never fail, delay, or block an MCP
  response. Use the platform's waitUntil.
• Respect the existing consent and account-deletion rules. Stop sending or
  anonymise when the user is deleted.

Prove it:

• Run the worker locally and trigger one tool call, one search, and one
  rejected call. Quote the decisive log lines that show each event and its
  properties, and name the log file. A log is valid proof for this item.
• Show the events reach PostHog in a local or preview environment, or show
  the exact captured payloads from a test transport.
• Add unit tests for the event builder: the property names, the absence of
  any secret in every payload, and the failure path that cannot throw.

Do not commit fixtures, end-to-end-only code, or test-only runtime flags.
Do not leave follow-ups.
Let the workflow open and maintain the PR.
Finish only after current-head continuous integration (CI) is green and
Kilobot recommends Merge.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

The incremental fix at HEAD 6c42d90 gates the OAuth failure emit on the winning pending → denied / expired transition and persists upstream expiry as a terminal record, so each pairing emits at most one failure event; no new issues found.

Files Reviewed (11 files)

• services/kilo-mcp/src/db/sqlite-schema.ts
• services/kilo-mcp/src/store/oauth-store.ts
• services/kilo-mcp/src/store/oauth-store.test.ts
• services/kilo-mcp/src/oauth-pages/authorize-page.ts
• services/kilo-mcp/src/oauth-pages/authorize-page.test.ts
• services/kilo-mcp/src/oauth-pages/org-picker.ts
• services/kilo-mcp/src/oauth-pages/org-picker.test.ts
• services/kilo-mcp/src/index.test.ts
• services/kilo-mcp/src/auth/authorize.test.ts
• services/kilo-mcp/src/auth/dcr.test.ts
• services/kilo-mcp/src/auth/token.test.ts

Previous Review Summaries (2 snapshots, latest commit 028ff55)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 028ff55)

Status: No Issues Found | Recommendation: Merge

Executive Summary

The two prior findings are resolved at HEAD 028ff55: caller-supplied tool names are normalized to unknown via PUBLISHED_TOOL_NAMES, and a persisted denial now answers without re-emitting; the incremental diff introduces no new issues.

Files Reviewed (4 files)

• services/kilo-mcp/src/index.ts
• services/kilo-mcp/src/index.test.ts
• services/kilo-mcp/src/oauth-pages/authorize-page.ts
• services/kilo-mcp/src/oauth-pages/authorize-page.test.ts

Previous review (commit 60a8653)

Status: 2 Issues Found | Recommendation: Address before merge

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	1

Issue Details (click to expand)

WARNING

File	Line	Issue
services/kilo-mcp/src/index.ts	331	Caller-controlled tool name emitted verbatim to PostHog

SUGGESTION

File	Line	Issue
services/kilo-mcp/src/oauth-pages/authorize-page.ts	179	Denied pairing re-emits sign-in failure on every poll

Files Reviewed (13 files)

• services/kilo-mcp/src/analytics.ts - 0 issues
• services/kilo-mcp/src/analytics.test.ts - 0 issues
• services/kilo-mcp/src/index.ts - 1 issue
• services/kilo-mcp/src/index.test.ts - 0 issues
• services/kilo-mcp/src/auth/authorize.ts - 0 issues
• services/kilo-mcp/src/auth/authorize.test.ts - 0 issues
• services/kilo-mcp/src/auth/token.ts - 0 issues
• services/kilo-mcp/src/auth/token.test.ts - 0 issues
• services/kilo-mcp/src/oauth-pages/authorize-page.ts - 1 issue
• services/kilo-mcp/src/oauth-pages/authorize-page.test.ts - 0 issues
• services/kilo-mcp/vitest.config.ts - 0 issues
• services/kilo-mcp/wrangler.jsonc - 0 issues
• services/kilo-mcp/worker-configuration.d.ts - generated file (regenerated/churn), not reviewed

Fix these issues in Kilo Cloud

---

_{Reviewed by deepseek-v4.1-flash · Input: 0 · Output: 0 · Cached: 0}

_{Review guidance: REVIEW.md from base branch kwf/kilo-remote-mcp-c204-l1}