feat(telemetry): add opt-in OTLP usage traces - #4821
Conversation
Add a redacted OTLP/HTTP exporter for model and tool usage records, with batching, endpoint configuration, shutdown flushing, and runtime documentation. Generated-by: OpenAI Codex Signed-off-by: seekskyworld <djh1813553759@gmail.com>
Astro-Han
left a comment
There was a problem hiding this comment.
Reviewed current head d193c6843dc779b0ec996bcdace8ef0232f839b3 (OPEN). Three P2s plus three P3s below, no P0/P1. Default-off is done right; the problems all sit behind the opt-in — new network paths that do not reuse this repo's existing safeguards. This is a new feature with a new outbound path, so it needs a human decision regardless.
P2 — insert is double-wrapped in a lease; every model/tool call re-reads the root marker file
recordLlmCall/recordToolInvocation now wrap in an extra run(...) inside admitSessionUsageMutation, which already runs once (usage-stores.ts:385) — run(run(insert)). Not a deadlock (beginOperation at root-authority.ts:738 is ref-counted, reentrant), but every runWithStorageRootLease awaits assertRootIdentity → confirmRootSnapshot → readAndValidateRootMarker (:1055-1069), a disk read plus validation — on the two hottest telemetry paths. And it is unrelated to adding an exporter. Fix: drop the inner run; the outer admission already guarantees lease context.
P2 — outbound request headers unvalidated while a hardened helper already exists
parseHeaders (otlp-telemetry-exporter.ts) splits OTEL_EXPORTER_OTLP_HEADERS and stuffs them into fetch headers verbatim: no name-charset check, no control-character rejection, no protected-header denylist — while packages/core/src/request-customization.ts already has the HEADER_NAME regex and PROTECTED_HEADERS set (authorization/host/content-length/…). Threat model note: these come from the operator's own env, not a remote attacker — but env may come from someone else's shell profile or CI config. Concretely: user headers spread after 'content-type': 'application/json', so a user header can override content-type while the body stays JSON — silently undecodable at the collector and painful to debug. Minimum: reject control characters and refuse to override content-type.
P2 — credentials allowed over plaintext http with no warning
resolveEndpoint accepts anything new URL() parses, scheme unrestricted — so http://… plus the README's own authorization=Bearer%20token example sends the token and all telemetry in cleartext. Do not ban http (LAN collectors legitimately use it), but warn once on the "non-https + authorization header" combination; the README example shows exactly this pairing and will be copy-pasted.
P2 — README claims "bounded error classes" that neither types nor code guarantee
errorClass is errorClass?: string (usage-stats/types.ts:131) — a free string. Most producers use constants, but at least one reads from an event: session-projection-helpers.ts:172 returns event.reason ?? event.code ?? 'unknown', and whether event.reason is bounded was not traced to the root — so leakage cannot be asserted, but neither can its absence. An export boundary promise ("never exported") should be enforced at the boundary, not by upstream discipline: allowlist or length-truncate errorClass in exportLlmCall/exportToolInvocation.
P3s (non-blocking)
- Nanosecond timestamps exceed double-precision range:
startedAt(~1.77e12 ms) × 1e6 ≈ 1.77e18 >MAX_SAFE_INTEGER— the last digits ofstartTimeUnixNanoare rounding artifacts, not real values. - Each span is its own trace, so these are events, not traces — name/shape expectations should say so.
- A dead collector logs one
console.errorper second with no backoff or kill-switch.
Checked and found sound
Attribute set matches the README promise (no prompts, contents, args/results, paths); default-off; per-principal quota logic untouched.
What I could not judge
No collector was started and the export path was not run end-to-end; all conclusions are from code. Whether event.reason can carry free text was not traced to the root — someone familiar with runtime event sources should confirm, since it decides whether the README sentence can stay.
Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.
简体中文
本条结论全部来自 @Opus-Qronos-AstroHan 的审查。我自己没有读这份 diff;我核的是当前 head 有没有漂移。当前 head 是 d193c68,未关闭。默认关闭做对了,三条 P2 都在打开之后:双层 lease 多读文件、出网头没校验、明文 http 不提醒、错误类别无界,另有三条小的提醒。新功能加出网,按规矩要人类拍板。
Summary
Fixes #3757
Add an opt-in OTLP/HTTP exporter at the Runtime Host Usage boundary. It discovers the standard OTEL endpoint, header, service, and resource environment variables, batches model/tool spans, flushes during shutdown, and keeps prompts, message contents, tool arguments/results, credentials, and session paths out of exported data. Without an endpoint configured, the exporter remains a no-op.
Verification
npm --workspace @maka/core run buildnpm --workspace @maka/storage run buildnpm --workspace @maka/storage run test:dist(1,130 tests: 1,122 passed, 8 skipped)node --test dist/__tests__/otlp-telemetry-exporter.test.js(4 passed after final exporter fixes)npx biome checkon changed source, test, and README filesnpm run check:asf-headersAI use
Tool(s) and scope: OpenAI Codex authored the exporter implementation, tests, and documentation under maintainer review.
Checklist
Does this PR entail a change in behavior?