Skip to content

feat(telemetry): add opt-in OTLP usage traces - #4821

Open
seekskyworld wants to merge 1 commit into
apache:mainfrom
seekskyworld:feat/3757-otlp-usage-traces
Open

feat(telemetry): add opt-in OTLP usage traces#4821
seekskyworld wants to merge 1 commit into
apache:mainfrom
seekskyworld:feat/3757-otlp-usage-traces

Conversation

@seekskyworld

Copy link
Copy Markdown
Contributor

Summary

Fixes #3757

Add an opt-in OTLP/HTTP exporter at the Runtime Host Usage boundary. It discovers the standard OTEL endpoint, header, service, and resource environment variables, batches model/tool spans, flushes during shutdown, and keeps prompts, message contents, tool arguments/results, credentials, and session paths out of exported data. Without an endpoint configured, the exporter remains a no-op.

Verification

  • npm --workspace @maka/core run build
  • npm --workspace @maka/storage run build
  • npm --workspace @maka/storage run test:dist (1,130 tests: 1,122 passed, 8 skipped)
  • node --test dist/__tests__/otlp-telemetry-exporter.test.js (4 passed after final exporter fixes)
  • npx biome check on changed source, test, and README files
  • npm run check:asf-headers

AI use

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: OpenAI Codex authored the exporter implementation, tests, and documentation under maintainer review.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

Add a redacted OTLP/HTTP exporter for model and tool usage records, with batching, endpoint configuration, shutdown flushing, and runtime documentation.

Generated-by: OpenAI Codex
Signed-off-by: seekskyworld <djh1813553759@gmail.com>
@github-actions github-actions Bot added the effort/M Under 500 readable lines label Sep 5, 2026

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed current head d193c6843dc779b0ec996bcdace8ef0232f839b3 (OPEN). Three P2s plus three P3s below, no P0/P1. Default-off is done right; the problems all sit behind the opt-in — new network paths that do not reuse this repo's existing safeguards. This is a new feature with a new outbound path, so it needs a human decision regardless.

P2 — insert is double-wrapped in a lease; every model/tool call re-reads the root marker file

recordLlmCall/recordToolInvocation now wrap in an extra run(...) inside admitSessionUsageMutation, which already runs once (usage-stores.ts:385) — run(run(insert)). Not a deadlock (beginOperation at root-authority.ts:738 is ref-counted, reentrant), but every runWithStorageRootLease awaits assertRootIdentityconfirmRootSnapshotreadAndValidateRootMarker (:1055-1069), a disk read plus validation — on the two hottest telemetry paths. And it is unrelated to adding an exporter. Fix: drop the inner run; the outer admission already guarantees lease context.

P2 — outbound request headers unvalidated while a hardened helper already exists

parseHeaders (otlp-telemetry-exporter.ts) splits OTEL_EXPORTER_OTLP_HEADERS and stuffs them into fetch headers verbatim: no name-charset check, no control-character rejection, no protected-header denylist — while packages/core/src/request-customization.ts already has the HEADER_NAME regex and PROTECTED_HEADERS set (authorization/host/content-length/…). Threat model note: these come from the operator's own env, not a remote attacker — but env may come from someone else's shell profile or CI config. Concretely: user headers spread after 'content-type': 'application/json', so a user header can override content-type while the body stays JSON — silently undecodable at the collector and painful to debug. Minimum: reject control characters and refuse to override content-type.

P2 — credentials allowed over plaintext http with no warning

resolveEndpoint accepts anything new URL() parses, scheme unrestricted — so http://… plus the README's own authorization=Bearer%20token example sends the token and all telemetry in cleartext. Do not ban http (LAN collectors legitimately use it), but warn once on the "non-https + authorization header" combination; the README example shows exactly this pairing and will be copy-pasted.

P2 — README claims "bounded error classes" that neither types nor code guarantee

errorClass is errorClass?: string (usage-stats/types.ts:131) — a free string. Most producers use constants, but at least one reads from an event: session-projection-helpers.ts:172 returns event.reason ?? event.code ?? 'unknown', and whether event.reason is bounded was not traced to the root — so leakage cannot be asserted, but neither can its absence. An export boundary promise ("never exported") should be enforced at the boundary, not by upstream discipline: allowlist or length-truncate errorClass in exportLlmCall/exportToolInvocation.

P3s (non-blocking)

  • Nanosecond timestamps exceed double-precision range: startedAt (~1.77e12 ms) × 1e6 ≈ 1.77e18 > MAX_SAFE_INTEGER — the last digits of startTimeUnixNano are rounding artifacts, not real values.
  • Each span is its own trace, so these are events, not traces — name/shape expectations should say so.
  • A dead collector logs one console.error per second with no backoff or kill-switch.

Checked and found sound

Attribute set matches the README promise (no prompts, contents, args/results, paths); default-off; per-principal quota logic untouched.

What I could not judge

No collector was started and the export path was not run end-to-end; all conclusions are from code. Whether event.reason can carry free text was not traced to the root — someone familiar with runtime event sources should confirm, since it decides whether the README sentence can stay.


Automated review notice: This comment was posted by an automated review agent operated by Astro-Han. It is not an independent human review and does not replace one.

简体中文

本条结论全部来自 @Opus-Qronos-AstroHan 的审查。我自己没有读这份 diff;我核的是当前 head 有没有漂移。当前 head 是 d193c68,未关闭。默认关闭做对了,三条 P2 都在打开之后:双层 lease 多读文件、出网头没校验、明文 http 不提醒、错误类别无界,另有三条小的提醒。新功能加出网,按规矩要人类拍板。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/M Under 500 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add Opentelemetry integration to export monitoring data to third-party APM platforms

2 participants