Skip to content

feat(otlp): add OTLP/JSON channel (@microsoft/applicationinsights-otlpchannel-js) - #2751

Draft
Jackson Weber (JacksonWeber) wants to merge 4 commits into
microsoft:mainfrom
JacksonWeber:jacksonweber/otlp-json-channel
Draft

feat(otlp): add OTLP/JSON channel (@microsoft/applicationinsights-otlpchannel-js)#2751
Jackson Weber (JacksonWeber) wants to merge 4 commits into
microsoft:mainfrom
JacksonWeber:jacksonweber/otlp-json-channel

Conversation

@JacksonWeber

@JacksonWeber Jackson Weber (JacksonWeber) commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds @microsoft/applicationinsights-otlpchannel-js (0.1.0), a preview browser channel that converts Application Insights telemetry to OTLP/JSON and exports it over OTLP/HTTP to /v1/traces and /v1/logs.

The channel is implemented as a production-capable replacement for the classic Sender's transferable behavior. It uses the SDK's existing core transport infrastructure and has no @opentelemetry/* dependency.

Telemetry mapping

Application Insights baseType OTLP representation
RequestData SERVER span
RemoteDependencyData CLIENT span, or INTERNAL for InProc
PageviewData INTERNAL span by default; configurable as a log
Native Common Schema OTelSpan Span preserving kind, parent, trace state, status, and attributes
MessageData LogRecord with severity
ExceptionData LogRecord with exception.* attributes and chained exceptions
EventData LogRecord with eventName
PageviewPerformanceData LogRecord
MetricData LogRecord by default so replacing Sender does not drop metrics

Context tags are promoted onto the OTLP Resource. Remaining tags, Part A extensions, Part C data, custom properties, and measurements become record attributes; Application Insights-specific values use the microsoft. namespace.

Conversion and batching

  • Converts telemetry on processTelemetry, before the send path.
  • Pre-serializes records and calculates their UTF-8 size once at ingestion.
  • Groups records by resource and OTLP signal.
  • Splits batches by complete encoded payload size, including resource/scope envelopes and separators.
  • Sends spans and logs to their signal-specific endpoints.
  • Carries per-record byte size, retry attempt, notification summary, and durable ID through split/requeue operations.
  • Supports pause, resume, synchronous/asynchronous flush, unload flush, teardown, and isCompletelyIdle().
  • Uses priority 1021 and continues the plugin chain unless consumeEvents is enabled.

Production durability

Includes Sender-equivalent integrated persistence rather than relying on the generic OfflineChannel contract:

  • Session-storage buffering is enabled by default.
  • Separate unsent and unacknowledged sent buffers survive page reloads.
  • Destination-first sent/unsent transitions tolerate interrupted or failed storage writes; temporary overlap is deduplicated by stable record ID.
  • Durable capacity includes both unsent and in-flight records.
  • Dynamic namePrefix, storage enablement, and custom bufferOverride changes use destination-first migration and dual-write while requests are in flight.
  • Browser online/offline detection keeps records persisted while offline and schedules replay after reconnect.
  • Offline unload does not consume retry budget or discard recoverable records.
  • Fetch keepalive only reports that an unload request was queued, so unload records remain persisted for at-least-once replay rather than being deleted before collector acknowledgement.
  • namePrefix and the classic IStorageBuffer override contract are supported.

getOfflineSupport() intentionally returns null: the generic OfflineChannel contract carries one endpoint per payload and cannot safely represent OTLP's separate trace and log envelopes. The invalid OfflineChannel custom-SKU example has been removed.

Transport and reliability

Uses SenderPostManager and includes:

  • Fetch/XHR plus custom httpXHROverride support.
  • No Beacon transport: Beacon cannot reliably preserve OTLP's application/json content type or authentication headers.
  • Custom headers/authentication, fetch credentials, XHR timeout, and explicit trace/log endpoints.
  • Redirect affinity for final Fetch/XHR collector endpoints, with bounded redirect learning.
  • Exponential retry with jitter and case-insensitive Retry-After support.
  • Fetch network failures (499 internal status) normalized as retryable failures.
  • Configurable retry codes, retry disablement, and bounded normal/unload retry attempts.
  • Retry attempt preservation across batch requeue and splitting.
  • Optional single-record unload splitting for keepalive-sized payloads.
  • OTLP partial-success parsing with aggregate accepted/discarded notifications (the protocol reports counts, not record indexes).
  • Optional asynchronous gzip through CompressionStream, also honoring the SDK zipPayload feature flag.
  • Sender-equivalent deterministic percentage sampling; metrics are never sampled out.
  • eventsSendRequest, eventsSent, eventsRetry, and eventsDiscarded lifecycle notifications for SDK Stats/listeners.
  • Dynamic configuration for endpoints, transports, batching, persistence, conversion, resources, privacy, sampling, retries, and compression.

The shared SenderPostManager Fetch path now forwards non-2xx response bodies and headers through its completion callback so channels can honor collector Retry-After headers.

Privacy

OTLP has no equivalent of Common Schema PII/customer-content metadata. piiMode defaults to drop; it can alternatively hash values or retain them with marker attributes for downstream scrubbing.

The instrumentation key is excluded from the resource by default and can be included explicitly with includeIKeyInResource.

Example and protocol validation

Adds examples/otlp, a multi-page test site with two isolated SDK instances per page and a local mock collector. It validates:

  • OTLP envelopes and signal endpoints.
  • Resource/scope structure and expected attributes.
  • Trace/span IDs, kinds, status, parents, and nanosecond timestamps.
  • Log timestamps and severity mapping.
  • AnyValue correctness and duplicate-key prevention.
  • PII drop/hash behavior.
  • Multi-instance isolation and independent unload.

The example can also send through a real OpenTelemetry Collector so malformed payloads are rejected by the protocol implementation.

Tests

The OTLP suite contains 130 passing tests, including:

  • Conversion and field-level fidelity.
  • Sampling, compression, retries, retry exhaustion, partial success, notifications, and idle state.
  • Crash-safe unsent/sent recovery and deduplication.
  • Offline unload/reload/reconnect behavior.
  • In-flight dynamic storage migration.
  • UTF-8 payload byte limits and unload splitting.
  • Redirect affinity and Fetch retry semantics.
  • Throwing transports and late async completion safety.

The shared AppInsightsCore suite contains 909 passing tests after the Fetch completion change.

Known protocol limitations

  • The OTLP metrics signal (/v1/metrics) is not implemented; MetricData is retained as logs.
  • Span events and links are not populated.
  • Conversion from the Application Insights telemetry shape back to OTLP cannot be perfectly lossless, although custom span attributes are preserved through baseData.properties.
  • At-least-once unload recovery can produce duplicates when a queued keepalive request reaches the collector but the page unloads before acknowledgement. This is intentional: duplicate-safe recovery is preferred over silent telemetry loss.
  • Collectors must permit the CORS preflight required by OTLP/JSON and custom authentication headers.

Repository wiring

Registers the package in rush.json, gruntfile.js, version.json, .aiAutoMinify.json, tools/release-tools/package_groups.json, and RELEASES.md.

This remains a draft while protocol limitations and final package/release decisions are evaluated.

…pchannel-js)

Adds a new preview channel that converts Application Insights telemetry to
OTLP/JSON in memory and exports it to an OTLP HTTP endpoint (/v1/traces and
/v1/logs) with no @opentelemetry/* dependency.

- Conversion and serialization happen on the processTelemetry path, with
  records buffered pre-grouped by resource and signal and incremental byte
  accounting, so flushing a batch performs no conversion work
- RequestData / RemoteDependencyData / PageviewData export as spans;
  MessageData / ExceptionData / EventData / PageviewPerformanceData export as
  log records; native Common Schema OTelSpan items export as spans directly
- Context tags are promoted onto the OTLP Resource, other values become record
  attributes namespaced under microsoft.
- Values marked as PII or customer content are dropped by default (piiMode)
- Implements getOfflineSupport() for use with the offline channel
- Adds examples/otlp, a multi page test site running two isolated SDK
  instances per page against a local mock OTLP collector, runnable headlessly
- Registers the package in rush.json, gruntfile.js, version.json,
  .aiAutoMinify.json and package_groups.json

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/validate.js Fixed
Comment thread examples/otlp/tools/server.js Fixed
Add Sender-equivalent sampling, compression, lifecycle notifications, retry controls, unload retry accounting, and idle-state reporting. Preserve retry attempts and original telemetry summaries across OTLP batching.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add crash-safe session-storage recovery, offline-aware replay, UTF-8 batch limits, unload at-least-once semantics, redirect affinity, and Fetch retry header propagation. Remove the incompatible generic OfflineChannel example.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants