From e1941fb9e4023dd93573ed360108f3fdc870bbdb Mon Sep 17 00:00:00 2001 From: Claude Date: Wed, 19 Aug 2026 17:03:38 +0000 Subject: [PATCH] Document per-source property presence in the export payloads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the `sourcePresence` / `source_presence` field to the Export Tracking Plan API reference and to the webhook and S3 publishing reference, for AVO-2293. Both pages state that the field is opt-in per workspace and absent by default, so a customer comparing the docs against their own payload is not left thinking it is missing in error. Both also state how to read it: the breakdown is complete over the sources the payload covers, so a missing entry is never a presence claim; it narrows under a source filter exactly as `required` does; the key is absent only when the payload covers no source for the event; and `required` keeps its current meaning, including the one legacy configuration that diverges from it. The webhook page additionally documents that the field is on the properties NDJSON line only, and carries the heads-up that a warehouse load job with a fixed schema rejects an unexpected column. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_016ZAPjCbWE49UCoKUYWSSng --- .../publishing/webhook-publishing.mdx | 38 +++++++++++++++++++ .../public-api/export-tracking-plan.mdx | 37 ++++++++++++++++++ 2 files changed, 75 insertions(+) diff --git a/pages/publishing/publishing/webhook-publishing.mdx b/pages/publishing/publishing/webhook-publishing.mdx index 0f58d9b93..e170e3617 100644 --- a/pages/publishing/publishing/webhook-publishing.mdx +++ b/pages/publishing/publishing/webhook-publishing.mdx @@ -127,6 +127,15 @@ The `eventType` of events that are generated from variants is `variant`, while r | ?enum | array(string) | Allowed enum values | | ?operations | array(string) | Array of property ops, e.g. "SnowplowContext" | | ?pattern | string | Regex pattern for string type properties | +| ?sourcePresence | array(SourcePresence) | Per-source presence for this property on this event. Opt-in per workspace — see [Per-source property presence](#per-source-property-presence) | + +##### SourcePresence Model + +| Attribute | Type | Description | +|:-----------|:----------------------------------------------------|:-----------------------------------------------------| +| sourceId | string | Source id — the join key | +| sourceName | string | Source display name, advisory only | +| presence | enum: ["alwaysSent", "sometimesSent", "neverSent"] | Whether the property is always, sometimes, or never sent from this source on this event | ##### NameMapping Model for regular workspaces @@ -297,3 +306,32 @@ Each line describes a property scoped to an event, user, or group. | maximum | number or null | Maximum numeric value if defined | | regex | string or null | Regex constraint (event override preferred; falls back to property rule) | | stakeholder_domains | array(StakeholderDomain) | Stakeholder domains for this property | +| ?source_presence | array(SourcePresenceNdjson) | Per-source presence for this property on this event. Opt-in per workspace, and absent when the export covers no source for the event — see [Per-source property presence](#per-source-property-presence) | + +##### SourcePresenceNdjson Model + +Keys are snake_case like every other NDJSON column; the values are camelCase, matching the `actions` column. + +| Attribute | Type | Description | +|:------------|:---------------------------------------------------|:-----------------------------------------------------| +| source_id | string | Source id — the join key | +| source_name | string | Source display name, advisory only | +| presence | enum: ["alwaysSent", "sometimesSent", "neverSent"] | Whether the property is always, sometimes, or never sent from this source on this event | + +#### Per-source property presence + +A property can be configured to be sent from some of an event's sources and not others. Both payload formats can carry that breakdown per property: `sourcePresence` on each property entry in the JSON Schema format, and `source_presence` on each line of the properties NDJSON file. + +**This is opt-in per workspace and off by default.** Existing workspaces see no change to either payload until the field is enabled for them — ask Avo support to turn it on. Workspaces created after the field shipped have it on. + +Each entry names one source and states whether the property is `alwaysSent`, `sometimesSent`, or `neverSent` from it on that event. + +- **The breakdown is complete.** Every source the payload covers for that event gets an entry, `alwaysSent` included, so you never have to infer presence from a missing entry. Match on the source id. +- **It covers the sources this payload covers.** Under a source filter it narrows to the filter, exactly as `required` does. In the NDJSON format the event's `sources` array matches that set exactly. In the JSON Schema format the event's `sources` array is not narrowed by the filter, so it can list sources the breakdown does not cover — infer nothing about a listed source that has no entry. +- **The key can be absent**, and only in one case: the export covers no source at all for that event. It is never emitted as an empty array or as null. +- **It is deliberately not on the events NDJSON line.** The property summary embedded there keeps its existing four keys; the breakdown lives on the properties line only, so the payload does not carry it twice. +- **`required` is unchanged.** It stays a single value per property across all covered sources. The breakdown decomposes it rather than replacing it: where any covered source is `sometimesSent` or `neverSent`, the property is not required. One legacy configuration diverges from that correspondence — a property using the older "excluded sources" setting reports `neverSent` for an excluded source while still being listed as required. + + + If you load the NDJSON export into a warehouse with a fixed schema — for example a BigQuery load job with `ignoreUnknownValues=false` — an unexpected column makes the load fail. Widen your schema before the field is enabled for your workspace. + diff --git a/pages/reference/public-api/export-tracking-plan.mdx b/pages/reference/public-api/export-tracking-plan.mdx index e8f4df05e..1ccae8a6f 100644 --- a/pages/reference/public-api/export-tracking-plan.mdx +++ b/pages/reference/public-api/export-tracking-plan.mdx @@ -77,6 +77,43 @@ An event is considered "changed" when the change impacts a source. For example, - When enabled, the response includes a `removedEvents` field containing an array of event names that were archived or lost their filtered sources on the branch. When disabled, this field is omitted. - Source-aware: when a source filter is applied, only changes relevant to that source are considered. +## Per-source property presence + +A property can be configured to be sent from some of an event's sources and not others. When enabled, each property entry carries that breakdown as `sourcePresence` in the JSON Schema format, and each line of the properties file carries it as `source_presence` in the `ndjson` format. + +**This is opt-in per workspace and off by default.** If you don't see the field, it hasn't been enabled for your workspace — ask Avo support to turn it on. Workspaces created after the field shipped have it on. It is purely additive: no existing field changes value or position when it is enabled. + +```json +"user_tier": { + "id": "p_user_tier", + "index": 2, + "nameMapping": [], + "isSystemProperty": false, + "type": ["string", "null"], + "sourcePresence": [ + { "sourceId": "src_web", "sourceName": "Website", "presence": "alwaysSent" }, + { "sourceId": "src_android", "sourceName": "Android", "presence": "sometimesSent" }, + { "sourceId": "src_ios", "sourceName": "iOS", "presence": "neverSent" } + ] +} +``` + +| Field | Type | Description | +|---|---|---| +| `sourceId` | string | Source id. This is the join key — match on it, not on the name | +| `sourceName` | string | Source display name. Advisory only | +| `presence` | `alwaysSent` \| `sometimesSent` \| `neverSent` | Whether the property is always, sometimes, or never sent from this source on this event | + +How to read it: + +- **The breakdown is complete.** Every source the export covers for that event gets an entry, `alwaysSent` included, so you never have to infer presence from a missing entry. +- **It covers the sources this export covers.** Under a source filter it narrows to the filter, exactly as `required` does. In the `ndjson` format the event's `sources` array matches that set exactly. In the JSON Schema format the event's `sources` array is not narrowed by the filter, so it can list sources the breakdown does not cover — infer nothing about a listed source that has no entry. +- **The key can be absent**, and only in one case: the export covers no source at all for that event. It is never emitted as an empty array or as null. +- **In the `ndjson` format it is on the properties file only**, deliberately not on the embedded property summary in the events file, so the payload does not carry it twice. +- **`required` is unchanged.** It stays a single value per property across all covered sources; the breakdown decomposes it rather than replacing it. Where any covered source is `sometimesSent` or `neverSent`, the property is not required. One legacy configuration diverges: a property using the older "excluded sources" setting reports `neverSent` for an excluded source while still being listed as required. + +If you load the `ndjson` export into a warehouse with a fixed schema — for example a BigQuery load job with `ignoreUnknownValues=false` — an unexpected column makes the load fail. Widen your schema before the field is enabled for your workspace. + ## Governance fields For workspaces using [Governance](/data-design/avo-tracking-plan/governance) (Enterprise plan), the JSON Schema export includes the governance values: