diff --git a/pages/publishing/publishing/webhook-publishing.mdx b/pages/publishing/publishing/webhook-publishing.mdx index 0f58d9b9..e170e361 100644 --- a/pages/publishing/publishing/webhook-publishing.mdx +++ b/pages/publishing/publishing/webhook-publishing.mdx @@ -127,6 +127,15 @@ The `eventType` of events that are generated from variants is `variant`, while r | ?enum | array(string) | Allowed enum values | | ?operations | array(string) | Array of property ops, e.g. "SnowplowContext" | | ?pattern | string | Regex pattern for string type properties | +| ?sourcePresence | array(SourcePresence) | Per-source presence for this property on this event. Opt-in per workspace — see [Per-source property presence](#per-source-property-presence) | + +##### SourcePresence Model + +| Attribute | Type | Description | +|:-----------|:----------------------------------------------------|:-----------------------------------------------------| +| sourceId | string | Source id — the join key | +| sourceName | string | Source display name, advisory only | +| presence | enum: ["alwaysSent", "sometimesSent", "neverSent"] | Whether the property is always, sometimes, or never sent from this source on this event | ##### NameMapping Model for regular workspaces @@ -297,3 +306,32 @@ Each line describes a property scoped to an event, user, or group. | maximum | number or null | Maximum numeric value if defined | | regex | string or null | Regex constraint (event override preferred; falls back to property rule) | | stakeholder_domains | array(StakeholderDomain) | Stakeholder domains for this property | +| ?source_presence | array(SourcePresenceNdjson) | Per-source presence for this property on this event. Opt-in per workspace, and absent when the export covers no source for the event — see [Per-source property presence](#per-source-property-presence) | + +##### SourcePresenceNdjson Model + +Keys are snake_case like every other NDJSON column; the values are camelCase, matching the `actions` column. + +| Attribute | Type | Description | +|:------------|:---------------------------------------------------|:-----------------------------------------------------| +| source_id | string | Source id — the join key | +| source_name | string | Source display name, advisory only | +| presence | enum: ["alwaysSent", "sometimesSent", "neverSent"] | Whether the property is always, sometimes, or never sent from this source on this event | + +#### Per-source property presence + +A property can be configured to be sent from some of an event's sources and not others. Both payload formats can carry that breakdown per property: `sourcePresence` on each property entry in the JSON Schema format, and `source_presence` on each line of the properties NDJSON file. + +**This is opt-in per workspace and off by default.** Existing workspaces see no change to either payload until the field is enabled for them — ask Avo support to turn it on. Workspaces created after the field shipped have it on. + +Each entry names one source and states whether the property is `alwaysSent`, `sometimesSent`, or `neverSent` from it on that event. + +- **The breakdown is complete.** Every source the payload covers for that event gets an entry, `alwaysSent` included, so you never have to infer presence from a missing entry. Match on the source id. +- **It covers the sources this payload covers.** Under a source filter it narrows to the filter, exactly as `required` does. In the NDJSON format the event's `sources` array matches that set exactly. In the JSON Schema format the event's `sources` array is not narrowed by the filter, so it can list sources the breakdown does not cover — infer nothing about a listed source that has no entry. +- **The key can be absent**, and only in one case: the export covers no source at all for that event. It is never emitted as an empty array or as null. +- **It is deliberately not on the events NDJSON line.** The property summary embedded there keeps its existing four keys; the breakdown lives on the properties line only, so the payload does not carry it twice. +- **`required` is unchanged.** It stays a single value per property across all covered sources. The breakdown decomposes it rather than replacing it: where any covered source is `sometimesSent` or `neverSent`, the property is not required. One legacy configuration diverges from that correspondence — a property using the older "excluded sources" setting reports `neverSent` for an excluded source while still being listed as required. + + + If you load the NDJSON export into a warehouse with a fixed schema — for example a BigQuery load job with `ignoreUnknownValues=false` — an unexpected column makes the load fail. Widen your schema before the field is enabled for your workspace. + diff --git a/pages/reference/public-api/export-tracking-plan.mdx b/pages/reference/public-api/export-tracking-plan.mdx index e8f4df05..1ccae8a6 100644 --- a/pages/reference/public-api/export-tracking-plan.mdx +++ b/pages/reference/public-api/export-tracking-plan.mdx @@ -77,6 +77,43 @@ An event is considered "changed" when the change impacts a source. For example, - When enabled, the response includes a `removedEvents` field containing an array of event names that were archived or lost their filtered sources on the branch. When disabled, this field is omitted. - Source-aware: when a source filter is applied, only changes relevant to that source are considered. +## Per-source property presence + +A property can be configured to be sent from some of an event's sources and not others. When enabled, each property entry carries that breakdown as `sourcePresence` in the JSON Schema format, and each line of the properties file carries it as `source_presence` in the `ndjson` format. + +**This is opt-in per workspace and off by default.** If you don't see the field, it hasn't been enabled for your workspace — ask Avo support to turn it on. Workspaces created after the field shipped have it on. It is purely additive: no existing field changes value or position when it is enabled. + +```json +"user_tier": { + "id": "p_user_tier", + "index": 2, + "nameMapping": [], + "isSystemProperty": false, + "type": ["string", "null"], + "sourcePresence": [ + { "sourceId": "src_web", "sourceName": "Website", "presence": "alwaysSent" }, + { "sourceId": "src_android", "sourceName": "Android", "presence": "sometimesSent" }, + { "sourceId": "src_ios", "sourceName": "iOS", "presence": "neverSent" } + ] +} +``` + +| Field | Type | Description | +|---|---|---| +| `sourceId` | string | Source id. This is the join key — match on it, not on the name | +| `sourceName` | string | Source display name. Advisory only | +| `presence` | `alwaysSent` \| `sometimesSent` \| `neverSent` | Whether the property is always, sometimes, or never sent from this source on this event | + +How to read it: + +- **The breakdown is complete.** Every source the export covers for that event gets an entry, `alwaysSent` included, so you never have to infer presence from a missing entry. +- **It covers the sources this export covers.** Under a source filter it narrows to the filter, exactly as `required` does. In the `ndjson` format the event's `sources` array matches that set exactly. In the JSON Schema format the event's `sources` array is not narrowed by the filter, so it can list sources the breakdown does not cover — infer nothing about a listed source that has no entry. +- **The key can be absent**, and only in one case: the export covers no source at all for that event. It is never emitted as an empty array or as null. +- **In the `ndjson` format it is on the properties file only**, deliberately not on the embedded property summary in the events file, so the payload does not carry it twice. +- **`required` is unchanged.** It stays a single value per property across all covered sources; the breakdown decomposes it rather than replacing it. Where any covered source is `sometimesSent` or `neverSent`, the property is not required. One legacy configuration diverges: a property using the older "excluded sources" setting reports `neverSent` for an excluded source while still being listed as required. + +If you load the `ndjson` export into a warehouse with a fixed schema — for example a BigQuery load job with `ignoreUnknownValues=false` — an unexpected column makes the load fail. Widen your schema before the field is enabled for your workspace. + ## Governance fields For workspaces using [Governance](/data-design/avo-tracking-plan/governance) (Enterprise plan), the JSON Schema export includes the governance values: