Skip to content

Document per-source property presence in the export payloads #1739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
logason merged 1 commit into from
Aug 24, 2026
Merged

Document per-source property presence in the export payloads #1739

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions pages/publishing/publishing/webhook-publishing.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,15 @@ The `eventType` of events that are generated from variants is `variant`, while r
| ?enum | array(string) | Allowed enum values |
| ?operations | array(string) | Array of property ops, e.g. "SnowplowContext" |
| ?pattern | string | Regex pattern for string type properties |
| ?sourcePresence | array(SourcePresence) | Per-source presence for this property on this event. Opt-in per workspace — see [Per-source property presence](#per-source-property-presence) |

##### SourcePresence Model

| Attribute | Type | Description |
|:-----------|:----------------------------------------------------|:-----------------------------------------------------|
| sourceId | string | Source id — the join key |
| sourceName | string | Source display name, advisory only |
| presence | enum: ["alwaysSent", "sometimesSent", "neverSent"] | Whether the property is always, sometimes, or never sent from this source on this event |

##### NameMapping Model for regular workspaces

Expand Down Expand Up @@ -297,3 +306,32 @@ Each line describes a property scoped to an event, user, or group.
| maximum | number or null | Maximum numeric value if defined |
| regex | string or null | Regex constraint (event override preferred; falls back to property rule) |
| stakeholder_domains | array(StakeholderDomain) | Stakeholder domains for this property |
| ?source_presence | array(SourcePresenceNdjson) | Per-source presence for this property on this event. Opt-in per workspace, and absent when the export covers no source for the event — see [Per-source property presence](#per-source-property-presence) |

##### SourcePresenceNdjson Model

Keys are snake_case like every other NDJSON column; the values are camelCase, matching the `actions` column.

| Attribute | Type | Description |
|:------------|:---------------------------------------------------|:-----------------------------------------------------|
| source_id | string | Source id — the join key |
| source_name | string | Source display name, advisory only |
| presence | enum: ["alwaysSent", "sometimesSent", "neverSent"] | Whether the property is always, sometimes, or never sent from this source on this event |

#### Per-source property presence

A property can be configured to be sent from some of an event's sources and not others. Both payload formats can carry that breakdown per property: `sourcePresence` on each property entry in the JSON Schema format, and `source_presence` on each line of the properties NDJSON file.

**This is opt-in per workspace and off by default.** Existing workspaces see no change to either payload until the field is enabled for them — ask Avo support to turn it on. Workspaces created after the field shipped have it on.

Each entry names one source and states whether the property is `alwaysSent`, `sometimesSent`, or `neverSent` from it on that event.

- **The breakdown is complete.** Every source the payload covers for that event gets an entry, `alwaysSent` included, so you never have to infer presence from a missing entry. Match on the source id.
- **It covers the sources this payload covers.** Under a source filter it narrows to the filter, exactly as `required` does. In the NDJSON format the event's `sources` array matches that set exactly. In the JSON Schema format the event's `sources` array is not narrowed by the filter, so it can list sources the breakdown does not cover — infer nothing about a listed source that has no entry.
- **The key can be absent**, and only in one case: the export covers no source at all for that event. It is never emitted as an empty array or as null.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- **It is deliberately not on the events NDJSON line.** The property summary embedded there keeps its existing four keys; the breakdown lives on the properties line only, so the payload does not carry it twice.
- **`required` is unchanged.** It stays a single value per property across all covered sources. The breakdown decomposes it rather than replacing it: where any covered source is `sometimesSent` or `neverSent`, the property is not required. One legacy configuration diverges from that correspondence — a property using the older "excluded sources" setting reports `neverSent` for an excluded source while still being listed as required.

<Callout type="warning">
If you load the NDJSON export into a warehouse with a fixed schema — for example a BigQuery load job with `ignoreUnknownValues=false` — an unexpected column makes the load fail. Widen your schema before the field is enabled for your workspace.
</Callout>
37 changes: 37 additions & 0 deletions pages/reference/public-api/export-tracking-plan.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,43 @@ An event is considered "changed" when the change impacts a source. For example,
- When enabled, the response includes a `removedEvents` field containing an array of event names that were archived or lost their filtered sources on the branch. When disabled, this field is omitted.
- Source-aware: when a source filter is applied, only changes relevant to that source are considered.

## Per-source property presence

A property can be configured to be sent from some of an event's sources and not others. When enabled, each property entry carries that breakdown as `sourcePresence` in the JSON Schema format, and each line of the properties file carries it as `source_presence` in the `ndjson` format.

**This is opt-in per workspace and off by default.** If you don't see the field, it hasn't been enabled for your workspace — ask Avo support to turn it on. Workspaces created after the field shipped have it on. It is purely additive: no existing field changes value or position when it is enabled.

```json
"user_tier": {
"id": "p_user_tier",
"index": 2,
"nameMapping": [],
"isSystemProperty": false,
"type": ["string", "null"],
"sourcePresence": [
{ "sourceId": "src_web", "sourceName": "Website", "presence": "alwaysSent" },
{ "sourceId": "src_android", "sourceName": "Android", "presence": "sometimesSent" },
{ "sourceId": "src_ios", "sourceName": "iOS", "presence": "neverSent" }
]
}
```

| Field | Type | Description |
|---|---|---|
| `sourceId` | string | Source id. This is the join key — match on it, not on the name |
| `sourceName` | string | Source display name. Advisory only |
| `presence` | `alwaysSent` \| `sometimesSent` \| `neverSent` | Whether the property is always, sometimes, or never sent from this source on this event |

How to read it:

- **The breakdown is complete.** Every source the export covers for that event gets an entry, `alwaysSent` included, so you never have to infer presence from a missing entry.
- **It covers the sources this export covers.** Under a source filter it narrows to the filter, exactly as `required` does. In the `ndjson` format the event's `sources` array matches that set exactly. In the JSON Schema format the event's `sources` array is not narrowed by the filter, so it can list sources the breakdown does not cover — infer nothing about a listed source that has no entry.
- **The key can be absent**, and only in one case: the export covers no source at all for that event. It is never emitted as an empty array or as null.
- **In the `ndjson` format it is on the properties file only**, deliberately not on the embedded property summary in the events file, so the payload does not carry it twice.
- **`required` is unchanged.** It stays a single value per property across all covered sources; the breakdown decomposes it rather than replacing it. Where any covered source is `sometimesSent` or `neverSent`, the property is not required. One legacy configuration diverges: a property using the older "excluded sources" setting reports `neverSent` for an excluded source while still being listed as required.

If you load the `ndjson` export into a warehouse with a fixed schema — for example a BigQuery load job with `ignoreUnknownValues=false` — an unexpected column makes the load fail. Widen your schema before the field is enabled for your workspace.

## Governance fields

For workspaces using [Governance](/data-design/avo-tracking-plan/governance) (Enterprise plan), the JSON Schema export includes the governance values:
Expand Down
Loading