Skip to content

fix(knowledge): align document tag provenance selections with the serialized request - #6332

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/knowledge-tag-provenance-count-divergence
Aug 6, 2026
Merged

fix(knowledge): align document tag provenance selections with the serialized request#6332
waleedlatif1 merged 2 commits into
stagingfrom
fix/knowledge-tag-provenance-count-divergence

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Problem

Same bug class as the selectTableRowSecretProvenance outage fixed in #6325: a private-provenance producer counting pre-serialization semantics while the consumer counts post-serialization.

Knowledge document writes (knowledge_create_document, knowledge_upsert_document):

  • ProducerselectKnowledgeDocumentWriteSecretProvenance counted one name/value selection pair per parseDocumentTags entry. parseDocumentTags keeps an entry whose value is merely truthy, then coerces value: String(entry.value).
  • ConsumerparseKnowledgeDocumentTagProvenanceTargets builds targets from the serialized documentTagsData and drops entries where value === ''.
  • resolveKnowledgeWriteSecretProvenance rejects the write with HTTP 400 when bundle.selections.length !== selectionKeys.length.

So any tag value that is truthy before coercion but stringifies to empty — [], [null], [undefined], an object whose toString() returns '' — was counted by the tool and dropped by the route.

Reproduced by executing both parsers against [{ tagName: 'kept', value: 'value' }, { tagName: 'dropped', value: [] }]: the tool emitted 6 selection keys, the route derived 4.

Not observed in production logs — found by code-level audit.

Semantics

Drop, matching the route. The functional write path agrees already: resolveDocumentTags skips any tag whose value fails its hasValue check, so an empty-stringifying tag never persists a value and has nothing to attribute provenance to.

Fix

Following the update_chunk pattern (its selector mirrors its own body builder exactly), the producer now describes exactly what goes on the wire: it runs the same formatDocumentTagsForAPI(parseDocumentTags(...)) the body builder runs, then feeds the resulting documentTagsData through the same target parser the route uses. parseKnowledgeDocumentTagProvenanceTargets moved from lib/knowledge/secret-provenance.ts (which pulls in @sim/db) to the db-free lib/knowledge/secret-provenance-selection.ts both sides already import, so there is one parser, not two rules that can drift.

Tests

apps/sim/tools/knowledge/secret-provenance.test.ts asserts the tool's selection keys equal the route-derived target keys for each divergent value, plus a non-divergent case.

Verified red before the fix (expected 6 keys to equal 4), green after.

Audit of the other provenance producer/consumer pairs

Pair Status
knowledge document tags fixed here
table rows (selectTableRowSecretProvenancecreateTableWriteProvenanceTargets) Aligned since #6325. Residual theoretical gap: the producer filters only undefined, but JSON.stringify also drops function- and symbol-valued keys, which the consumer's Object.entries over the parsed row would then not see. Not reachable from JSON-sourced tool params — left alone.
workspace files (file_write, file_append) Fixed arity ['content'] on both sides.
memory (memory_add) Fixed arity 1 on both sides.
knowledge chunks (upload_chunk, update_chunk) Fixed arity; update_chunk's conditional producer mirrors both its body builder and the route's validatedData.content === undefined check.
workflow executor Fixed arity 1 on both sides.

Checks

  • bunx tsc --noEmit -p tsconfig.json — clean
  • bun run lint — clean (9 pre-existing unrelated warnings)
  • app/api/knowledge/secret-provenance.test.ts + new test — 14 passed
  • tools/knowledge/knowledge.test.ts fails identically on a clean checkout (postcss/tailwind env issue), unrelated

…ialized request

The create/upsert document tools counted one provenance selection pair per
parseDocumentTags entry, while the write route built targets from the
serialized documentTagsData and dropped entries whose value is the empty
string. A tag value that is truthy before coercion but stringifies to empty
(`[]`, `[null]`, `{ toString: () => '' }`) was therefore counted by the tool
and not by the route, and the bundle length check rejected the write with 400.

Both sides now read one shared parser over the exact bytes that go on the
wire, so their counts cannot diverge.
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Aug 6, 2026 6:55pm

Request Review

@cursor

cursor Bot commented Aug 6, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes secret provenance selection for knowledge document writes; incorrect alignment could still cause write failures (400) or wrong attribution, but scope is limited to tag provenance plumbing with regression tests.

Overview
Fixes a producer/consumer mismatch on knowledge document writes (knowledge_create_document, knowledge_upsert_document): the tool counted tag provenance selections from pre-serialization parseDocumentTags rules, while the API route counted targets from serialized documentTagsData and dropped values that stringify to empty. That mismatch triggered HTTP 400 from resolveKnowledgeWriteSecretProvenance when selection key counts differed (e.g. truthy values like [] that become "").

The tool path now mirrors the wire format: formatDocumentTagsForAPI(parseDocumentTags(...)), then parseKnowledgeDocumentTagProvenanceTargets(documentTagsData)—the same parser the route uses. parseKnowledgeDocumentTagProvenanceTargets moves from secret-provenance.ts to db-free secret-provenance-selection.ts so both sides share one parser.

Adds secret-provenance.test.ts asserting tool selection keys match server-derived keys for empty-stringifying tag values and a normal multi-tag case.

Reviewed by Cursor Bugbot for commit 6e07db1. Configure here.

@greptile-apps

greptile-apps Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

The PR aligns knowledge-document tag provenance selections with the serialized request consumed by the API route.

  • Centralizes parsing of serialized tag provenance targets in a database-independent module.
  • Makes the producer derive selections from the same formatted tag data as the consumer.
  • Adds regression coverage for values that become empty during serialization.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename Overview
apps/sim/lib/knowledge/secret-provenance-selection.ts Centralizes the shared parser that filters serialized document-tag provenance targets.
apps/sim/tools/knowledge/secret-provenance.ts Derives producer-side provenance selections from the same serialized representation used by the API.
apps/sim/app/api/knowledge/secret-provenance.ts Updates the route-side import to use the shared database-independent target parser.
apps/sim/lib/knowledge/secret-provenance.ts Removes the former database-coupled copy of the document-tag target parser.
apps/sim/tools/knowledge/secret-provenance.test.ts Covers divergent pre- and post-serialization tag values and fully applies the previously requested const assertion.

Reviews (2): Last reviewed commit: "test(knowledge): use as const for the em..." | Re-trigger Greptile

Comment thread apps/sim/tools/knowledge/secret-provenance.test.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cursor review

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 6e07db1. Configure here.

@waleedlatif1
waleedlatif1 merged commit 8e3e608 into staging Aug 6, 2026
5 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/knowledge-tag-provenance-count-divergence branch August 6, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant