feat: add insert_workflow op for template insertion into existing documents - #183
christian-byrne wants to merge 6 commits into
Conversation
Adds a transactional insert_workflow op that merges template nodes, links, and definitions.subgraphs into an existing document. Colliding definition ids dedupe when identical or fork with a content-hash suffix. ID allocation is the minter's job (remapWorkflowIds); the applier validates only. Rejection codes: malformed_op, invalid_node_payload, node_id_collision, link_id_collision, catalog_required. Invariants touched: KA-1..KA-5, FC-1, FC-3, FC-4. Design: Notion TDD (agent subgraph lifecycle), ADR-T8 / Linear BE-10305.
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueComment |
christian-byrne
left a comment
There was a problem hiding this comment.
Three correctness blockers: order-dependent definitions, dangling links accepted, and deterministic fork ids overwritten.
Full context for agent readers
Gates on Node 25.9.0: build passed; typecheck passed; lint passed with 1,707 existing warnings and no errors; purity, pins, profile claims, CodeRabbit config, statelessness, and corpus checks passed. The full test run reached 989 passing and 8 failing tests; every failure was in the dependency-cruiser gate because dependency-cruiser does not support Node 25. The direct import check was therefore inconclusive (exit 2).
Reviewer probes: existing node-id collision rejection passed in the PR suite; exact replay and widget-value preservation passed in the PR suite; a dangling endpoint probe failed because the op was applied; opposite-order insert probes failed with different projections; and a preoccupied deterministic fork-id probe failed because the existing definition was replaced. The PR suite also covers remapped link references and colliding definition ids. Malformed shape rejection remained byte-identical in the PR suite, but dangling-link atomic rejection failed as noted inline.
Could not verify dependency-cruiser/import-gate success or a completely green full suite under the mandated Node 25 environment. Mutation testing was not run because it is nightly rather than a normal CI gate.
Addresses review feedback: #183 (comment) #183 (comment) #183 (comment)
christian-byrne
left a comment
There was a problem hiding this comment.
Three blockers: groups disappear, private keys leak, and id-less definitions apply.
Full context for agent readers
Findings
- Major,
src/applier.ts:492:groupsis part of the serialized fragment contract, but the applier only reads nodes, links, and definitions. A probe inserting one group projectedgroups: undefined. Minimal fix: validate and persist groups in the same transaction with a deterministic merge policy. - Major,
src/applier.ts:575:createNodeMapandmintDefinitionretain arbitrary__*payload properties, while node and definition projection emits them. A probe projected both node and definition__secretvalues. Minimal fix: reject or recursively scrub private keys before queuing writes. - Major,
src/applier.ts:553:String(sg["id"])turns an absent id into the live key"undefined"; the op reports applied and projection emits an id-less definition. Minimal fix: require a present legal definition id before minting.
All three previously resolved threads were checked at exact head. Dangling link endpoints now reject atomically. Conflicting live definition content now rejects in either tested order. The removed fork path can no longer overwrite a preoccupied derived id.
Verification
npm ci: passed; 353 packages installed. npm reported 5 dependency vulnerabilities (2 moderate, 3 high).npm run build: passed.npm run typecheck: passed.npm run lint: passed with 0 errors and 1,705 warnings.npm test: 78 files; 77 passed, 1 failed. 999 tests; 991 passed, 8 failed. All eight failures are the dependency-cruiser test file under unsupported Node 25.9.0.npm run check:purity: passed.npm run check:pins: passed; 189 files, 19 citation sites, 6 pins.npm run check:profile-claims: passed; 60 presence and 10 absence claims.npm run check:coderabbit: passed; 5 blocks from 4 profiles.npm run check:stateless: passed; 3/3 probe tests, 18 source files linted.npm run verify:corpus: passed; 8 files.- Reviewer probes: 6 tests; 4 passed and 2 failed. Nested-definition id independence, late malformed-fragment atomicity, exact replay, and opposite-order semantic replay passed. Group insertion and private-key exclusion failed. A separate id-less-definition probe was accepted and reproduced finding 3.
Could not verify
npm run check:imports: inconclusive, exit 2, because dependency-cruiser 18.2.0 does not support mandated Node 25.9.0.npm run check:mutation-report: inconclusive, exit 2, because no mutation report exists. Full mutation testing was not run.- A fully green
npm testunder mandated Node 25 could not be obtained for the same dependency-cruiser engine restriction.
Glossary
- KA-1: invariant prohibiting direct Yjs merging of independently edited documents.
- KA-4: invariant requiring identical projection for every legal ordering of the same semantic operations.
- CRDT: conflict-free replicated data type.
christian-byrne
left a comment
There was a problem hiding this comment.
Four correctness gaps. Opposite-order replay diverges. Definition and error contracts mismatch.
Full context for agent readers
Reviewed the full diff and surrounding applier, mint, projection, stamp, and type code at the pinned head. Focused probes found four contract failures: colliding node inserts retain whichever op arrived first; conflicting definitions retain whichever definition arrived first instead of deterministic forking; nested definitions are not entered into the addressable definitions map; and dangling endpoints return malformed_op rather than the unknown-node failure path. Exact replay and recursive private-key scrubbing passed. The repository suite under required Node 25 ran 994/1002 tests; all eight failures are dependency-cruiser rejecting unsupported Node 25. Build, typecheck, lint, purity, pins, profile claims, CodeRabbit config, stateless, and corpus checks passed. check:imports and mutation-report were inconclusive.
christian-byrne
left a comment
There was a problem hiding this comment.
Four correctness gaps: remapping, convergence, definition forking, shape validation.
Full context for agent readers
Reviewed the pinned head against the deterministic and idempotent applier invariant (KA-4) and the V1.5 insert_workflow contract. Reviewer probes reproduced partial-overlap divergence and definition-conflict rejection. Exact replay, recursive private-key scrubbing, and atomic rejection passed existing or probe coverage. Node 25 causes the same eight dependency-cruiser test failures on the base branch.
Addresses review feedback: #183 (comment) #183 (comment) #183 (comment) #183 (comment)
christian-byrne
left a comment
There was a problem hiding this comment.
One operation-order convergence blocker found.
Full context for agent readers
Finding: insert_workflow becomes an applied whole-op no-op when a carried link targets a pre-existing node that has already been concurrently deleted. Reversing the legal delivery order inserts the new node first, then deletes only the pre-existing endpoint, so projections diverge. The inline comment contains the reproducer and required behavior.
Verification at 7c0e69e8bef4fdaccea3444225f8b5e98e4c1ca3 on Node 25.9.0:
pnpm install --frozen-lockfile: passed.pnpm run typecheck: passed.pnpm run build: passed.pnpm test: 996 passed, 8 failed across 78 files. All eight failures are dependency-cruiser Node 25 incompatibility; base produced the same eight failures, with 968 passed across 77 files.check:purity,check:pins,check:profile-claims,check:coderabbit,check:stateless, andverify:corpus: passed.check:imports: inconclusive because dependency-cruiser rejects Node 25.- Reviewer-only operation-order convergence probe: failed as expected.
insert_workflowthendelete_nodeleft the inserted node;delete_nodetheninsert_workflowreturnedno-opand left no inserted node. The probe used one shared encoded seed and semantic ops only, respecting the semantic-ops-only rule. Existing PR tests cover deterministic remapping, exact replay, definition conflicts, nested addressing, private-key scrubbing, and rejection atomicity.
Not verified: remote reachability of pinned SHAs; full mutation run; check:imports under a supported Node release.
christian-byrne
left a comment
There was a problem hiding this comment.
One blocker: wrong repository target.
Full context for agent readers
The implementation and focused semantic probes look correct, but this repository explicitly declares itself read-only in AGENTS.md; active development and release ownership moved to ComfyUI_frontend/packages/comfy-multi-player. Landing the op only here would not update the canonical package consumed by the product.
Verification: typecheck, lint, build, purity, pins, profile claims, CodeRabbit config, and stateless checks passed. The full suite had 997 passes and 8 failures, all from dependency-cruiser rejecting mandated Node 25; check:imports was likewise inconclusive. check:mutation-report was inconclusive because no mutation report exists. pnpm install --frozen-lockfile could not run because the repository has no pnpm lockfile, so dependencies were installed from the committed npm lockfile with npm ci.
Reviewer-only probes passed 5/5: deterministic remapping and envelope separation, opposite-order convergence, per-link dangling endpoint drops, malformed-payload atomicity, nested-definition addressing, private-key scrubbing, and exact-replay idempotence. The probe file was removed.
|
Superseded by Comfy-Org/ComfyUI_frontend#17501 (package moved into ComfyUI_frontend). |
) Stacked on define_subgraph PR #17454; merge after it. - Adds deterministic `insert_workflow` CRDT application. - [V1.5 agent subgraph technical design](https://www.notion.so/3d16d73d365081719b85ed515af98276). - Supersedes [standalone package PR 183](Comfy-Org/comfy-multi-player#183). <details><summary>Full context for agent readers</summary> - Rebase now shares `define_subgraph` definition validation, tree-wide ID uniqueness, conflict detection, digest storage, and nested encoding helpers. The package now accepts `insert_workflow` with required `nodes` and optional `links`, `groups`, and `definitions` subgraphs. The shared applier owns deterministic ID remapping from the immutable envelope ID plus each original ID; CLI and cloud producers emit and relay the raw payload. It drops each link whose remapped endpoint is missing without rejecting the remaining insertion, makes exact-envelope replay idempotent, rejects malformed payloads atomically, recursively removes private `__*` keys from projected content, and converges independently of legal arrival order. Semantic ops remain the replication unit; direct Yjs document merging is not used. Decision 9 redirects delivery from the retired standalone repository into `packages/comfy-multi-player`. The accepted choices are package-owned remapping, required nodes with optional remaining workflow sections, and partial per-link endpoint dropping. This change preserves the package invariants for semantic-op replication (KA-1), embedded ordering identity (KA-2), portability (KA-3), deterministic idempotency (KA-4), collision-free IDs (KA-5), no independently merged Yjs updates (FC-1), no runtime coupling (FC-3), and no full-document mutation primitive (FC-4). Tests cover complete insertion; exact replay; colliding live and concurrent IDs; top-level and nested definitions; opaque widgets; optional sections; groups; recursive private-key scrubbing; both-order convergence; partial dangling-link dropping; interior writes after insertion; malformed payload rejection with byte identity; definition conflicts; legacy fork-shaped IDs; public remapping; batch policy; full-op permutations; and type-negative states. The failing-first spot check produced 22 failures and 2 passes after temporarily removing the applier implementation. Gates: package build passed; 73 test files and 949 tests passed; purity passed; import graph passed with 18 modules, 53 dependencies, and 0 violations; pins passed with 178 files, 19 citation sites, and 6 pins; profile claims passed with 60 presence and 10 absence claims; CodeRabbit config passed with 5 blocks from 4 profiles; corpus verification passed for 8 files. Root typecheck passed. Root lint passed with 0 errors and 189 pre-existing warnings. `git diff --check` passed. Linear: [FE-2033](https://linear.app/comfyorg/issue/FE-2033) </details>
Draft. Adds
insert_workflowop: one transaction merges template nodes, links, anddefinitions.subgraphsinto an existing doc. Gates green (997 tests). Contract fixed here first; cloud and comfy-cli consumers follow.Design: Notion TDD — agent subgraph lifecycle, V1.5 row and the accepted decision record on template insertion. Linear: agent editing subgraph — human tech review needed, frontend agent subgraph follower. Local decision record: insert_workflow op decision record.
Full context for agent readers
Why
Today the agent
get_templatetool resets the whole document to load a template, which destroys existing subgraph definitions and breaks the frontend follower lifecycle (see the Linear ticket on agent editing subgraphs). The TDD V1.5 decision (the TDD's open question on template insertion, decided in the accepted TDD decision record on 2026-09-08) is a new cmp op that inserts a template into an existing document instead of replacing it.Contract
insert_workflowrequiresnodes;links,groups, anddefinitionsare optional and default to empty, applied in one transaction with a frozen stamped LWW envelope.op_idgate; not batchable.malformed_op,invalid_node_payload,node_id_collision,link_id_collision,catalog_required.Invariants touched
The known-answer invariants one through five and the follower-contract invariants one, three, and four in the invariants document were updated;
test/ka4-rejection-byte-identity.test.tsandtest/permutation/full-op-pool.permutation.test.tswere extended to cover the new op.Verification (Node v24.21.0)
npm test: 78 files, 997 tests pass (newtest/insert-workflow.test.ts).npm run typecheck,npm run lint: pass.check:imports: 19 modules, 54 dependencies, 0 violations (scripts/check-import-graph.mjsupdated forsrc/remap.ts).check:purity,check:pins,check:profile-claims,check:coderabbit,check:stateless,verify:corpus: pass.git diff --check: clean.Decisions to veto
malformed_oprather than a link-specific code.docs/api-contract-proposal.mdis left unchanged; the local decision record amends the vocabulary. The comfy-cli side lands in its own draft PR.Ship order
cmp (this PR) → cloud
services/agentget_templatemintsinsert_workflow→ comfy-cliworkflow insert-workflow→ frontend follower. Consumer PRs: cloud agent tool draft PR, comfy-cli draft PR. Cloud dochost pins@comfyorg/comfy-multi-player0.2.1 and needs a bump to the first published version that includes this op.