Skip to content

Require semantic approval before compilation - #34

Merged
raghubetina merged 1 commit into
mainfrom
codex/improve-authoring-readback
Aug 13, 2026
Merged

Require semantic approval before compilation#34
raghubetina merged 1 commit into
mainfrom
codex/improve-authoring-readback

Conversation

@raghubetina

Copy link
Copy Markdown
Contributor

Summary

  • add a compact schema/import/Compiler capability matrix and correct the current Field, Reference, Association, ordering, naming, and Scaffold boundaries
  • require an exact entity-by-entity semantic read-back before a candidate can reach Compilation and Publication, while preserving the known-invalid diagnostic Compile exception
  • add paired read-back/post-approval evals and regression coverage, and prepare the backward-compatible plugin 0.1.2 source candidate

Accounts are intentionally excluded from this slice while that implementation is being finished. This PR does not change the First Draft service, CLI, Foundation Plan schema, or public marketplace catalog.

The 0.1.2 candidate is unpublished and unpromoted. Its deterministic package SHA-256 against exact CLI 0.1.0 is 8b4c0543ccb6ff056183792ba143083d4f83107cbe7860249405763071571725; npm next, npm latest, and the marketplace remain on immutable 0.1.1.

Process/documentation portion of https://github.com/firstdraft/firstdraft/issues/382.

Verification

  • sh script/check — 66/66 pass
  • exact CLI package check at d37d8b6775a0b97ce10bd651485bd308fed1dda2
  • gh skill publish --dry-run
  • Skill validator
  • blind local semantic-read-back forward test
  • independent service-truth and regression-falsification reviews
  • exact pre-PR documentation review — PASS, no findings

Make the capability boundary scannable across schema, import, and compiler layers, and require an exact entity-by-entity read-back before a publish-capable Compile. Keep known-invalid diagnostic runs available without weakening product intent.

Prepare the backward-compatible 0.1.2 source candidate while leaving the public catalog on immutable 0.1.1.
@raghubetina
raghubetina merged commit a6434c7 into main Aug 13, 2026
2 checks passed
@raghubetina

Copy link
Copy Markdown
Contributor Author

Post-merge review, hosted CI green. This is the origin PR of the semantic-approval gate that the later arc
(#37 through #41) refined, and both of its contributions hold up. The capability matrix gives the Skill an
honest statement of what the schema, importer, and Compiler each support, correcting Field, Reference,
Association, ordering, naming, and Scaffold boundaries in one place; every later "did we overclaim" review
leans on that matrix existing. The entity-by-entity read-back requirement before Compilation is the durable
control; the known-invalid diagnostic Compile exception was correctly preserved so validation feedback loops
do not require approval ceremony.

The paired evals (precompile-semantic-read-back, compile-prepared-movie-catalog) are well-constructed:
expectations assert the read-back is organized around the actual semantic model rather than restating
readiness, and the fifth expectation ("does not delete, loosen, flatten, relabel, or substitute product
meaning merely to fit current capabilities") guards the exact failure mode a capability-limited agent drifts
toward. Succession note: the surrounding ceremony changed substantially in #37 to #41; the read-back itself
and these fixtures survived every revision.

@raghubetina

Copy link
Copy Markdown
Contributor Author

Before this PR, an agent could take a staged Foundation Plan straight into Compilation, which ends in a real
GitHub repository. After it, the agent must first present a semantic read-back and get explicit approval. The
pattern deserves a name and a place in your toolkit: read-back before execute.

A read-back is not a summary of intent ("I will now compile your app"). It is the agent explaining the
artifact back from its own reading: this Plan has one Movie Entity with a required short_text Title as its
Primary Descriptor, one public read-only index, an iPhone client navigating under movies.example.com, and
Publication will create one private repository. The human approves that model, not the button press. The
difference matters because the dangerous failure is not the agent acting without permission; it is the agent
acting on a misreading with permission. Sonar operators repeat back coordinates for the same reason.

The eval fixtures teach good agent-testing craft. Each case has a prompt, should_trigger, and expectations
written as observable behaviors. Two are worth stealing. First, an expectation that the read-back be organized
around the domain model "rather than merely repeating that the application is ready" blocks the vacuous
compliance answer. Second, "does not delete, loosen, flatten, relabel, or substitute product meaning merely to
fit current capabilities" pins the subtle failure where an agent quietly downgrades what you asked for to
something it knows how to build. Naming a failure mode precisely is most of the work of testing for it.

The capability matrix added alongside is the honesty infrastructure: one table saying what schema, import, and
Compiler each actually support, so neither the agent nor the docs can claim by vibes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant