fix: parse README pin and changelog as markdown - #1482
Merged
Conversation
pinReadmeAssets matched raw markdown with string patterns (src="assets/ and ](assets/), so an assets/ ref inside a fenced code block or inline code span got rewritten even though registries render it as literal text, srcset attributes were never pinned at all, and reference-style definitions ([ref]: assets/…) were missed. The README now parses to a position-tracked GFM mdast tree (mdast-util-from-markdown was already in the catalog; the GFM pair is added so tables and footnotes parse the way GitHub renders them) and every edit lands on a parser-reported byte offset. Raw HTML arrives as mdast html nodes whose source slices go through parse5 with source locations on, so only real src/srcset attribute values are touched. No serializer round-trip — untouched bytes stay byte-identical. The exported signature and the pin/restore bracket are unchanged. The shared parse helper lives in _shared/markdown-ast.mts so the changelog flows read the same tree. Ported from the v1.x rework (6e5adcb) to this branch's publish-infra shape; needs the matching pass in the shared fleet template or the next cascade reverts it.
The changelog flows located the [Unreleased] block, the next release heading, and the merge insertion point by scanning raw lines for '## ', and counted entries with a '- ' pattern. A '## ' or '- ' line inside a fenced code block read as structure: a promote could truncate the Unreleased block at a fence line, and a bullet lookalike in fenced shell output satisfied the empty-changelog guard. Structure now comes from the same GFM mdast parse the README pin uses: level-2 headings from parser positions, entries from real listItem nodes. The exported APIs and output for well-formed changelogs are unchanged; the new tests fail on the line-scan version and pass here. Ported from the v1.x rework (6e5adcb); parseSectionBullets keeps its line loop (generated-section inputs only) and is noted for the upstream template pass.
John-David Dalton (jdalton)
force-pushed
the
test/main-ast-markdown
branch
from
August 6, 2026 04:58
227a305 to
1049dfd
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port of the v1.x AST-markdown rework (landed there as 6e5adcb) to main's fleet shape: markdown structure questions are answered from a position-tracked GFM mdast parse instead of line scans and string patterns, so content inside code fences can never read as structure.
What ported where (main's surfaces differ from v1.x):
.github/scripts/pin-readme-assets.mjs(standalone script)scripts/fleet/publish-infra/pin-readme.mtspinReadmeAssets()(pure function inside the pin/restore bracket)src/srcsetvalues, edits applied at parser-reported byte offsets, no serializer round-trip. Signature unchanged.scripts/release/changelog.mtsscripts/fleet/lib/changelog-render.mtsunreleasedRange()+scripts/fleet/lib/changelog.mtssectionHasEntries()andmergeUnreleased()'s insertion scan##boundaries from heading node positions, entries from reallistItemnodes. Exported APIs unchanged.scripts/fleet/_shared/markdown-ast.mtsparseMarkdownGfm,h2LineIndexes,hasListItem) so the pin and the changelog flows read the same tree.Behavior fixed on main by the port, each with a test:
assets/inside fenced code blocks or inline code spans stays as written;srcsetattributes are now pinned (the string version only handledsrc=/](); reference-style definitions are now pinned; a##line inside a code fence can no longer truncate a promoted[Unreleased]block or misplace the merge insertion point; a-lookalike in fenced shell output no longer satisfies the empty-changelog guard. Differential receipt: with the port reverted (tests kept), 10 of the 18 new tests fail; with the port, 18/18 pass.Dependencies:
mdast-util-gfm@3.1.0,micromark-extension-gfm@3.0.0, andparse5@8.0.1added to the pnpm catalog + root devDependencies (mdast-util-from-markdown,micromark, and@types/mdastwere already in the catalog).Cascade note:
scripts/fleet/**is managed by the shared fleet template. This lands the fix downstream to stop the bleeding; the same change needs a matching upstream template pass (includingparseSectionBullets, which keeps a line loop here since it only sees generated sections) or the next cascade reverts it.Tests: 13 in
test/fleet/pin-readme.test.mts, 5 intest/fleet/changelog-markdown-structure.test.mts.Suite:
pnpm test --alllocally — 7552 passed, 425 skipped, 8 failed. All 8 failures are cmd-optimize integration snapshot mismatches (packages/cli/test/integration/cli/cmd-optimize.test.mts,cmd-optimize-output-and-paths.test.mts) that fail identically on an untouched origin/main checkout (verified by stashing this branch's changes and rerunning both files) — pre-existing local-environment snapshot drift, not from this diff.pnpm run checkpasses on the modified scope except the pre-existingdispatch-table-is-currentfailure, which also fails on untouched origin/main.Note
Medium Risk
Touches release changelog promotion/merge and publish-time README rewriting; behavior changes are intentional but could affect edge-case CHANGELOGs or READMEs that relied on the old heuristics.
Overview
Replaces line scans and regex with a shared GFM mdast parse (
scripts/fleet/_shared/markdown-ast.mts) so markdown inside code fences is never treated as structure.Publish README pin (
pinReadmeAssets): relativeassets/…refs are rewritten at parser byte offsets (markdown image/link/definition plus parse5-locatedsrc/srcset), with no serializer round-trip. Adds coverage forsrcset, reference definitions, and skipping lookalikes in fences or inline code.Changelog flows:
[Unreleased]section bounds, merge insertion, andsectionHasEntriesuse real##headings andlistItemnodes instead ofstartsWith('## ')//^\s*-\s/patterns.Adds catalog deps
mdast-util-gfm,micromark-extension-gfm, andparse5, plus fleet tests for pin and changelog structure.Reviewed by Cursor Bugbot for commit 227a305. Configure here.