Symptom
On Fern-hosted OpenAPI reference docs, markdown-content-parity reports 45–70% content "missing" from markdown on pages where manual inspection shows the markdown is complete. Observed at scale on docs.nvidia.com/nemo-platform: 13–18 of 50 sampled pages flagged (avg 15–18% missing), almost all OpenAPI endpoint pages. One page reported 100% missing (a tutorials landing page) whose markdown is also complete.
Example page: https://docs.nvidia.com/nemo-platform/documentation/reference/api-reference/virtual-models/create-virtual-model (reported 56% missing; markdown is complete).
Root cause (two comparison artifacts)
-
Schema-row serialization. The HTML renders schema fields as adjacent inline elements that extract as run-together text: workspacestringRequired, default_model_entitystringOptional, autoprovisionedbooleanOptionalDefaults to false. The markdown carries the same information as structured list items: - \workspace` (string, required)`. Line/segment matching can never align these, so every schema row counts as missing.
-
Typographic quote normalization. The HTML uses curly quotes (“workspace/name”) where the markdown has straight quotes ("workspace/name"), which breaks matching for otherwise-identical prose lines.
Suggested fixes
- Normalize typographic characters (curly quotes, en/em dashes, ellipsis, NBSP) on both sides before comparison.
- For lines that fail exact matching, fall back to token-level containment (a line whose word tokens all appear in the markdown, in order or as a set, is not missing) — this absorbs the run-together schema-row rendering without loosening detection of genuinely absent prose.
- Consider treating breadcrumb/toolbar strings (
Ask a question|Copy page|View as Markdown) as chrome even when they appear inside <article>.
Impact
Fern is a major docs platform (and afdocs powers Fern's Agent Score), so this false positive dings a whole class of well-behaved sites — the parity check was the largest deduction on an otherwise 95+ property.
Symptom
On Fern-hosted OpenAPI reference docs,
markdown-content-parityreports 45–70% content "missing" from markdown on pages where manual inspection shows the markdown is complete. Observed at scale on docs.nvidia.com/nemo-platform: 13–18 of 50 sampled pages flagged (avg 15–18% missing), almost all OpenAPI endpoint pages. One page reported 100% missing (a tutorials landing page) whose markdown is also complete.Example page:
https://docs.nvidia.com/nemo-platform/documentation/reference/api-reference/virtual-models/create-virtual-model(reported 56% missing; markdown is complete).Root cause (two comparison artifacts)
Schema-row serialization. The HTML renders schema fields as adjacent inline elements that extract as run-together text:
workspacestringRequired,default_model_entitystringOptional,autoprovisionedbooleanOptionalDefaults to false. The markdown carries the same information as structured list items:- \workspace` (string, required)`. Line/segment matching can never align these, so every schema row counts as missing.Typographic quote normalization. The HTML uses curly quotes (“workspace/name”) where the markdown has straight quotes ("workspace/name"), which breaks matching for otherwise-identical prose lines.
Suggested fixes
Ask a question|Copy page|View as Markdown) as chrome even when they appear inside<article>.Impact
Fern is a major docs platform (and afdocs powers Fern's Agent Score), so this false positive dings a whole class of well-behaved sites — the parity check was the largest deduction on an otherwise 95+ property.