fix: escape makeindex control characters in plain index entries

[eastagiletracker]

This PR proposes escaping makeindex control characters in plain index entries, so terms like Snap! program and SciSnap! stop being split into a main entry and a stray sub-entry in the printed index. We include this PR work along with a full history of your repo at https://eastagiletracker.com/projects/209. You can sign in with your GitHub ID to claim ownership of the project.

What's wrong

rewriteIndexEntry() in _support/plugins/latex-shims.mjs returns an index entry untouched whenever it contains neither a code span nor a non-ASCII character:

if (!hasCode && !hasNonAscii) return value;

That early return skips quoteForMakeindex(), so for that whole class of terms a literal ! reaches \index{...} unquoted — and makeindex reads it as the sub-entry separator, exactly as docs/STYLEGUIDE.md and the comment above quoteForMakeindex() both describe. The escaping ends up coupled to an unrelated condition: the same character is quoted in an entry that happens to contain backticks and left bare in the one next to it.

Reproduced on main at eecf1d9:

$ npx myst build --tex
$ grep -rhoE '\\index\{[^{}]*(\{[^{}]*\}[^{}]*)*\}' output/ | grep -E 'Snap!|block!\}' | sort -u
\index{SciSnap! library}
\index{SciSnap!}
\index{Snap! program}
\index{block!!}
\index{source files for Snap!}
\index{starting Snap!}
\index{Snap"! website option@\texttt{Snap"! website} option}   <- correctly quoted, only because it has backticks

So in the shipped index, Snap! program files under Snap with a "program" sub-entry, and SciSnap!, starting Snap! and source files for Snap! each hang an empty sub-entry off a truncated term. In a manual whose index is a published artifact, the language's own name is the term most likely to be looked up.

This is also what the note at 01-blocks-scripts-and-sprites.md:1928 is about — <!-- I cannot figure out an index format that compiles this... -->. The directive under it is :::{index} `!` block, block; !: the first half has backticks so it takes the rewrite path and compiles fine, while block; ! produces \index{block!!}. I left that comment in place rather than assume it should go; happy to drop it here or in a follow-up.

The change

Quote on the early-return path too, so the escaping the function's own comment describes applies to every entry:

if (!hasCode && !hasNonAscii) return quoteForMakeindex(value);

Sub-entry separators are untouched, because myst-to-tex joins entry and subEntry with ! after this transform runs — :::{index} Snap; logo menu still emits \index{Snap!logo menu}, and the diff below confirms it.

Verification

Same build, after the change — the generated LaTeX differs in exactly six entries out of 880, and in nothing else:

$ diff <baseline index dump> <after index dump>
< \index{SciSnap! library}          > \index{SciSnap"! library}
< \index{SciSnap!}                  > \index{SciSnap"!}
< \index{Snap! program}             > \index{Snap"! program}
< \index{block!!}                   > \index{block!"!}
< \index{source files for Snap!}    > \index{source files for Snap"!}
< \index{starting Snap!}            > \index{starting Snap"!}

npx myst build --tex and myst build --html were run before and after: the error set is byte-identical (28 lines both times — the pre-existing imagemagick/SVG and mystRoleError notices), and no new failures appear. The HTML build is unaffected by construction, since the transform returns early on non-LaTeX builds.

The PR also adds _support/plugins/latex-shims.test.mjs with 8 unit tests for the transform, wired to npm test through Node's built-in test runner — no new dependencies. Two of them fail on main and pass here; I confirmed that by reverting only the behavioral line and re-running:

$ npm test        # with the fix reverted
✖ plain entries with a literal ! are quoted, not split
✖ a bare ! sub-entry survives as a literal
ℹ pass 6   ℹ fail 2

$ npm test        # with the fix
ℹ pass 8   ℹ fail 0

They also cover the cases that must not change — palette, hat block, Ball, Michael — plus @, | and ", trailing backslashes, code spans, the ⚡/➔ sort keys, and non-string input.

How this was managed

This work was tracked as a story on the board, on a project imported from this repository's own issues and pull requests — 53 stories in all — and used to manage the change end to end.

If you'd rather not receive contributions like this, reply no-more-prs on this pull request and we won't open any further ones on your repositories.

---

Lawrence W. Sinclair
CEO / East Agile
linkedin.com/in/lwsinclair/
eastagile.com