Skip to content

Share LaTeX token and mode boundaries - #336

Open
Happypig375 wants to merge 1 commit into
masterfrom
issue-296-shared-latex-tokens
Open

Share LaTeX token and mode boundaries#336
Happypig375 wants to merge 1 commit into
masterfrom
issue-296-shared-latex-tokens

Conversation

@Happypig375

Copy link
Copy Markdown
Collaborator

Closes #296

Summary

  • add one internal UTF-16 source-span tokenizer for control words/symbols, groups, whitespace/raw text, and $/$$ boundaries
  • make both the math command dictionary and text parser consume the shared command boundary
  • use one production mode-transition contract for $, $$, \(/\), and \[/\]
  • preserve the control-sequence and whitespace behavior merged in Respect TeX control-sequence boundaries #276, including \@ as a control symbol
  • keep all public parser and painter entry points unchanged; global settings and the later parser migration remain outside this PR

Validation

  • CSharpMath.Core.AtomTests.LaTeXParserTest: 319 passed
  • CSharpMath.Rendering.Text.Tests.TextLaTeXParserTests: 252 passed
  • scoped format verification and git diff --check: passed
  • independent architecture review: approved

The focused tests cover every token kind and exact UTF-16 span, control-word termination, escaped symbols/braces, nested groups, all text/math delimiter forms, exact malformed/unclosed-mode positions, and independent recovery after a failed parse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Define shared LaTeX tokens and text/math mode transitions

1 participant