refactor(indexing): parse Markdown front matter with SnakeYAML - #206
Closed
adityamparikh wants to merge 2 commits into
Closed
adityamparikh wants to merge 2 commits into
adityamparikh wants to merge 2 commits into
Conversation
The front matter is YAML, but its values came from the commonmark extension's line-by-line reader plus a hand-rolled splitter for flow sequences, so a quoted title containing a comma or a colon, a quoted list element, or a nested mapping came out wrong. The block is now sliced out verbatim via commonmark's source spans and handed to SnakeYAML, which is already on the classpath through Spring Boot. Implicit type resolution is switched off so every scalar stays the text as written, exactly as before; Solr's schema guessing types it. The flow-sequence splitter is gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiUHyyXLTo9ATdgg8eRFZJ Signed-off-by: Aditya Parikh <aditya.m.parikh@gmail.com>
adityamparikh
added a commit
to adityamparikh/solr-mcp
that referenced
this pull request
Sep 14, 2026
Two titles contained ': ' unquoted, which a YAML parser rejects and only the line-by-line reader tolerated. The files are now emitted by a YAML library, so they hold under apache#206 and under the current reader alike. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CiUHyyXLTo9ATdgg8eRFZJ Signed-off-by: Aditya Parikh <aditya.m.parikh@gmail.com>
Port shows-markdown/ (61 show records) and ShowsSampleDataTest from PR apache#201. Validates that SnakeYAML front matter parsing produces documents matching shows.json across all 61 records. Signed-off-by: Aditya Parikh <aditya.m.parikh@gmail.com> Co-authored-by: Junie <junie@jetbrains.com>
This was referenced Sep 14, 2026
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Markdown front matter is YAML, but its values did not come from a YAML parser: the commonmark front-matter extension reads the block line by line, and
MarkdownDocumentCreatoradded a hand-rolled splitter for flow sequences ([a, b]). A quoted title containing a comma or a colon, a quoted list element, or a nested mapping came out wrong or was lost.The block is now sliced out verbatim using commonmark's source spans and handed to SnakeYAML, which Spring Boot already ships. Implicit type resolution is switched off, so every scalar stays the text as written (
2026-01-01is not turned into a Date,8.4not into a Double), exactly as before; Solr's schema guessing types it. Sequences become multi-valued fields, nested mappings flatten with underscores (ratings_imdb), nulls are skipped. The flow-sequence splitter is deleted.What changed
MarkdownDocumentCreator: parser built withincludeSourceSpans(BLOCKS);addFrontMatterFieldsloads the block with SnakeYAML'sSafeConstructorand a resolver with no implicit tags; ~30 lines of custom parsing removed, ~25 lines of library wiring added.MarkdownDocumentCreatorTest(plain JUnit): quoted scalars with delimiters, flow and block sequences, nested mapping, typed-looking scalars kept as text, invalid YAML reported, no-front-matter document unchanged.MarkdownIndexingTest(existing, Spring) passes unchanged.Independent of #202, #203 and #205; any merge order.
Verification
./gradlew buildon Java 25: green../gradlew nativeTest -Pnativeon GraalVM CE 25.0.2: green (SnakeYAML'sSafeConstructorbuilds only JDK collections and strings).🤖 Generated with Claude Code
https://claude.ai/code/session_01CiUHyyXLTo9ATdgg8eRFZJ