perf(indexing): soft-commit instead of hard-commit, keeping documents searchable

[adityamparikh]

Summary

Two small changes to inline indexing.

• Payload guidance. Inline payloads are generated by the model as tool arguments, so every byte costs output tokens and one oversized call is truncated at the model's output limit. The server instructions (spring.ai.mcp.server.instructions) now tell the model: send data in the format it already has; when converting (for example from a paste), emit CSV and never convert to XML; pass JSON as the documents array, not a string; split inputs larger than a few hundred KB across several calls, not further than needed because each call commits. These are directives rather than a cost comparison, because a live run showed the model ignoring the comparison and emitting escaped JSON. Instructions are sent once per session, so this is the one home for it; tool descriptions are unchanged.
• Soft commit per call. Splitting multiplies commits, so indexDocuments now ends with a soft commit (waitFlush=false, waitSearcher=true, softCommit=true) instead of a hard commit. Documents are still searchable when the tool returns; the segment fsync is left to Solr's autoCommit, which the _default configset enables at 15 s, and the transaction log covers durability in between. README notes that a configset with autoCommit disabled should enable it.

Tests

• IndexingServiceTest verifies the soft-commit overload is used and the hard-commit overload is not.
• New McpServerInstructionsTest pins the splitting clause and the convert-to-CSV directive in application.properties.

Verification

./gradlew build on Java 25: green (406 tests, 0 failures, 7 skipped at the previous head; unchanged test set plus one assertion) (the OTLP suite, skipped on main until #198). Independent of #197: the only shared file is IndexingServiceTest, and neither branch stubs commit there.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CiUHyyXLTo9ATdgg8eRFZJ

[adityamparikh]

Rebased onto #205 and pushed. This PR now depends on #205 and should merge after it. The first four commits shown are #205's; GitHub cannot retarget the base to a branch on the fork, so they appear here until #205 lands.

The rebase was necessary rather than cosmetic. #205 introduces forward(), the CSV/XML path. Before the rebase this PR could not reach it, so a caller that splits a large input across several calls does not pay for a hard commit per call was only true for JSON and markdown; CSV and XML still issued setAction(COMMIT, waitFlush=true, waitSearcher=true). Neither PR is wrong alone — the gap only exists once both are merged, which is exactly why it survived review twice.

Three fixes on top:

Soft commit for CSV and XML (a4fb3d3). Measured against Solr directly, same endpoint and document, median of 10: hard commit 22.79 ms vs soft 8.26 ms. Across the MCP tools, 61 single-document CSV calls go from 1853 ms to 585 ms (per extra call 30.4 ms to 9.5 ms). JSON and markdown, which this did not touch, stayed flat — that control is what makes the number trustworthy rather than a warm-cache artifact.

The size threshold was wrong, and unreachable (205a32d). Observed live: Sonnet 5 in Claude Desktop was asked to index an 86 KB markdown file. After three minutes nothing had reached the server — the model was still emitting the payload as tool arguments, about 76,000 output tokens, past the output budget, so the call could only ever truncate and fail. The instructions said to split inputs larger than a few hundred KB; for record data one byte costs roughly one output token, so that threshold is multiples past any output budget. Now ~20 KB per call, with the byte-per-token rule stated so a caller can size its own input, and an instruction to count and batch before writing the first call. The guidance also moved into the tool descriptions: instructions is returned once at initialize and not every client puts it in the model's context, whereas tool descriptions always are. That reverses 6f15281 deliberately — happy to drop it if you would rather keep the single source.

Input format vs conversion target (4b0815f). Send data in the format you already have; when you must convert, emit CSV, and never convert to XML put those two clauses side by side, so a model holding XML could read never XML as applying to its input — and converting XML is the worst possible move, since #205 forwards it to Solr unparsed. Now stated separately, with conversion scoped to formats that have no tool of their own. Only index-markdown-documents previously carried an as-is rule; all four do now.

Build green: 392 tests, 0 failures, 7 skipped (the pre-existing @Disabled OTLP class).

[adityamparikh]

Reduced to one commit. All payload-splitting and format guidance is reverted — the instructions, the tool descriptions and the index-data prompt are untouched by this PR. What remains is the commit-mode change, verified against the condition that documents stay searchable.

Why the guidance went away. It did not work, and the MCP maintainers consider the shape of it an anti-pattern. Tested live: Sonnet 5 in Claude Desktop, an 86 KB markdown paste. With the guidance in place the model did batch correctly — 15 calls of 10 documents, no truncation — and still took over 15 minutes, because splitting does not reduce total output tokens and each extra call re-reads the whole conversation. Meanwhile discussion #1197 records the consensus as keep the data plane out of JSON-RPC arguments entirely, naming "passing entire file content through LLM context" as an anti-pattern, and the File Uploads WG charter names "prose instructions asking for base64 strings or local paths" as the problem it exists to remove. Tuning prose was treating a symptom the spec intends to delete; the real answer is the handle-based path (SEP-2356, SEP-2631), which is bigger than this PR.

Why the commit change stays. Both paths hard-committed — indexDocuments via solrClient.commit(collection), and #205's CSV/XML forward() via setAction(COMMIT, waitFlush=true, waitSearcher=true) — so every tool call waited on an fsync. Solr's own _default configset does the opposite: autoCommit at 15 s with openSearcher=false for tlog truncation, autoSoftCommit at 3 s for visibility. We were fighting the configset.

Measured, 20 interleaved reps, same endpoint and document, only the commit parameter varying:

mode	median	p90
no commit	4.05 ms	4.64 ms
soft commit	8.61 ms	10.65 ms
hard commit	18.94 ms	41.32 ms

2.2x faster, and far tighter — the hard commit's p90 is four times its median, which is fsync variance. Over the MCP tools, 61 single-document CSV calls go from 1853 ms to 585 ms.

Searchability is preserved, which was the condition: waitSearcher=true means documents are visible when the call returns, verified 30/30 with zero delay on both the JSON and CSV paths. Durability is unchanged — the transaction log is written on the add, before any commit, so documents survive a crash in every commit mode; a hard commit governs how much tlog is replayed on recovery, not whether data is lost. One caveat for the docs: an operator running a custom configset with autoCommit disabled should enable it.

Still depends on #205 and should merge after it. Build green: 390 tests, 0 failures.