docs(file formats): json_pointer routes a jsonl dataset through Spice's JSON reader

[claudespice]

Summary

The File Formats reference states, flatly:

Setting file_format: jsonl uses the DataFusion JSON Lines reader directly, without json_format, flatten_json, or json_pointer support.

The json_pointer half is wrong, and has been since v2.0.0. The JSONL dispatch arm branches on whether a pointer is configured:

(Some("jsonl" | "ndjson" | "ldjson"), _) | (None, Some("jsonl" | "ndjson" | "ldjson")) => {
    // If json_pointer or json_path is set, route through SpiceJsonFormat
    // so the pointer extraction is applied (DataFusion's JsonFormat doesn't
    // support json_pointer).
    let has_pointer = /* json_pointer or json_path present and non-empty */;
    if has_pointer {
        Ok((Some(self.get_json_format(dataset, params, Format::Auto, …)?), …))
    } else {
        Ok((Some(self.get_jsonl_format(dataset, params, …)?), …))
    }
}

With a non-empty json_pointer (or its json_path alias) the dataset is read by SpiceJsonFormat, not DataFusion's JsonFormat — so the pointer is applied, and get_json_format also reads json_format and flatten_json on that path. All three of the parameters the sentence rules out are live.

There is a second, quieter consequence the sentence hid: that branch passes Format::Auto as the default parsing mode, so the extracted sub-value is content-detected (array / single object / line-delimited) rather than being read as JSON Lines. A reader who set file_format: jsonl for a line-delimited sub-document gets auto unless they also set json_format: jsonl, which the page gave them no reason to expect.

What changed

One sentence, rewritten to keep the two branches distinct: the plain JSONL path (where json_format and flatten_json really are ignored), the pointer path (where all three take effect), and the auto starting mode on that path.

Version scope

The branch landed in spiceai/spiceai#9743, first tagged v2.0.0 — so version-2.0.x, version-2.1.x, version-2.2.x and vNext all carry the wrong sentence and are all fixed here. version-1.11.x and older say "without json_format or flatten_json support" — no json_pointer claim, correct for those releases — and are deliberately untouched.

Source refs

• crates/data-connector-api/src/listing/connector.rs — the jsonl | ndjson | ldjson arm and get_json_format / get_jsonl_format (origin/trunk); same arm at crates/runtime/src/dataconnector/listing/connector.rs in v2.0.1 / v2.1.5
• JSON file format improvements spiceai#9743 — JSON file format improvements (commit 32772f7521, first tag v2.0.0)

Test plan

• cd website && npm run build passes
• Versioned-docs propagation checked — grep -rn "uses the DataFusion JSON Lines reader directly" docs/ versioned_docs/ returns 5 hits: the 4 corrected here plus version-1.11.x, which is correct as written
• Files updated: 4 (matches the diff)

[github-actions]

✅ Pull with Spice Passed

Passing checks:

• ✅ Title meets minimum length requirement (10 characters)
• ✅ Has at least one of the required labels: area/blog, area/docs, area/cookbook, dependencies
• ✅ No banned labels detected
• ✅ Has at least one assignee: claudespice

[github-actions]

🚀 deployed to https://8d94a93e.spiceai-org-website.pages.dev

[claudespice]

@copilot review

[Copilot]

@copilot review

Reviewed the current branch state for this PR comment. No additional changes are needed from this review request.

[Copilot]

🟡 Changes recommended

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Corrects JSONL reader behavior documentation across supported releases.

Changes:

• Documents pointer-based routing through Spice’s JSON reader.
• Clarifies auto, json_format, and flatten_json behavior.

File summaries

File	Description
website/docs/reference/file_format.md	Updates current reference.
website/versioned_docs/version-2.0.x/reference/file_format.md	Updates 2.0 reference.
website/versioned_docs/version-2.1.x/reference/file_format.md	Updates 2.1 reference.
website/versioned_docs/version-2.2.x/reference/file_format.md	Updates 2.2 reference.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 4
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

🚀 deployed to https://c0a9c23f.spiceai-org-website.pages.dev