Move artifact and dataset reads onto the results reader

[lewisjared]

Moves two areas of the API off hand-rolled queries and onto the climate_ref results reader.

Artifact paths

Four routes joined the results root by hand, so a bad output_fragment or ExecutionOutput.filename could resolve to a path outside it. They now go through Reader.artifacts, which does the join with a containment check.

• GET /executions/{group_id}/logs
• GET /executions/{group_id}/metric_bundle
• GET /executions/{group_id}/archive
• GET /results/{result_id}

A resolve_artifact helper turns a containment failure into a 404, so the resolved path is never reported back to the caller.

Dataset listing

The list endpoint walked Dataset.__subclasses__() to find the concrete type for facet filtering. That is now DatasetFilter plus select_datasets, the query builder that also backs reader.datasets and adapter.load_catalog. Fetching by slug goes through reader.datasets.get.

Three behaviours change:

• Only the latest version of each dataset is listed, so superseded versions no longer appear as duplicates.
• Fetching by slug returns the latest version rather than failing when several versions share a slug.
• An unknown dataset_type or an unknown facet key returns a 400 instead of being silently ignored.

name_contains stays as a slug ilike applied to the built statement. DatasetFilter has no slug axis, and filtering in Python would break pagination and total_count. Worth raising upstream.

Not included

reader.executions is still blocked on three upstream gaps: no id on OutputView, no dataset_count, and Python-side pagination. Those are better filed as climate-ref issues than worked around here.

Testing

Four new tests cover the unknown dataset type, the unknown facet, facet filtering and name_contains. Full backend suite passes, along with ruff and mypy. The frontend pre-commit hooks fail on this branch and on a clean main alike, so they are untouched by this change.

Summary by CodeRabbit

• New Features

  • Dataset listings now support source-type and facet filtering, name search, pagination, retracted datasets, and latest-version results.
  • Dataset details now return the latest version for a requested slug.
  • Dataset and execution artefacts are resolved from configured locations with safe path handling.

• Bug Fixes

  • Invalid dataset filters and unsafe artefact paths now return appropriate HTTP errors.
  • Dataset metadata is now populated consistently for supported dataset types.

• Tests

  • Added coverage for dataset filtering, invalid types, and unknown facets.

[netlify]

✅ Deploy Preview for climate-ref canceled.

Name	Link
🔨 Latest commit	4844031
🔍 Latest deploy log	https://app.netlify.com/projects/climate-ref/deploys/6a7d364c95e27800081bb4c5

[coderabbitai]

Warning

Review limit reached

@lewisjared, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 101 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 098160f2-2f99-46a9-a70b-84e8db7f0255

📥 Commits

Reviewing files that changed from the base of the PR and between 4d2b4c4 and 4844031.

📒 Files selected for processing (3)

• backend/tests/test_api/test_routes/test_datasets.py
• changelog/49.fix.md
• changelog/49.improvement.md

📝 Walkthrough

Walkthrough

Dataset routes now use shared dataset queries and latest-version views. Execution and result routes resolve artifact paths through reader configuration, with invalid paths returning HTTP 404.

Changes

Dataset API

Layer / File(s)	Summary
Dataset parsing and response conversion backend/src/ref_backend/api/routes/datasets.py, backend/src/ref_backend/models/datasets.py	Dataset filters validate source types and facets. Dataset.build_from_view converts dataset views into API responses.
Dataset selection and retrieval backend/src/ref_backend/api/routes/datasets.py, backend/tests/test_api/test_routes/test_datasets.py, changelog/48.improvement.md	Listing uses shared selection queries, latest versions, retracted datasets, name filtering, pagination, and HTTP 400 validation. Slug retrieval returns the latest dataset view. Tests cover these filters and errors.

Execution artifact paths

Layer / File(s)	Summary
Contained artifact resolution backend/src/ref_backend/core/file_handling.py	resolve_artifact resolves configured paths and converts containment failures into HTTP 404 responses.
Execution and result route integration backend/src/ref_backend/api/routes/executions.py, backend/src/ref_backend/api/routes/results.py, changelog/48.fix.md	Execution logs, metric bundles, archives, and result files use reader artifact configuration and resolved paths. Existing streaming and missing-file handling remains in place.

Sequence Diagram(s)

sequenceDiagram
  participant DatasetRoute
  participant SourceAdapter
  participant select_datasets
  participant Dataset
  DatasetRoute->>SourceAdapter: select latest dataset versions
  SourceAdapter->>select_datasets: build filtered query
  select_datasets-->>DatasetRoute: paginated dataset views
  DatasetRoute->>Dataset: build_from_view
  Dataset-->>DatasetRoute: API dataset response

Loading

Possibly related PRs

• Climate-REF/ref-app#44: Related dataset facet parsing and malformed-facet validation changes.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main refactor to use the results reader for artifact and dataset reads.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch refactor/results-reader-paths-and-datasets

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 8cd21b5b-0695-4be5-8a4d-63972c3ff7fa

📥 Commits

Reviewing files that changed from the base of the PR and between b106eaf and 4d2b4c4.

📒 Files selected for processing (8)

• backend/src/ref_backend/api/routes/datasets.py
• backend/src/ref_backend/api/routes/executions.py
• backend/src/ref_backend/api/routes/results.py
• backend/src/ref_backend/core/file_handling.py
• backend/src/ref_backend/models/datasets.py
• backend/tests/test_api/test_routes/test_datasets.py
• changelog/48.fix.md
• changelog/48.improvement.md