Skip to content

Assessment: Include input columns in results #286

Description

@vprashrex

Is your feature request related to a problem?
The assessment results sheet currently shows only the output columns for runs with more than 100 rows, leaving out the submission's input columns. This omission prevents reviewers from understanding the basis for the scores given.

Describe the solution you'd like

  1. Frontend-only approach: Use the existing endpoint with include_signed_url=True to fetch and parse the full file from S3, eliminating the row limit without backend changes.
  2. Backend solution: Modify the backend to include full source rows in items[].input on the assessment results, allowing for direct data joining.
  3. Increase the limit: Change the endpoint validation to allow more than 100 rows, although this alters the intended design of the preview endpoint.

Option 2 is preferred for durability; option 1 offers a quick frontend fix.

Acceptance

  • Source columns for a run of 1000 rows are displayed in the results sheet and CSV download.
  • Source columns appear before output columns.
  • Pre-filter-rejected rows show source columns with their verdicts.
  • No 422 errors when accessing results for any run size.
Original issue

Problem

The assessment results sheet shows only the columns a run produced. The submission's own columns — including the ones the config never mapped — are absent, so a reviewer cannot see the input a score was given for.

PR #285 joins the submission's rows back onto the results by row_index, but it can only do so for runs of 100 rows or fewer.

Why 100

GET /api/v1/assessment/datasets/{dataset_id} validates limit_rows with ge=1, le=100:

https://github.com/ProjectTech4DevAI/kaapi-backend/blob/main/backend/app/api/routes/assessment/datasets.py

Anything above the cap returns 422. The endpoint is a preview by design — it downloads and parses the whole file from S3 on each call, which is what the cap protects.

PR #285 adds no cap of its own — it requests exactly the run's row count so that this endpoint stays the single place the ceiling lives, and raising le= needs no matching frontend change. Until that happens, a run above 100 rows 422s and silently joins nothing: no error, no source columns, just the output-only sheet we have today.

Options

  1. Signed URL, frontend-only. The same endpoint already serves include_signed_url=True. The frontend fetches the file from S3 and parses it with the xlsx dependency already in package.json. No cap, no backend change. Needs CORS on the bucket to allow browser reads, and pulls the xlsx bundle onto the results route.
  2. Backend returns the full source row in items[].input on the assessment results. Cleanest — the join happens where the data already is, and the frontend keeps doing what it does now. Backend work.
  3. Raise le=100. One line, and with no frontend cap in the way it takes effect immediately. It does turn a deliberately bounded preview endpoint into a bulk-data one, so it is a stopgap rather than the end state.

Option 2 is the most durable; option 1 unblocks the frontend without backend changes.

Acceptance

  • A run of 1000 rows shows its source columns in the results sheet and the CSV download
  • Source columns come before the run's output columns
  • Pre-filter-rejected rows still show their source columns alongside the verdict
  • No 422 in the network log when opening results for a run of any size

Notes

Column ordering is handled separately in PR #285 — output columns now follow the config's output_schema order rather than a first-seen union across rows, which is why they used to shuffle between runs.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions