Skip to content

feat(run-detail): Add score summary and AI notes - #283

Open
kaapi-agent-bot[bot] wants to merge 1 commit into
mainfrom
agent/ui-fix-271
Open

kaapi-agent-bot[bot] wants to merge 1 commit into
mainfrom
agent/ui-fix-271

Conversation

@kaapi-agent-bot

@kaapi-agent-bot kaapi-agent-bot Bot commented Sep 18, 2026

Copy link
Copy Markdown

Issue

Closes #271

Summary

  • Added overall score summary and AI diagnostic note rendering on the run detail page.
    • OverallScoreCard
      • Displays overall_score, verdict badge, and per-metric breakdown.
    • AiSummaryNote
      • Shown above the "Detailed Results" table when ai_summary is present.
  • Introduced new types OverallScore and ScoreBreakdownItem in NewScoreObjectV2.
  • Added helper functions: getOverallScore, getAiSummary, and getVerdictColor.

Checklist

Before submitting a pull request, please ensure that you mark these task.

  • Ran npm run dev and npm run build in the repository root and test.
  • If you've fixed a bug or added code that is tested
Original PR description

Closes #271

What changed

v2 evaluation runs carry score.overall (a weighted rollup) and an ai_summary diagnostic string, but neither was rendered on the run detail page. This adds:

  • OverallScoreCard — new card rendered at the top of the run detail page (above Metrics Overview) showing overall_score, a verdict badge, and the per-metric breakdown (name, score, weight, delta, verdict) as a grid of chips.
  • AiSummaryNote — a diagnostic note rendered above the per-row "Detailed Results" table when ai_summary is present.
  • New types OverallScore / ScoreBreakdownItem on NewScoreObjectV2, plus overall?: OverallScore | null and ai_summary?: string | null fields.
  • Helpers getOverallScore, getAiSummary, and getVerdictColor in app/lib/utils/evaluation.ts, reusing the existing status-success / status-warning / status-error / status-default design tokens.

Graceful degradation

  • overall absent (v1 runs, or before the backend populates it): OverallScoreCard simply isn't rendered — no error, no empty card.
  • ai_summary is null/absent: the diagnostic note isn't rendered.
  • Both are additive fields on the existing v2 score object, so v1 runs and older score shapes are unaffected.

Assumptions

Since this run can't reach the backend to confirm the exact v2 response contract, I inferred the shape from the issue text and existing `NewScoreObjectV2ig fields:

  • score.overall is shaped as { overall_score: number; verdict: string; breakdown: { name: string; score: number; weight: number; delta?: number | null; verdict: string }[] }. The name field on each breakdown item (to label it, e.g. against Correctness/Cosine/etc.) isn't explicitly listed in the issue but is needed to render each metric — please confirm the actual field name if it differs (e.g. metric instead of name).
  • verdict values are free-form strings (e.g. pass/fail/warning/excellent/etc.); getVerdictColor maps a small known set to success/warning/error styling and falls back to a neutral "default" style for anything else, so unknown values still render sensibly.
  • ai_summary is a plain string (not markdown) and is rendered as-is inside the note.

Test plan

  • npm run lint passes (no new errors; only pre-existing warnings elsewhere)
  • npm run build passes
  • Manual verification against a real v2 run response once the backend contract is confirmed (not possible from this sandboxed run — no backend/network access)

…etail page

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • ready-for-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 3d2bdfba-9dd6-471c-baa1-237f3729abeb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot changed the title fix: display v2 overall score summary and AI diagnostic note feat(run-detail): Add score summary and AI notes Sep 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluation: Display overall score summary

0 participants