Skip to content

feat(metrics): Introduce VerdictBadge component - #284

Open
kaapi-agent-bot[bot] wants to merge 1 commit into
mainfrom
agent/ui-fix-270
Open

kaapi-agent-bot[bot] wants to merge 1 commit into
mainfrom
agent/ui-fix-270

Conversation

@kaapi-agent-bot

@kaapi-agent-bot kaapi-agent-bot Bot commented Sep 18, 2026

Copy link
Copy Markdown

Issue

Closes #270

Summary

  • Introduce VerdictBadge component to represent judge-metric scores.
    • Good: 4–5
    • Needs Refinement: 2–3
    • Needs Improvement: 0–1
  • Badge rendered in:
    • MetricsOverview (run summary scores)
    • ScoreDisplay (evaluation run list cards)
    • DetailedResultsTable and GroupedResultsTable (per-trace metric cells)
    • Avg Correctness column in CategoryMetricsTable
  • No badge for cosine-similarity scores or unscoreable entries.
  • Verdict logic handled in app/lib/utils/evaluation.ts, using existing color tokens.

Checklist

Before submitting a pull request, please ensure that you mark these task.

  • Ran npm run dev and npm run build in the repository root and test.
  • If you've fixed a bug or added code that is tested
Original PR description

Summary

Judge-metric scores were shown as raw numbers with no indication of whether they were good, mediocre, or poor, forcing readers to remember the cutoffs. This adds a VerdictBadge component that bands a numeric score into a traffic-light verdict:

  • Good: 4–5
  • Needs Refinement: 2–3
  • Needs Improvement: 0–1

The badge is rendered next to:

  • Each run-level metric card in MetricsOverview (the run's summary scores)
  • Each score chip in ScoreDisplay (used on the evaluation run list cards)
  • Each per-trace metric cell in DetailedResultsTable and GroupedResultsTable
  • The Avg Correctness column in CategoryMetricsTable

Cosine-similarity scores (detected by name match /cosine/i, and the dedicated avg_cosine column) and categorical / N/A / unscoreable entries render no badge, per the issue.

The verdict logic (getVerdictBand) and labels live in app/lib/utils/evaluation.ts, reusing the existing status-success / status-warning / status-error color tokens from globals.css rather than introducing new colors.

Assumptions

  • There's no backend verdict field on scores today (confirmed by reading app/lib/types/evaluation.ts) — the band is computed client-side purely from the numeric value using the cutoffs given in the issue.
  • Non-cosine numeric scores (TraceScore/SummaryScore with data_type: "NUMERIC") are assumed to be on the 0–5 judge-metric scale described in the issue. If a future metric uses a different numeric scale, it will need to be excluded from banding the same way cosine is.
  • There's no single backend-provided "overall run-level verdict" field, so the run-level surfaces (MetricsOverview, ScoreDisplay) get a badge per metric rather than one combined badge — this covers "run-level" per the parent issue (Evaluation 2.0: Transition to v2 API #265) until a dedicated overall-verdict field exists.
  • Cosine detection is done via a case-insensitive "cosine" substring match on the score name, since there's no structural flag distinguishing cosine from judge metrics in TraceScore/SummaryScore.

Closes #270

Add a VerdictBadge component that bands numeric judge-metric scores
into Good (4-5), Needs Refinement (2-3), and Needs Improvement (0-1),
and render it next to metric scores in the run summary, detailed and
grouped results tables, category metrics table, and run list score
chips. Cosine scores and categorical/N/A entries are excluded.

Closes #270

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • ready-for-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 80021c1d-c9a9-46ee-838b-cbde87ccb32e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot changed the title fix: add verdict badges for judge-metric scores feat(metrics): Introduce VerdictBadge component Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluation: Add verdict badges for scores

0 participants