Skip to content

feat(evaluations): Update cost tooltip for judges - #259

Open
Ayush8923 wants to merge 3 commits into
mainfrom
agent/ui-fix-258
Open

Ayush8923 wants to merge 3 commits into
mainfrom
agent/ui-fix-258

Conversation

@Ayush8923

@Ayush8923 Ayush8923 commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Update the cost tooltip on /evaluations to include judge cost.
  • Additions include:
    • New optional judge?: EvalCostEntry field in EvalCost (app/lib/types/evaluation.ts).
    • Render a Judge scoring row in the tooltip within EvalRunCard (app/components/evaluations/EvalRunCard.tsx) when job.cost.judge exists.
    • Ensure the new row matches the backend's cost breakdown order.

Checklist

Before submitting a pull request, please ensure that you mark these task.

  • Ran npm run dev and npm run build in the repository root and test.
  • If you've fixed a bug or added code that is tested
Original PR description

Summary

The cost tooltip on /evaluations only listed Response generation cost, omitting the judge cost entry returned by the backend. This made the tooltip breakdown not add up to the displayed total_cost_usd whenever a run included judge scoring.

Changes

  • EvalCost (app/lib/types/evaluation.ts) now includes an optional judge?: EvalCostEntry field alongside response and embedding.
  • EvalRunCard (app/components/evaluations/EvalRunCard.tsx) renders a Judge scoring row in the cost tooltip when job.cost.judge is present, positioned between the existing "Response generation" and "Cosine similarity calculation" rows to match the backend's cost breakdown order.

Why

Response, judge, and embedding costs together make up total_cost_usd. Without the judge entry, users saw a total that didn't match the sum of the visible line items, causing confusion.

Closes #258

@github-actions github-actions Bot changed the title fix: include judge cost in evaluation cost tooltip feat(evaluations): Update cost tooltip for judges Sep 15, 2026
@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • ready-for-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: ebe81d0e-ebfb-4f6a-8dda-ceffe287ef3e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Ayush8923

Copy link
Copy Markdown
Collaborator Author

Not needed.

@Ayush8923 Ayush8923 closed this Sep 15, 2026
@Ayush8923 Ayush8923 reopened this Sep 15, 2026
@Ayush8923 Ayush8923 self-assigned this Sep 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluation: Include judge cost tooltip

1 participant