Skip to content

fix(evaljob): Hide cosine similarity metrics - #282

Open
kaapi-agent-bot[bot] wants to merge 1 commit into
mainfrom
agent/ui-fix-272
Open

kaapi-agent-bot[bot] wants to merge 1 commit into
mainfrom
agent/ui-fix-272

Conversation

@kaapi-agent-bot

@kaapi-agent-bot kaapi-agent-bot Bot commented Sep 18, 2026

Copy link
Copy Markdown

Issue

Closes #272

Summary

  • Hide cosine similarity metrics and Langfuse resync flow for judge runs.
  • Render changes based on is_judge_run flag on EvalJob:
    • EvalRunCard:
      • Hides "Cosine similarity calculation" cost.
      • Filters out "cosine" summary scores in ScoreDisplay.
    • MetricsOverview:
      • Excludes cosine-named scores and hides "Resync" button.
    • CategoryMetricsTable:
      • Hides "Avg Cosine" column.
    • DetailedResultsTable / GroupedResultsTable:
      • Filters out cosine-named score columns and adds a "Trace" link column when score_trace_url is present.
  • v1 rendering remains unchanged.

Checklist

Before submitting a pull request, please ensure that you mark these task.

  • Ran npm run dev and npm run build in the repository root and test.
  • If you've fixed a bug or added code that is tested
Original PR description

Summary

Judge runs (v2) never compute cosine similarity and never touch Langfuse, but the run cards, run list, and detail page were still built to render cosine metrics and a Langfuse-based trace resync flow unconditionally. This branches all of that rendering on a new is_judge_run flag on EvalJob, and preserves a v1-only "Resync" (Langfuse trace) flow while introducing a native per-item trace link for judge runs.

  • EvalRunCard: hides the "Cosine similarity calculation" cost line item and passes isJudgeRun down to ScoreDisplay, which filters any summary score named with "cosine" out of the chip row.
  • MetricsOverview (run detail page): filters cosine-named summary scores out of the metrics overview, and hides the "Resync" button (which resyncs scores via Langfuse trace lookup) for judge runs.
  • CategoryMetricsTable: hides the "Avg Cosine" column for judge runs, keeping "Avg Correctness".
  • DetailedResultsTable / GroupedResultsTable: filter out cosine-named per-item score columns for judge runs, and add a new "Trace" column that links out via the native score_trace_url field when present (this only appears when the backend supplies it, so v1 rows are unaffected).
  • v1 rendering is unchanged — all new logic is gated behind job.is_judge_run, which defaults to falsy for existing v1 jobs.

Assumptions

The backend v2 API (parent #265) hasn't landed in this repo yet — there is no is_judge_run field on evaluation jobs, no score_trace_url field on trace items, and no existing "Langfuse trace link" UI element anywhere in the codebase today (the closest analog is the "Resync" button + "No Langfuse Traces Available" modal in the run detail page, which calls get_trace_info=true&resync_score=true). Given that, I made these assumptions for the reviewer to confirm/correct once the v2 contract is finalized:

  • Added is_judge_run?: boolean to EvalJob and score_trace_url?: string to TraceItem/IndividualScore as the shape the v2 API will send.
  • Treated "Avg Cosine" (in CategoryMetricsTable) and any summary/per-item score whose name matches /cosine/i as the "cosine similarity" data to hide for judge runs, since scores are rendered generically by name rather than as a dedicated cosine field elsewhere.
  • Interpreted the existing "Resync" button (Langfuse-based trace resync) as the "Langfuse trace" feature referenced in the issue, and hid it entirely for judge runs rather than repointing it at a different endpoint.
  • Added a new "Trace" link column to DetailedResultsTable driven by score_trace_url as the "native" replacement path — this is net-new UI since no per-item trace link existed before.
  • Left EvalRunsList unchanged since it only forwards job objects to EvalRunCard and has no cosine/Langfuse rendering of its own.

Test plan

  • npm run lint passes (0 errors; pre-existing complexity warnings unrelated to this change)
  • npm run build passes
  • Manual verification against a real is_judge_run: true / score_trace_url payload once the v2 backend contract is available (not possible in this sandboxed run — no backend/network access)

Closes #272

🤖 Generated with Claude Code

Judge runs (v2) never compute cosine similarity or interact with
Langfuse, so run cards, run list, and the detail page should not show
cosine metrics or the Langfuse-based resync flow for them. A new
native score_trace_url per item is surfaced as a "View Trace" link
instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 18, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (1)
  • ready-for-review

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 4e20085f-3edd-4a33-9f14-86a483b3fd07

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot changed the title fix: Hide cosine and Langfuse resync for judge runs fix(evaljob): Hide cosine similarity metrics Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Evaluation: Fix judge run display

0 participants