Commit 6065380
fix(evaluation): force file_search so KB metric scores
Fast eval runs left tool_choice at auto. Models that decline to call
file_search — gpt-4.1 in practice — return no chunks, so the
knowledge_base metric is dropped on every row and "Adherence to
Knowledge Base" comes back blank for the whole run.
Force the tool wherever we already request file_search_call.results.
The metric now measures groundedness given retrieval instead of the
model's willingness to retrieve, so scores shift on every model.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent a2e44ab commit 6065380
1 file changed
Lines changed: 5 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
505 | 505 | | |
506 | 506 | | |
507 | 507 | | |
508 | | - | |
509 | | - | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
510 | 512 | | |
511 | 513 | | |
| 514 | + | |
512 | 515 | | |
513 | 516 | | |
514 | 517 | | |
| |||
0 commit comments