-
Notifications
You must be signed in to change notification settings - Fork 553
Pull requests: huggingface/lighteval
Author
Label
Projects
Milestones
Reviews
Assignee
Sort
Pull requests list
fix: apply strip_strings/normalize in avg_at_n (preprocess was skipped)
#1385
opened Sep 10, 2026 by
Abelo9996
Loading…
fix: isolate few-shot sampler RNG and pool from shared and global state (#1307, #1309)
#1379
opened Sep 7, 2026 by
Abelo9996
Loading…
2
Pass max_tokens as an int on the litellm judge backend
#1378
opened Sep 4, 2026 by
gyanu2507
Loading…
Add unit tests for VLLMModel core methods (#724)
#1377
opened Sep 2, 2026 by
sneha4175
Loading…
3 of 4 tasks
fix(metrics): add actionable BERTScore baseline guidance
#1373
opened Aug 31, 2026 by
LarryHu0217
Loading…
fix(tasks): skip few-shot reuse warning for zero-shot tasks
#1372
opened Aug 31, 2026 by
LarryHu0217
Loading…
fix(judge): honor concurrent_requests for the OpenAI/TGI judge backend
#1371
opened Aug 30, 2026 by
mohansree14
Loading…
Key results and details output paths on the model revision
#1370
opened Aug 30, 2026 by
thisisreyy
Loading…
feat: add OrcaRouter as a first-class provider in the LiteLLM backend
#1367
opened Aug 30, 2026 by
kuswardhanietidims-svg
Loading…
fix: resolve SamplingMetric string normalize argument
#1366
opened Aug 29, 2026 by
Abelo9996
Loading…
fix: honor task-level num_samples and warn when it has no effect
#1361
opened Aug 27, 2026 by
Abelo9996
Loading…
ci: bump pr_style_bot to current style-bot-action reusable workflow
#1360
opened Aug 27, 2026 by
Abelo9996
Loading…
Fix continuation tokens after distributed gathering
#1357
opened Aug 25, 2026 by
Excelius-Wang
Loading…
fix(sglang): size batch truncation from the longest input in the batch, not the first
#1355
opened Aug 24, 2026 by
shoemoney
Loading…
docs: fix broken code examples in evaluating-a-custom-model.mdx
#1352
opened Aug 22, 2026 by
mohansree14
Loading…
fix: keep every task's responses when reloading them from details
#1350
opened Aug 19, 2026 by
tonycoder-hub
Loading…
2 tasks done
Fix relational comparisons with different condition counts
#1349
opened Aug 19, 2026 by
Excelius-Wang
Loading…
Parse hellaswag_arabic endings with ast.literal_eval
#1348
opened Aug 19, 2026 by
Nakul-Sinha
Loading…
Previous Next
ProTip!
Follow long discussions with comments:>50.