-
Notifications
You must be signed in to change notification settings - Fork 526
Pull requests: huggingface/lighteval
Author
Label
Projects
Milestones
Reviews
Assignee
Sort
Pull requests list
Prevent LiteLLM api_key from leaking into pushed results
#1326
opened Aug 10, 2026 by
NathanHB
Member
Loading…
Fix stale openai reference in endpoint command help text
#1324
opened Aug 9, 2026 by
RudrenduPaul
Loading…
Expose inspect-ai sample_shuffle in the eval command
#1323
opened Aug 7, 2026 by
mshimkus-positron
Loading…
fix: restore hydra-core compatibility
#1321
opened Aug 6, 2026 by
Kloudedge-apex
•
Draft
6 of 8 tasks
Pass custom_tasks through to Registry in the Inspect backend
#1320
opened Aug 2, 2026 by
MohammadSadeghSalehi
Loading…
fix(utils): preserve keys for numpy array values in flatten_dict
#1319
opened Jul 31, 2026 by
cupkk
Loading…
Restore log-likelihood HellaSwag and PIQA variants
#1315
opened Jul 27, 2026 by
kunwar-vikrant
Loading…
Fix always-true conditional that made DEFAULT_FORMAT unreachable
#1314
opened Jul 25, 2026 by
AbdullahRasheed45
Loading…
Fix litellm judge sending max_tokens as a 1-tuple
#1313
opened Jul 25, 2026 by
AbdullahRasheed45
Loading…
Add support for Likelihood Accuracy metric in HellaSwag task
#1312
opened Jul 24, 2026 by
SauravP97
Loading…
fix metric_class names in avg_at_n / avg_at_n_math / maj_at_n test cases
#1305
opened Jul 20, 2026 by
jayzuccarelli
Loading…
Fix unreachable response_format default in JudgeLM
#1303
opened Jul 17, 2026 by
oraziooztas
Loading…
Fix PEFT example and base-model config resolution for adapter models
#1301
opened Jul 17, 2026 by
oraziooztas
Loading…
fix: correct stop_sequence -> stop_sequences typo in _continuous_greedy_until
#1300
opened Jul 17, 2026 by
RudrenduPaul
Loading…
Fix asdiv prompt truncating multi-character gold answers
#1298
opened Jul 14, 2026 by
vineethsaivs
Loading…
Fix max_tokens sent as a list by the litellm judge backend
#1297
opened Jul 14, 2026 by
AbhilakshSinghReen
Loading…
fix: MixEval judge extractors use last-match (verdict-injection defense)
#1295
opened Jul 13, 2026 by
AUTHENSOR
Loading…
4 tasks done
Use ast.literal_eval instead of eval in hellaswag_arabic_pfn
#1294
opened Jul 11, 2026 by
jayzuccarelli
Loading…
Fix string-name normalization in SamplingMetric (inspect.getmembers returns a list, not a dict)
#1292
opened Jul 8, 2026 by
teddytennant
Loading…
Add a regression test for truncated-then-full sample-cache runs
#1291
opened Jul 8, 2026 by
shipbehaves
Contributor
Loading…
fix(normalizations): exclude -1 padding from token-norm length + guard mismatch (#1170)
#1290
opened Jul 8, 2026 by
WatchTree-19
Loading…
Add Armenian language evaluation suite (ArmBench-LLM)
#1289
opened Jul 8, 2026 by
bugdaryan
Loading…
Previous Next
ProTip!
Add no:assignee to see everything that’s not assigned.