Introduce `model_compare()` for model comparison with support of new predictive measures - #380

Draft

florence-bockting wants to merge 63 commits into

pred_measurefrom

integrate-loo_compare

Commits on Jul 7, 2026

refactor: update loo_compare to work with loo_pred_measure
Florence Bockting
committed
89d5e9a
refactor: update pred_measure API for integration with loo_compare()
Florence Bockting
committed
7ce2d50
tests: new tests for loo_compare with loo_pred_measure
Florence Bockting
committed
7394ba3
docs: update documentation and glossary
Florence Bockting
committed
494d24f
docs: update documentation
Florence Bockting
committed
9611656
chore: update developer-notes
Florence Bockting
committed
a3019e3

Commits on Jul 8, 2026

Merge branch 'pred_measure' into integrate-loo_compare
florence-bockting
authored
3e1bf93
refactor: clean-up refactoring and adjust for renamed arg
Florence Bockting
committed
06d6281
fix: update missing argument and corresponding docs
Florence Bockting
committed
b46ff58
chore: update NEWS.md
Florence Bockting
committed
f1fcaf0

Commits on Jul 16, 2026

Merge branch 'pred_measure' into integrate-loo_compare
florence-bockting
authored
e17e46d

Commits on Aug 12, 2026

refactor: update computation of SE for loo_compare with pred_measure_loo
Florence Bockting
committed
abc74b5

Commits on Aug 25, 2026

refactor: create model_compare and workout design
Florence Bockting
committed
faa27c5

Commits on Aug 27, 2026

update model_compare() functionality
Florence Bockting
committed
3e078aa

Commits on Aug 28, 2026

Merge branch 'pred_measure' into integrate-loo_compare
florence-bockting
authored
8897e1f
refactor: split model_compare.R and drop duplicated loo_compare internals

Florence Bockting
and
claude
committed
1eb66de
fix: restore the simplify argument to print.compare.loo

Florence Bockting
and
claude
committed
166dab6
review: update the model_compare function
Florence Bockting
committed
c0c1fc0
update: deprecate loo_compare but maintain backwards comptability
Florence Bockting
committed
c594818
review: code refactoring review
Florence Bockting
committed
aa30e54

Commits on Aug 29, 2026

docs: update function documentation to roxygen2 8.0.0
Florence Bockting
committed
b1f0b32
docs: update glossary to match loo_compare and model_compare
Florence Bockting
committed
63c16a8
fix: throw deprecation warning for loo_compare only once per session.
Florence Bockting
committed
d874044
fix: remove old 'estimates_only' label
Florence Bockting
committed
9e930e3
refactor: avoid duplicate computation
Florence Bockting
committed
4fe4b0d
docs: add loo_compare and model_compare to seealso statement
Florence Bockting
committed
88890b6
refactor: add warning for disagreeing measure name in attribute and list
Florence Bockting
committed
51b3c9c
review: minor style corrections and doc adjustment
Florence Bockting
committed
81ae8eb
review: removed dead code
Florence Bockting
committed
fed3433
vignette: update tutorial on model comparison
Florence Bockting
committed
585b77c
chore: update warning messages
Florence Bockting
committed
3f63357
review: update function docs and inline small helper
Florence Bockting
committed
5307f23

Commits on Aug 31, 2026

Merge branch 'pred_measure' into integrate-loo_compare
Florence Bockting
committed
795cd4e
tests: shrink the model-comparison fixture
Florence Bockting
committed
69346f3
print: accept simplify on the pred_measure path
Florence Bockting
committed
e52ab8e
model_compare: accept named models in dots
Florence Bockting
committed
d55d9bf
internal: share the k-fold K-mismatch warning
Florence Bockting
committed
0f830e3
internal: drop model_compare_checks.psis_loo_ss_list()
Florence Bockting
committed
769b37a
internal: drop model_compare_matrix.psis_loo_ss_list()
Florence Bockting
committed
b80c946
docs: correct the Sivula et al. year in the glossary
Florence Bockting
committed
ca8da41
pred_measure: keep a renamed measure's own orientation
Florence Bockting
committed
47af38e
docs: remove draft scaffolding from the comparison article
Florence Bockting
committed
99d71a2
docs: let the comparison article fit its own models
Florence Bockting
committed
0e07f63
docs: point the custom-measure chunks at the new fits
Florence Bockting
committed
4317ce7
docs: rename the comparison article to model-comparison
Florence Bockting
committed
0c1de0c
docs: correct the article path in the model_compare links
Florence Bockting
committed
c66e1e2
internal: drop the vignette fits from the data generator
Florence Bockting
committed
5f6bcb9
docs: give the two remaining articles their real vignette title
Florence Bockting
committed
6dbd05d

Commits on Sep 1, 2026

model_compare: return the default object for subsampled loo
Florence Bockting
committed
91bfdba
model_compare: let simplify = FALSE follow measures
Florence Bockting
committed
6621d2d
model_compare: name the fold count and the test size
Florence Bockting
committed
0944a95
vignettes: update model-comparison tutorial with other pred_measure family types
Florence Bockting
committed
0140dc1

Commits on Sep 9, 2026

refactor: allow attribute measure_se_diff for custom measures
Florence Bockting
committed
67f4d12
docs: update developer-notes with review comments from Jonah
Florence Bockting
committed
e339f48

Commits on Sep 16, 2026

refactor: add user facade custom_measure() and remove custom_se_fn parameter from model_compare()
Florence Bockting
committed
4c5a1a8

Commits on Sep 19, 2026

Merge remote-tracking branch 'origin/pred_measure' into integrate-loo_compare
Florence Bockting
committed
aa7f286

Commits on Sep 21, 2026

docs: update developer-notes wrt custom_measure
Florence Bockting
committed
6971e13
fix: model_compare throws warning when kfold_pred_measure use different folds; update vignette example
Florence Bockting
committed
8815998
fix: pass moment_match loss into pred_measure
Florence Bockting
committed
eec2140
fix: use separate digits for measures in print method
Florence Bockting
committed
dcf5fde
fix: update snapshots
Florence Bockting
committed
ba29be1

Commits on Sep 25, 2026

refactor: remove the 'rank_by' argument for model_compare
Florence Bockting
committed
c5af96b
fix: check whether pothoc measures have been used and warn for measures other than elpd, mlpd, and ic
Florence Bockting
committed
9ffff9c