create a high-level function for posterior summaries

[StefanoMezzini]

we could create a function that runs a table of posterior coefficient summary with the following columns:

• mean
• SD or variance
• p-value
• s-value
• threshold of interest
• direction relative to threshold (left/right or lower/higher)
• P(direction)
• I(direction)
• KL divergence to compare to a normal with the same mean and variance
• any other functions from the family

It could be called something like posterior_summary() and would be useful for quick table generation in reports.

[joethorley]

I think this is really cool. My preference would be for the SD as this is how I think. and then perhaps threshold before s/p-value and only s-value where its a two-sidede one. and the P(direction), I(direction) should be median side so perhaps we should give median before threshold? perhaps even do in this order

• median
• threshold
• svalue
• probability_direction
• directional_information
• mean
• sd
• normal_divergence

[StefanoMezzini]

I think that order makes sense. One can then compare median to mean, too. However, we also need a column of direction to indicate whether the probability and information are for the left or right side of the threshold.

Here is an example with a draft function I've written:

> posterior_summary(rnorm(100), side = "right")
      median    lower    upper    svalue    direction probability_direction directional_information       mean       sd normal_divergence
1 0.04683354 -2.03063 2.419619 0.9714308 estimate > 0                  0.51               0.0577155 0.09299927 1.166285       -0.02337182

A few additional notes:

• the lower and upper limits for the credible interval are currently based on the quantiles of x
• the s-value is measured in bits, as usual
• the probability of direction could also be renamed to P(direction)
• the probability of direction could be replaced by the odds ratio of direction
• we could possibly allow a few other common prior distributions for the divergence (e.g., exponential)

I'll make the branch and PR for this function and the kl_divergence() one once the other PR branches have been merged and I've heard some feedback from both @nehill197 and @aylapear about this issue.

[joethorley]

We already have coef that includes lower and upper and svalue o may be we don't need these columns here which could focus on the direction/shape of the posterior

I agree with direction but it should be be "left" or "right" as its easier for brain to process. I'm assuming we would have threshold as a column. This would give

> posterior_summary(rnorm(100), side = "right")
      median threshold   direction probability_direction directional_information       mean       sd normal_divergence
1 0.04683354         0   "right"                    0.51               0.0577155 0.09299927 1.166285       -0.02337182

[StefanoMezzini]

Yeah, dropping the columns present in mcmcr::cief() does result in a cleaner and more cohesive summary. My concern with left/right was that some people confuse left and right, but that would then require editing all the other functions, too 😅

Here's the updated output for comparison. The normal divergence is more complicated to calculate than I hoped because of low-density areas in long-tailed distributions.

      median threshold direction probability_direction directional_information      mean       sd normal_divergence
1 0.08706983         0     right                  0.53               0.1733316 0.0730277 1.011855               NaN

[joethorley]

yes looks like we need to drop the divergence.

[aylapear]

The function name looks good. I agree with the direction just being right or left. I don't have much of an opinion on the order of the columns. I think this is a great idea to simplify getting this information without having to run several functions individually.

[joethorley]

@StefanoMezzini So I think we should add this function as a generic to the universals package and then implement in the mcmcr package like we did for coef.

[joethorley]

I wonder if it shouldn't be called something like posterior_direction() and just focus on the direction stuff

• term
• median
• threshold
• direction
• probability_direction
• directional_information

[StefanoMezzini]

I agree. I think we could:

1. move mean, sd, and normal_divergence into a separate function called normal_divergence() or even a generalized divergence(), since the divergence metric isn't really related to all the directional stuff.
2. rename the summary function to something like directional_summary() to make it a bit more focused.

I opened an issue on {universals}: poissonconsulting/universals#11

[joethorley]

See the issue on universals for other suggested names.

and yes a more generic one which just does normal by default may make sense

we could call it

• summary_divergence()
• tidy_divergence()
• divergence_summary()

as possibilities

[aylapear]

Not sure if you are familiar with the broom package (https://broom.tidymodels.org). I haven't used it since my masters but it was great at making summary tables look nice. Just thought I share in case it provides any inspiration or thoughts on how we structure these tables.

[joethorley]

yes its the broom package that introduces the tidy function which I really like and which makes me like tidy_direction() and tidy_divergence() as generics as tidy() is already implemented - it gives us the same result as coef().

[StefanoMezzini]

using tidy_*() makes sense, and I like the sound of the function. We would have to add a dependency on {tibble} though

[joethorley]

Yes we could just define the generic in universals and then have actual implementations in nlist and mcmcr like we do for the tidy() function.