Specifying priors for Bayesian regression

General principle

morie_bayes_lm() fits a Bayesian linear regression by random-walk Metropolis MCMC. Each regression coefficient is given an independent zero-mean Normal prior whose standard deviation is the prior_sd hyperparameter. The prior encodes how large the coefficients are expected to be before seeing the data: a large prior_sd is weakly informative, a small one shrinks estimates toward zero.

Applied example

library(rmorie)
set.seed(1)
d <- data.frame(x = rnorm(100))
d$y <- 1 + 2 * d$x + rnorm(100)

# weakly informative
fit <- morie_bayes_lm(y ~ x, d, prior_sd = 10, chains = 2, iter = 1000)
print(fit)
#> <morie_bayes_fit>
#>   2 chains x 1000 iter
#>   converged: TRUE (max R-hat 1.002)
#> (Intercept)           x 
#>      0.9629      2.0090

# a per-coefficient prior vector is also accepted
fit2 <- morie_bayes_lm(y ~ x, d, prior_sd = c(5, 1), chains = 2, iter = 1000)
print(fit2)
#> <morie_bayes_fit>
#>   2 chains x 1000 iter
#>   converged: TRUE (max R-hat 1.001)
#> (Intercept)           x 
#>      0.9657      1.9896

The data were simulated with intercept 1 and slope 2. The second fit gives the slope a prior standard deviation of 1, which pulls its estimate slightly toward zero relative to the first fit (with 100 observations the likelihood dominates a prior this wide).

Tighter priors (small prior_sd) regularise the fit; inspect convergence with morie_bayes_diagnostics(fit) and the posterior with morie_bayes_plot(fit).