Bootstrap & Simulation

marginaleffects offers an inferences() function to compute uncertainty estimates using the bootstrap and simulation-based inference.

WARNING: The inferences() function is experimental. It may be renamed, the user interface may change, or the functionality may migrate to arguments in other marginaleffects functions.

Consider a simple model:

library(marginaleffects)

mod <- lm(Sepal.Length ~ Petal.Width * Petal.Length + factor(Species), data = iris)

We will compute uncertainty estimates around the output of comparisons(), but note that the same approach works with the predictions() and slopes() functions as well.

Delta method

The default strategy to compute standard errors and confidence intervals is the delta method. This is what we obtain by calling:

avg_comparisons(mod, by = "Species", variables = "Petal.Width")
#> 
#>         Term Contrast    Species Estimate Std. Error      z Pr(>|z|)   S  2.5 % 97.5 %
#>  Petal.Width mean(+1) setosa      -0.1103      0.285 -0.387    0.699 0.5 -0.669  0.449
#>  Petal.Width mean(+1) versicolor  -0.0201      0.160 -0.125    0.900 0.2 -0.334  0.293
#>  Petal.Width mean(+1) virginica    0.0216      0.169  0.128    0.898 0.2 -0.309  0.353
#> 
#> Columns: term, contrast, Species, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predicted 
#> Type:  response

Since this is the default method, we obtain the same results if we add the inferences() call in the chain:

avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
  inferences(method = "delta")
#> 
#>         Term Contrast    Species Estimate Std. Error      z Pr(>|z|)   S  2.5 % 97.5 %
#>  Petal.Width mean(+1) setosa      -0.1103      0.285 -0.387    0.699 0.5 -0.669  0.449
#>  Petal.Width mean(+1) versicolor  -0.0201      0.160 -0.125    0.900 0.2 -0.334  0.293
#>  Petal.Width mean(+1) virginica    0.0216      0.169  0.128    0.898 0.2 -0.309  0.353
#> 
#> Columns: term, contrast, Species, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predicted 
#> Type:  response

Bootstrap

marginaleffects supports three bootstrap frameworks in R: the well-established boot package, the newer rsample package, and the so-called “bayesian bootstrap” in fwb.

boot

avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
  inferences(method = "boot")
#> 
#>         Term Contrast    Species Estimate Std. Error  2.5 % 97.5 %
#>  Petal.Width mean(+1) setosa      -0.1103      0.272 -0.653  0.471
#>  Petal.Width mean(+1) versicolor  -0.0201      0.162 -0.345  0.299
#>  Petal.Width mean(+1) virginica    0.0216      0.181 -0.352  0.388
#> 
#> Columns: term, contrast, Species, estimate, predicted_lo, predicted_hi, predicted, std.error, conf.low, conf.high 
#> Type:  response

All unknown arguments that we feed to inferences() are pushed forward to boot::boot():

est <- avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
  inferences(method = "boot", sim = "balanced", R = 500, conf_type = "bca")
est
#> 
#>         Term Contrast    Species Estimate Std. Error  2.5 % 97.5 %
#>  Petal.Width mean(+1) setosa      -0.1103      0.290 -0.643  0.492
#>  Petal.Width mean(+1) versicolor  -0.0201      0.167 -0.352  0.324
#>  Petal.Width mean(+1) virginica    0.0216      0.183 -0.367  0.375
#> 
#> Columns: term, contrast, Species, estimate, predicted_lo, predicted_hi, predicted, std.error, conf.low, conf.high 
#> Type:  response

We can extract the original boot object from an attribute:

attr(est, "inferences")
#> 
#> BALANCED BOOTSTRAP
#> 
#> 
#> Call:
#> (function (model, INF_FUN, ...) 
#> {
#>     conf_type <- attr(model, "inferences_conf_type")
#>     checkmate::assert_choice(conf_type, choices = c("perc", "norm", 
#>         "basic", "bca"))
#>     modcall <- insight::get_call(model)
#>     modeldata <- get_modeldata(model, additional_variables = FALSE)
#>     dots <- list(...)
#>     dots[["vcov"]] <- FALSE
#>     attr(model, "inferences_method") <- NULL
#>     out <- do.call(INF_FUN, c(list(model), dots))
#>     if (is.null(dots[["conf_level"]])) {
#>         conf_level <- 0.95
#>     }
#>     else {
#>         conf_level <- dots[["conf_level"]]
#>     }
#>     bootfun <- function(data, indices) {
#>         d <- data[indices, , drop = FALSE]
#>         modcall[["data"]] <- d
#>         modboot <- eval(modcall)
#>         modboot <- eval(modboot)
#>         args <- c(list(modboot, modeldata = d), dots)
#>         out <- do.call(INF_FUN, args)$estimate
#>         return(out)
#>     }
#>     args <- list(data = modeldata, statistic = bootfun)
#>     args <- c(args, attr(model, "inferences_dots"))
#>     args <- args[unique(names(args))]
#>     B <- do.call(boot::boot, args)
#>     B$call <- match.call()
#>     pr <- utils::capture.output(print(B))
#>     pr <- pr[(grep("^Bootstrap Statistics :", pr) + 1):length(pr)]
#>     pr <- gsub("std. error", "std.error", pr)
#>     pr <- paste(pr, collapse = "\n")
#>     pr <- utils::read.table(text = pr, header = TRUE)
#>     out$std.error <- pr$std.error
#>     ci_list <- lapply(seq_along(B$t0), boot::boot.ci, boot.out = B, 
#>         conf = conf_level, type = conf_type)
#>     pos <- pmatch(conf_type, names(ci_list[[1]]))
#>     if (conf_type == "norm") {
#>         cols <- 2:3
#>     }
#>     else {
#>         cols <- 4:5
#>     }
#>     ci <- lapply(ci_list, function(x) x[[pos]])
#>     ci <- do.call("rbind", ci)[, cols]
#>     if (is.matrix(ci)) {
#>         out$conf.low <- ci[, 1]
#>         out$conf.high <- ci[, 2]
#>     }
#>     else {
#>         out$conf.low <- ci[1]
#>         out$conf.high <- ci[2]
#>     }
#>     attr(out, "inferences") <- B
#>     attr(out, "posterior_draws") <- t(B$t)
#>     return(out)
#> })(model = list(c(3.78396410453894, -0.157380650291409, 0.854286226046439, 
#> -1.46293310566858, -1.98423971100058, 0.0322348822561755), c(0.14248554202263, 
#> -0.0575144579773941, -0.171441137727629, -0.443587778227166, 
#> 0.0424855420226042, 0.204781851364483, -0.346289276464122, -0.0435877782271651, 
#> -0.557514457977398, -0.154490610917879, 0.356412221772835, -0.329661098476933, 
#> -0.168739639490674, -0.411486725209057, 1.01463218252214, 0.678217887154265, 
#> 0.551653922944046, 0.153710723535878, 0.494523716318892, 0.0673150544635492, 
#> 0.1842655812733, 0.0782178871542641, -0.0132211769783286, -0.0849600135899265, 
#> -0.587881059226235, -0.129661098476933, -0.108500130740626, 0.156412221772835, 
#> 0.242485542022603, -0.429661098476932, -0.329661098476933, 0.378217887154265, 
#> 0.145509389082121, 0.542485542022602, -0.143587778227165, 0.214632182522137, 
#> 0.628558862272369, -0.0687396394906735, -0.47144113772763, 0.0564122217728345, 
#> 0.140106392608208, -0.359893607391792, -0.47144113772763, -0.0873391630043206, 
#> -0.268654184425299, -0.146289276464121, -0.0296610984769329, 
#> -0.357514457977398, 0.256412221772835, 0.0424855420226023, 0.672051123873723, 
#> 0.253166504128602, 0.392111084356321, -0.201202445409389, 0.262902649185532, 
#> -0.449298231899122, -0.0267735353888003, -0.189170005977571, 
#> 0.361082610802931, -0.408416827162209, -0.266474227638095, 0.0239580689578126, 
#> 0.290265218210599, -0.227948876126278, 0.257274183782397, 0.641875642235248, 
#> -0.546833495871399, 0.0016131073803374, 0.0531665041286021, -0.0179162100099076, 
#> -0.516829096062078, 0.398797554590611, -0.207888915643679, -0.229124216863755, 
#> 0.429940082696772, 0.541875642235248, 0.382109617753215, 0.101088477215358, 
#> -0.146833495871398, 0.433525772361906, -0.028941750357084, 0.0562215507013825, 
#> 0.185250250939325, -0.689118350451327, -0.746833495871398, -0.145601127857536, 
#> 0.372638794242462, 0.240320925398824, -0.190821602707336, -0.201202445409389, 
#> -0.5612337914376, -0.138007370005769, 0.0959534424639401, -0.0891700059775713, 
#> -0.280440760005283, -0.182640174486829, -0.180440760005282, 0.229940082696771, 
#> 0.282853926865501, -0.090821602707335, -0.715513357931099, -0.369916919883858, 
#> 0.191096047245598, -0.325369702016127, -0.319664171555401, 0.0457104120965136, 
#> -0.723062154511679, 0.0360139881085847, -0.107831504837638, 0.090999298900214, 
#> 0.329381355194633, 0.0469765796495075, 0.259887838759361, -0.378743045749488, 
#> -0.373425544491401, 0.0415908893829797, -0.0341388006053718, 
#> 0.0476531581355767, -0.143891438994048, -0.0768461652547543, 
#> 0.170220307288723, -0.38686744669361, 0.0593717703005698, 0.31324660785916, 
#> -0.0245080569975198, 0.209706692340851, 0.304477509269917, 0.11324660785916, 
#> -0.232310109119078, 0.398084828521244, 0.31455057778297, 0.534998567468207, 
#> -0.234623578153396, 0.132889979802175, -0.516115825878858, 0.598849725194466, 
#> -0.339250516222031, -0.134138800605371, 0.104477509269917, 0.452085786637802, 
#> 0.0607494837779695, 0.727276180430108, -0.369916919883858, -0.115464983758407, 
#> -0.0350513284250357, 0.434433534906543, 0.221636330349459, 0.237505756138754, 
#> -0.251251756140585, -0.269215194962349), c(-71.5659253183152, 
#> 8.26761414419049, 3.15185596602052, -1.28216612276567, -2.35695868745888, 
#> 0.148835738547779, -0.34044390299275, -0.0297961938363073, -0.53755926731116, 
#> -0.12697985615609, 0.370203806163693, -0.322033120361454, -0.134674631629571, 
#> -0.357758958050014, 1.04691458573913, 0.664571130803259, 0.548771791848064, 
#> 0.15955609700725, 0.483050211435926, 0.0673874684834752, 0.185729953113399, 
#> 0.0645711308032583, 0.0313884387894265, -0.12230927191902, -0.598743899936895, 
#> -0.122033120361454, -0.127529199719144, 0.170203806163693, 0.26244073268884, 
#> -0.422033120361454, -0.322033120361454, 0.364571130803259, 0.17302014384391, 
#> 0.56244073268884, -0.129796193836307, 0.246914585739133, 0.654677659213986, 
#> -0.0346746316295704, -0.445322340786014, 0.0702038061636923, 
#> 0.151724725531025, -0.348275274468975, -0.445322340786014, -0.133025279076834, 
#> -0.303830191286352, -0.14044390299275, -0.0220331203614546, -0.337559267311161, 
#> 0.270203806163692, 0.0624407326888396, 0.594789558050431, 0.176856861428741, 
#> 0.311460651399506, -0.264180100742889, 0.185507808921432, -0.521608343293154, 
#> -0.106472045222185, -0.219019640205962, 0.286906008196792, -0.473871637880121, 
#> -0.302400725234302, -0.0490959810493328, 0.23914656219485, -0.305210441949569, 
#> 0.201762493297323, 0.569041609576474, -0.62314313857126, -0.052543980319319, 
#> -0.0231431385712591, -0.0703400809382657, -0.598877806997993, 
#> 0.33581989925711, -0.288539348600495, -0.303948838676953, 0.361362953726952, 
#> 0.469041609576474, 0.30337220754175, 0.0192597836278137, -0.223143138571259, 
#> 0.397599274765698, -0.0787178364254678, 0.0142181897373586, 0.128482733414449, 
#> -0.771595063245931, -0.823143138571259, -0.223910536210312, 0.294158756414123, 
#> 0.169877305216898, -0.255665749252942, -0.264180100742889, -0.629286999142677, 
#> -0.213793091440888, 0.0369287869030239, -0.119019640205963, -0.347151397762996, 
#> -0.246179106119826, -0.247151397762995, 0.161362953726952, 0.254260119676911, 
#> -0.155665749252942, -0.678723653276398, -0.360583343049196, 0.211187321696557, 
#> -0.315946525934339, -0.29775293742558, 0.0746124702030272, -0.712593384760171, 
#> 0.0460443645433155, -0.0982348429407229, 0.130610274230016, 0.339059047319985, 
#> 0.0572649359471636, 0.274944379692861, -0.369933390179568, -0.362371391203294, 
#> 0.0563808814348634, -0.0248023674311462, 0.084408364095087, -0.096461814506441, 
#> -0.0678038520186099, 0.193170207449537, -0.378925827679121, 0.0829380473128167, 
#> 0.322062583588007, -0.00693414930529032, 0.219476840052893, 0.3132067420912, 
#> 0.122062583588007, -0.215994884806214, 0.401524204301706, 0.328658051932603, 
#> 0.55596073481416, -0.216011004430172, 0.140847095474081, -0.515882047438504, 
#> 0.629959533464211, -0.31604324367809, -0.124802367431146, 0.1132067420912, 
#> 0.465883644191937, 0.0839567563219099, 0.737986218427527, -0.360583343049196, 
#> -0.0884351295431265, -0.00672543579563604, 0.447183549931194, 
#> 0.230492517452623, 0.248051484819536, -0.234421787061469, -0.260225733418376
#> ), 6, c(4.95751445797737, 4.95751445797739, 4.87144113772763, 
#> 5.04358777822716, 4.9575144579774, 5.19521814863552, 4.94628927646412, 
#> 5.04358777822716, 4.9575144579774, 5.05449061091788, 5.04358777822716, 
#> 5.12966109847693, 4.96873963949067, 4.71148672520906, 4.78536781747786, 
#> 5.02178211284574, 4.84834607705595, 4.94628927646412, 5.20547628368111, 
#> 5.03268494553645, 5.2157344187267, 5.02178211284574, 4.61322117697833, 
#> 5.18496001358993, 5.38788105922623, 5.12966109847693, 5.10850013074063, 
#> 5.04358777822716, 4.9575144579774, 5.12966109847693, 5.12966109847693, 
#> 5.02178211284574, 5.05449061091788, 4.9575144579774, 5.04358777822716, 
#> 4.78536781747786, 4.87144113772763, 4.96873963949067, 4.87144113772763, 
#> 5.04358777822716, 4.85989360739179, 4.85989360739179, 4.87144113772763, 
#> 5.08733916300432, 5.3686541844253, 4.94628927646412, 5.12966109847693, 
#> 4.9575144579774, 5.04358777822716, 4.9575144579774, 6.32794887612628, 
#> 6.1468334958714, 6.50788891564368, 5.70120244540939, 6.23709735081447, 
#> 6.14929823189912, 6.3267735353888, 5.08917000597757, 6.23891738919707, 
#> 5.60841682716221, 5.26647422763809, 5.87604193104219, 5.7097347817894, 
#> 6.32794887612628, 5.3427258162176, 6.05812435776475, 6.1468334958714, 
#> 5.79838689261966, 6.1468334958714, 5.61791621000991, 6.41682909606208, 
#> 5.70120244540939, 6.50788891564368, 6.32912421686375, 5.97005991730323, 
#> 6.05812435776475, 6.41789038224679, 6.59891152278464, 6.1468334958714, 
#> 5.26647422763809, 5.52894175035708, 5.44377844929862, 5.61474974906067, 
#> 6.68911835045133, 6.1468334958714, 6.14560112785754, 6.32736120575754, 
#> 6.05967907460118, 5.79082160270733, 5.70120244540939, 6.0612337914376, 
#> 6.23800737000577, 5.70404655753606, 5.08917000597757, 5.88044076000528, 
#> 5.88264017448683, 5.88044076000528, 5.97005991730323, 4.8171460731345, 
#> 5.79082160270733, 7.0155133579311, 6.16991691988386, 6.9089039527544, 
#> 6.62536970201613, 6.8196641715554, 7.55428958790349, 5.62306215451168, 
#> 7.26398601189142, 6.80783150483764, 7.10900070109979, 6.17061864480537, 
#> 6.35302342035049, 6.54011216124064, 6.07874304574949, 6.1734255444914, 
#> 6.35840911061702, 6.53413880060537, 7.65234684186442, 7.84389143899405, 
#> 6.07684616525475, 6.72977969271128, 5.98686744669361, 7.64062822969943, 
#> 5.98675339214084, 6.72450805699752, 6.99029330765915, 5.89552249073008, 
#> 5.98675339214084, 6.63231010911908, 6.80191517147876, 7.08544942221703, 
#> 7.36500143253179, 6.6346235781534, 6.16711002019782, 6.61611582587886, 
#> 7.10115027480553, 6.63925051622203, 6.53413880060537, 5.89552249073008, 
#> 6.4479142133622, 6.63925051622203, 6.17272381956989, 6.16991691988386, 
#> 6.91546498375841, 6.73505132842504, 6.26556646509346, 6.07836366965054, 
#> 6.26249424386125, 6.45125175614058, 6.16921519496235), c(0, 1, 
#> 2, 3, 3, 4), list(c(-12.2474487139159, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, -14.6887734908898, 
#> 9.30429649857169, 0.099297934524247, 0.099297934524247, 0.099297934524247, 
#> 0.0778024888416312, 0.0885502116829391, 0.099297934524247, 0.099297934524247, 
#> 0.110045657365555, 0.099297934524247, 0.099297934524247, 0.110045657365555, 
#> 0.110045657365555, 0.099297934524247, 0.0778024888416312, 0.0778024888416312, 
#> 0.0885502116829391, 0.0885502116829391, 0.0885502116829391, 0.099297934524247, 
#> 0.0778024888416312, 0.099297934524247, 0.0670547660003233, 0.099297934524247, 
#> 0.099297934524247, 0.0778024888416312, 0.099297934524247, 0.099297934524247, 
#> 0.099297934524247, 0.099297934524247, 0.0778024888416312, 0.110045657365555, 
#> 0.099297934524247, 0.099297934524247, 0.099297934524247, 0.099297934524247, 
#> 0.110045657365555, 0.099297934524247, 0.099297934524247, 0.0885502116829391, 
#> 0.0885502116829391, 0.099297934524247, 0.0563070431590154, 0.0778024888416312, 
#> 0.0885502116829391, 0.099297934524247, 0.099297934524247, 0.099297934524247, 
#> 0.099297934524247, -0.0296747395714478, -0.0404224624127557, 
#> -0.0404224624127557, -0.0189270167301399, -0.0404224624127557, 
#> -0.0189270167301399, -0.0511701852540636, 0.0133161517937838, 
#> -0.0189270167301399, -0.0296747395714478, 0.0133161517937838, 
#> -0.0404224624127557, 0.0133161517937838, -0.0296747395714478, 
#> -0.0189270167301399, -0.0296747395714478, -0.0404224624127557, 
#> 0.0133161517937838, -0.0404224624127557, 0.00256842895247588, 
#> -0.0726656309366794, -0.0189270167301399, -0.0404224624127557, 
#> -0.008179293888832, -0.0189270167301399, -0.0296747395714478, 
#> -0.0296747395714478, -0.0619179080953715, -0.0404224624127557, 
#> 0.0133161517937838, 0.00256842895247588, 0.0133161517937838, 
#> -0.008179293888832, -0.0511701852540636, -0.0404224624127557, 
#> -0.0511701852540636, -0.0404224624127557, -0.0189270167301399, 
#> -0.0189270167301399, -0.0189270167301399, -0.008179293888832, 
#> -0.0296747395714478, -0.008179293888832, 0.0133161517937838, 
#> -0.0189270167301399, -0.008179293888832, -0.0189270167301399, 
#> -0.0189270167301399, 0.00256842895247588, -0.0189270167301399, 
#> -0.147899690825835, -0.0834133537779873, -0.104908799460603, 
#> -0.0726656309366794, -0.115656522301911, -0.104908799460603, 
#> -0.0619179080953715, -0.0726656309366794, -0.0726656309366794, 
#> -0.147899690825835, -0.0941610766192952, -0.0834133537779873, 
#> -0.104908799460603, -0.0941610766192952, -0.137151967984527, 
#> -0.126404245143219, -0.0726656309366794, -0.115656522301911, 
#> -0.126404245143219, -0.0404224624127557, -0.126404245143219, 
#> -0.0941610766192952, -0.0941610766192952, -0.0726656309366794, 
#> -0.104908799460603, -0.0726656309366794, -0.0726656309366794, 
#> -0.0726656309366794, -0.104908799460603, -0.0511701852540636, 
#> -0.0834133537779873, -0.0941610766192952, -0.115656522301911, 
#> -0.0404224624127557, -0.0296747395714478, -0.126404245143219, 
#> -0.137151967984527, -0.0726656309366794, -0.0726656309366794, 
#> -0.104908799460603, -0.137151967984527, -0.126404245143219, -0.0834133537779873, 
#> -0.126404245143219, -0.147899690825835, -0.126404245143219, -0.0834133537779873, 
#> -0.0941610766192952, -0.126404245143219, -0.0726656309366794, 
#> -46.0259122668959, 20.7480275407855, 5.81762435765699, 0.00153822106493895, 
#> 0.0187273680177468, 0.0442240300146605, 0.0572594194454153, 0.00153822106493895, 
#> 0.0187273680177468, -0.0369938303627295, 0.00153822106493895, 
#> -0.0156509258878689, -0.0198046834099217, 0.0317627574485016, 
#> 0.0531056619233624, 0.0786023239202761, 0.112980617825892, 0.0572594194454153, 
#> 0.00569197858699196, 0.0400702724926075, -0.0328400728406766, 
#> 0.0786023239202761, 0.0874839558289779, 0.0827560814423291, -0.0672183667462922, 
#> -0.0156509258878689, 0.0614131769674683, 0.00153822106493895, 
#> 0.0187273680177468, -0.0156509258878689, -0.0156509258878689, 
#> 0.0786023239202761, -0.0369938303627295, 0.0187273680177468, 
#> 0.00153822106493895, 0.0531056619233624, 0.0359165149705545, 
#> -0.0198046834099217, 0.0359165149705545, 0.00153822106493895, 
#> 0.0744485663982231, 0.0744485663982231, 0.0359165149705545, 0.138477279822805, 
#> 0.00984573610904497, 0.0572594194454153, -0.0156509258878689, 
#> 0.0187273680177468, 0.00153822106493895, 0.0187273680177468, 
#> -0.0861298642928877, -0.0132195189596035, -0.0819761067708348, 
#> -0.00433788705090166, -0.0304086659124113, -0.0902836218149406, 
#> -0.00906576143755052, 0.000389987335747218, -0.107472768767748, 
#> 0.0513833113295747, -0.0339883065698684, 0.0383479218988198, 
#> -0.119934041333907, -0.0861298642928877, 0.0644187007603295, 
#> -0.0345624234344644, -0.0132195189596035, -0.137123188286715, 
#> -0.0132195189596035, -0.064212842953431, 0.0508091944649789, 
#> -0.00433788705090166, -0.0819761067708348, -0.163193967148225, 
#> -0.055905327909325, -0.0345624234344644, -0.103319011245695, 
#> -0.0221011508683054, -0.0132195189596035, -0.0339883065698684, 
#> -0.0470236960006232, -0.068366600475484, -0.0256807915257624, 
#> -0.0778223492487816, -0.0132195189596035, 0.0253125324680651, 
#> -0.0475978128652191, -0.0730944748621329, -0.0215270340037094, 
#> -0.00433788705090166, -0.111626526289801, -0.0689407173400799, 
#> -0.0428699384785703, 0.000389987335747218, -0.0387161809565173, 
#> -0.0772482323841859, -0.0387161809565173, -0.055905327909325, 
#> 0.0904894796218392, -0.0215270340037094, 0.114263791024965, 0.037773805034224, 
#> -0.0226752677329013, -0.0867039811574835, 0.0330459306475752, 
#> -0.142999296402556, 0.0638445838957336, -0.207028009827138, -0.121082275063099, 
#> 0.0970746440721575, 0.0763058564618926, 0.0033955111286084, 0.0460813200783299, 
#> 0.0934950034147003, 0.230434062172567, 0.157523716839283, -0.0695148342046757, 
#> -0.121656391927695, -0.117502634405642, -0.0991652537236425, 
#> 0.0887671290280513, 0.110684150367508, -0.198720494783032, 0.033620047512171, 
#> 0.0117030261727143, -0.155460568968715, 0.0508091944649789, 0.033620047512171, 
#> 0.0288921731255222, -0.198146377918436, -0.134117664493854, -0.147153053924609, 
#> 0.0674242245531908, -0.11635440067645, -0.240832186868158, 0.0200105412168203, 
#> 0.144488327408528, -0.0695148342046757, 0.0508091944649789, 0.0632704670311377, 
#> 0.144488327408528, 0.191902010744898, 0.037773805034224, 0.0543888351224357, 
#> 0.165831231883389, 0.17471286379209, 0.0549629519870318, 0.0591167095090847, 
#> 0.140334569886475, -0.000758246393444495, -4.08248290463863, 
#> 0.680689113282832, 1.8868601435992, 5.41384839674038, 0.0333924811538591, 
#> 0.0273909956668533, 0.0207108967780348, 0.0398463755754067, 0.0333924811538591, 
#> 0.052527959951231, 0.0398463755754067, 0.0463002699969543, 0.0460740655296834, 
#> 0.0267123822650405, 0.0204846923107638, 0.014483206823758, 0.00157541798066284, 
#> 0.0207108967780348, 0.0400725800426776, 0.0271647911995824, 0.0527541644185019, 
#> 0.014483206823758, 0.00757690346766862, 0.014709411291029, 0.0656619532615971, 
#> 0.0463002699969543, 0.0209371012453057, 0.0398463755754067, 0.0333924811538591, 
#> 0.0463002699969543, 0.0463002699969543, 0.014483206823758, 0.052527959951231, 
#> 0.0333924811538591, 0.0398463755754067, 0.0204846923107638, 0.0269385867323115, 
#> 0.0460740655296834, 0.0269385867323115, 0.0398463755754067, 0.0142570023564872, 
#> 0.0142570023564872, 0.0269385867323115, -0.00442606750634296, 
#> 0.0402987845099485, 0.0207108967780348, 0.0463002699969543, 0.0333924811538591, 
#> 0.0398463755754067, 0.0333924811538591, -0.0905195047068495, 
#> -0.116108877925769, -0.0902933002395786, -0.123015181281858, 
#> -0.109654983504221, -0.0907457091741204, -0.115882673458498, 
#> -0.130147689105219, -0.0842918147525728, -0.14215066007923, -0.117239900262124, 
#> -0.135470561190412, -0.0849704281543855, -0.0905195047068495, 
#> -0.148830758968049, -0.109881187971492, -0.116108877925769, -0.0785165337328379, 
#> -0.116108877925769, -0.104105906951757, -0.134791947788599, -0.123015181281858, 
#> -0.0902933002395786, -0.0651563359552009, -0.103653498017216, 
#> -0.109881187971492, -0.0840656102853019, -0.10920257456968, -0.116108877925769, 
#> -0.117239900262124, -0.110559801373305, -0.104332111419028, -0.116787491327582, 
#> -0.0900670957723077, -0.116108877925769, -0.128790462301593, 
#> -0.103201089082674, -0.097199603595668, -0.116561286860311, -0.123015181281858, 
#> -0.0845180192198437, -0.0969733991283971, -0.110333596906034, 
#> -0.130147689105219, -0.110107392438763, -0.0974258080629389, 
#> -0.110107392438763, -0.103653498017216, -0.162190956745686, -0.116561286860311, 
#> 0.03859518390109, 0.0565996403621074, 0.0828676269828397, 0.10155069684567, 
#> 0.0637321481854677, 0.128044887933673, 0.0432394425844703, 0.146727957796503, 
#> 0.114458485688765, 0.0450490783226376, 0.043918055986283, 0.0695074292052026, 
#> 0.0570520492966492, 0.0374641615647354, -0.00680828151701423, 
#> 0.0187810917019054, 0.0950968024241221, 0.121817197979396, 0.122043402446667, 
#> 0.100872083443857, 0.0445966693880958, 0.0310102671431879, 0.147180366731045, 
#> 0.0563734358948365, 0.0699598381397444, 0.12736627453186, 0.0499195414732888, 
#> 0.0563734358948365, 0.0635059437181968, 0.139821654440414, 0.121138584577584, 
#> 0.127818683466402, 0.0508243593423724, 0.107325977865405, 0.152277034348967, 
#> 0.0704122470742863, 0.0254611905907238, 0.0950968024241221, 0.0499195414732888, 
#> 0.0505981548751016, 0.0254611905907238, 0.00587330285881012, 
#> 0.0565996403621074, 0.0575044582311911, 0.0192335006364472, 0.0123271972803578, 
#> 0.0501457459405598, 0.0503719504078307, 0.025234986123453, 0.0692812247379317, 
#> -4.08248290463863, 4.44239210774059, -0.424722128220301, -3.4890454943047, 
#> 1.11564257129667, 0.0809550703643557, 0.0246957946242551, 0.0162883406803865, 
#> 0.00432538523132849, -0.00408206871254008, 0.0162883406803865, 
#> 0.0282512961294445, -0.0160450241615981, -0.0519338905087721, 
#> -0.0196005256667876, 0.0570291594662397, 0.0331032485681237, 
#> 0.0246957946242551, 0.0605846609714292, 0.0366587500733131, 0.0402142515785026, 
#> 0.0570291594662397, -0.0435264365649036, 0.101325479757282, 0.0641401624766186, 
#> 0.0282512961294445, 0.0689921149152977, 0.0162883406803865, 0.00432538523132849, 
#> 0.0282512961294445, 0.0282512961294445, 0.0570291594662397, -0.00408206871254008, 
#> 0.00432538523132849, 0.0162883406803865, -0.0196005256667876, 
#> -0.00763757021772947, -0.0160450241615981, -0.00763757021772947, 
#> 0.0162883406803865, 0.0127328391751971, 0.0127328391751971, -0.00763757021772947, 
#> 0.109732933701151, 0.104880981262472, 0.0246957946242551, 0.0282512961294445, 
#> 0.00432538523132849, 0.0162883406803865, 0.00432538523132849, 
#> 0.0688922571589243, 0.0653367556537348, 0.113188577449967, -0.0352188403774084, 
#> 0.0772997111027928, 0.0245959368678816, 0.109633075944778, -0.180070756699594, 
#> 0.0365588923169396, -0.0268113864335399, -0.156144845801478, 
#> 0.0294478893065608, -0.0963300685561882, 0.0688922571589243, 
#> -0.0830706621736405, 0.0330033908117502, 0.0653367556537348, 
#> -0.0843671131071303, 0.0653367556537348, -0.0879226146123196, 
#> 0.162336850179689, -0.0352188403774084, 0.113188577449967, 0.0281514383730711, 
#> 0.000670025969765625, 0.0330033908117502, 0.0808552126079822, 
#> 0.165892351684878, 0.0653367556537348, -0.156144845801478, -0.0998855700613777, 
#> -0.132218934903362, -0.0675522052193931, 0.157484897741009, 0.0653367556537348, 
#> 0.0857071650466615, 0.0892626665518508, 0.0126329814188237, -0.0232558849283504, 
#> -0.0352188403774084, -0.00773742797410292, 0.0569293017098662, 
#> -0.0555892497703351, -0.180070756699594, -0.0112929294792924, 
#> -0.031663338872219, -0.0112929294792924, 0.000670025969765625, 
#> -0.195589213653842, -0.0232558849283504, 0.126796326095687, -0.103092729303395, 
#> 0.0333517330749229, -0.063648361451031, 0.0417591870187913, 0.117092421218329, 
#> -0.215611280783596, 0.0200923266923751, -0.039722450552915, 0.138759281544745, 
#> -0.082722319910468, -0.0791668184052785, -0.0145000887213093, 
#> -0.0946852753595259, -0.00124068233876155, 0.0023148191664278, 
#> -0.0756113169000891, 0.149425786060314, 0.193722106351356, -0.196537322324159, 
#> 0.0501666409626599, -0.106648230808584, 0.10868496727446, -0.147389049594437, 
#> 0.00942582217680681, -0.015796539654799, -0.159352005043495, 
#> -0.147389049594437, -0.00253713327225133, -0.0804632693387682, 
#> 0.0165368251871856, 0.0727961009272863, 0.0178332761206753, -0.184574366875101, 
#> -0.145129999022738, 0.0980184627588919, 0.0585740949065285, -0.0756113169000891, 
#> -0.159352005043495, -0.0264630441703672, 0.0585740949065285, 
#> -0.0216110917316883, -0.103092729303395, 0.074092551860776, 0.0909074597485131, 
#> -0.00964813628263018, -0.115055684752453, -0.0707593644614099, 
#> 0.0142777746154859, -0.123463138696321, -70.9625343450096, 56.3907660022668, 
#> 3.08793638683961, -8.84664926411501, -4.44414200339704, 4.61722606476297, 
#> 0.0342936864802481, -0.0102831456627638, -0.0400113570729299, 
#> -0.0824223866626909, -0.0102831456627638, 0.019445065747402, 
#> -0.114316400626108, -0.209998442516358, -0.0994677798932618, 
#> 0.13399533633709, 0.0832021237297617, 0.0342936864802481, 0.116980913050993, 
#> 0.0618560953371631, 0.049173277157568, 0.13399533633709, -0.158924202713594, 
#> 0.252596184837844, 0.1086296999779, 0.019445065747402, 0.159391942640754, 
#> -0.0102831456627638, -0.0400113570729299, 0.019445065747402, 
#> 0.019445065747402, 0.13399533633709, -0.0824223866626909, -0.0400113570729299, 
#> -0.0102831456627638, -0.0994677798932618, -0.0697395684830956, 
#> -0.114316400626108, -0.0697395684830956, -0.0102831456627638, 
#> 0.00673127762333313, 0.00673127762333313, -0.0697395684830956, 
#> 0.299338819534107, 0.235581761551747, 0.0342936864802481, 0.019445065747402, 
#> -0.0400113570729299, -0.0102831456627638, -0.0400113570729299, 
#> 0.0400912555211666, 0.0397792583812566, 0.0460703712528764, -0.00407298704557971, 
#> 0.0413520365991616, 0.0254489295764534, 0.0457583741129665, -0.144868055440661, 
#> 0.0313533129008601, 0.0101826093519202, -0.120064473472343, 0.0350609237275416, 
#> -0.0580555185515471, 0.0400912555211666, -0.0276905203432064, 
#> 0.0288755132076992, 0.0397792583812566, -0.045653727567388, 0.0397792583812566, 
#> -0.0502973298137996, 0.0465008632629185, -0.00407298704557971, 
#> 0.0460703712528764, 0.0344241369293669, 0.0136401629276402, 0.0288755132076992, 
#> 0.0438298362923223, 0.0403154527430761, 0.0397792583812566, -0.120064473472343, 
#> -0.0605333182447079, -0.0952608915040245, -0.030137350091893, 
#> 0.0433862767715827, 0.0397792583812566, 0.0469444227836583, 0.0429248148170664, 
#> 0.0195445462520469, 0.00183139627882688, -0.00407298704557971, 
#> 0.0102135792963945, 0.0363526747500107, -0.0220671642142355, 
#> -0.144868055440661, 0.00773577960323348, -0.00592679245892061, 
#> 0.00773577960323348, 0.0136401629276402, -0.142421225691975, 
#> 0.00183139627882688, -0.153524218361304, 0.0622209129727484, 
#> -0.0408146856742596, 0.043427416680614, -0.0503829173625188, 
#> -0.120768945385461, 0.090433176138536, 0.00895501058768085, 0.0335781577969189, 
#> -0.173609465675908, 0.0563912620556448, 0.0480400489825519, 0.00487346273214073, 
#> 0.0656474966039941, 0.0330726583872312, 0.00739502488813044, 
#> 0.0483520461224615, -0.172673474256178, -0.244663250441498, 0.0839667384232573, 
#> -0.0556195439442767, 0.0749037311523434, -0.0917084907179436, 
#> 0.0778998227735473, -0.0179706114710594, 0.0237288989132235, 
#> 0.0828244522153946, 0.0778998227735473, -0.00654857436945939, 
#> 0.0755586953766377, -0.00868340697823562, -0.063939787072896, 
#> -0.023207238052817, 0.0855395166411624, 0.110062071414045, -0.118634112776684, 
#> -0.0565245654195321, 0.0483520461224615, 0.0828244522153946, 
#> 0.0162954998337408, -0.0565245654195321, 0.0389023093043342, 
#> 0.0622209129727484, -0.0871268283604807, -0.0932684764174937, 
#> 0.0231486670962325, 0.0693113449678469, 0.0471350275072955, -0.00835861731997131, 
#> 0.068050563889852), c(1.08164965809277, 1.09929793452425, 1.03591651497055, 
#> 1.03984637557541, 1.00432538523133, 1.18478854894442), 1:6, 1e-07, 
#>     6), 144, list("contr.treatment"), list(c("setosa", "versicolor", 
#> "virginica")), lm(formula = Sepal.Length ~ Petal.Width * Petal.Length + 
#>     factor(Species), data = iris), Sepal.Length ~ Petal.Width * 
#>     Petal.Length + factor(Species), list(c(5.1, 4.9, 4.7, 4.6, 
#> 5, 5.4, 4.6, 5, 4.4, 4.9, 5.4, 4.8, 4.8, 4.3, 5.8, 5.7, 5.4, 
#> 5.1, 5.7, 5.1, 5.4, 5.1, 4.6, 5.1, 4.8, 5, 5, 5.2, 5.2, 4.7, 
#> 4.8, 5.4, 5.2, 5.5, 4.9, 5, 5.5, 4.9, 4.4, 5.1, 5, 4.5, 4.4, 
#> 5, 5.1, 4.8, 5.1, 4.6, 5.3, 5, 7, 6.4, 6.9, 5.5, 6.5, 5.7, 6.3, 
#> 4.9, 6.6, 5.2, 5, 5.9, 6, 6.1, 5.6, 6.7, 5.6, 5.8, 6.2, 5.6, 
#> 5.9, 6.1, 6.3, 6.1, 6.4, 6.6, 6.8, 6.7, 6, 5.7, 5.5, 5.5, 5.8, 
#> 6, 5.4, 6, 6.7, 6.3, 5.6, 5.5, 5.5, 6.1, 5.8, 5, 5.6, 5.7, 5.7, 
#> 6.2, 5.1, 5.7, 6.3, 5.8, 7.1, 6.3, 6.5, 7.6, 4.9, 7.3, 6.7, 7.2, 
#> 6.5, 6.4, 6.8, 5.7, 5.8, 6.4, 6.5, 7.7, 7.7, 6, 6.9, 5.6, 7.7, 
#> 6.3, 6.7, 7.2, 6.2, 6.1, 6.4, 7.2, 7.4, 7.9, 6.4, 6.3, 6.1, 7.7, 
#> 6.3, 6.4, 6, 6.9, 6.7, 6.9, 5.8, 6.8, 6.7, 6.7, 6.3, 6.5, 6.2, 
#> 5.9), c(0.2, 0.2, 0.2, 0.2, 0.2, 0.4, 0.3, 0.2, 0.2, 0.1, 0.2, 
#> 0.2, 0.1, 0.1, 0.2, 0.4, 0.4, 0.3, 0.3, 0.3, 0.2, 0.4, 0.2, 0.5, 
#> 0.2, 0.2, 0.4, 0.2, 0.2, 0.2, 0.2, 0.4, 0.1, 0.2, 0.2, 0.2, 0.2, 
#> 0.1, 0.2, 0.2, 0.3, 0.3, 0.2, 0.6, 0.4, 0.3, 0.2, 0.2, 0.2, 0.2, 
#> 1.4, 1.5, 1.5, 1.3, 1.5, 1.3, 1.6, 1, 1.3, 1.4, 1, 1.5, 1, 1.4, 
#> 1.3, 1.4, 1.5, 1, 1.5, 1.1, 1.8, 1.3, 1.5, 1.2, 1.3, 1.4, 1.4, 
#> 1.7, 1.5, 1, 1.1, 1, 1.2, 1.6, 1.5, 1.6, 1.5, 1.3, 1.3, 1.3, 
#> 1.2, 1.4, 1.2, 1, 1.3, 1.2, 1.3, 1.3, 1.1, 1.3, 2.5, 1.9, 2.1, 
#> 1.8, 2.2, 2.1, 1.7, 1.8, 1.8, 2.5, 2, 1.9, 2.1, 2, 2.4, 2.3, 
#> 1.8, 2.2, 2.3, 1.5, 2.3, 2, 2, 1.8, 2.1, 1.8, 1.8, 1.8, 2.1, 
#> 1.6, 1.9, 2, 2.2, 1.5, 1.4, 2.3, 2.4, 1.8, 1.8, 2.1, 2.4, 2.3, 
#> 1.9, 2.3, 2.5, 2.3, 1.9, 2, 2.3, 1.8), c(1.4, 1.4, 1.3, 1.5, 
#> 1.4, 1.7, 1.4, 1.5, 1.4, 1.5, 1.5, 1.6, 1.4, 1.1, 1.2, 1.5, 1.3, 
#> 1.4, 1.7, 1.5, 1.7, 1.5, 1, 1.7, 1.9, 1.6, 1.6, 1.5, 1.4, 1.6, 
#> 1.6, 1.5, 1.5, 1.4, 1.5, 1.2, 1.3, 1.4, 1.3, 1.5, 1.3, 1.3, 1.3, 
#> 1.6, 1.9, 1.4, 1.6, 1.4, 1.5, 1.4, 4.7, 4.5, 4.9, 4, 4.6, 4.5, 
#> 4.7, 3.3, 4.6, 3.9, 3.5, 4.2, 4, 4.7, 3.6, 4.4, 4.5, 4.1, 4.5, 
#> 3.9, 4.8, 4, 4.9, 4.7, 4.3, 4.4, 4.8, 5, 4.5, 3.5, 3.8, 3.7, 
#> 3.9, 5.1, 4.5, 4.5, 4.7, 4.4, 4.1, 4, 4.4, 4.6, 4, 3.3, 4.2, 
#> 4.2, 4.2, 4.3, 3, 4.1, 6, 5.1, 5.9, 5.6, 5.8, 6.6, 4.5, 6.3, 
#> 5.8, 6.1, 5.1, 5.3, 5.5, 5, 5.1, 5.3, 5.5, 6.7, 6.9, 5, 5.7, 
#> 4.9, 6.7, 4.9, 5.7, 6, 4.8, 4.9, 5.6, 5.8, 6.1, 6.4, 5.6, 5.1, 
#> 5.6, 6.1, 5.6, 5.5, 4.8, 5.4, 5.6, 5.1, 5.1, 5.9, 5.7, 5.2, 5, 
#> 5.2, 5.4, 5.1), c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
#> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
#> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 
#> 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 
#> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 
#> 3, 3, 3, 3, 3, 3, 3, 3, 3))), INF_FUN = function (model, newdata = NULL, 
#>     variables = NULL, comparison = "difference", type = NULL, 
#>     vcov = TRUE, by = FALSE, conf_level = 0.95, transform = NULL, 
#>     cross = FALSE, wts = FALSE, hypothesis = NULL, equivalence = NULL, 
#>     p_adjust = NULL, df = Inf, eps = NULL, numderiv = "fdforward", 
#>     ...) 
#> {
#>     dots <- list(...)
#>     if ("transform_post" %in% names(dots)) {
#>         transform <- dots[["transform_post"]]
#>         insight::format_warning("The `transform_post` argument is deprecated. Use `transform` instead.")
#>     }
#>     if ("transform_pre" %in% names(dots)) {
#>         comparison <- dots[["transform_pre"]]
#>         insight::format_warning("The `transform_pre` argument is deprecated. Use `comparison` instead.")
#>     }
#>     scall <- rlang::enquo(newdata)
#>     newdata <- sanitize_newdata_call(scall, newdata, model, by = by)
#>     if (isTRUE(by)) {
#>         modeldata <- get_modeldata(model, additional_variables = FALSE, 
#>             modeldata = dots[["modeldata"]], wts = wts)
#>     }
#>     else {
#>         modeldata <- get_modeldata(model, additional_variables = by, 
#>             modeldata = dots[["modeldata"]], wts = wts)
#>     }
#>     call_attr <- c(list(name = "comparisons", model = model, 
#>         newdata = newdata, variables = variables, type = type, 
#>         vcov = vcov, by = by, conf_level = conf_level, comparison = comparison, 
#>         transform = transform, cross = cross, wts = wts, hypothesis = hypothesis, 
#>         equivalence = equivalence, p_adjust = p_adjust, df = df), 
#>         dots)
#>     if ("modeldata" %in% names(dots)) {
#>         call_attr[["modeldata"]] <- modeldata
#>     }
#>     call_attr <- do.call("call", call_attr)
#>     bycols <- NULL
#>     sanity_dots(model, ...)
#>     sanity_df(df, newdata)
#>     conf_level <- sanitize_conf_level(conf_level, ...)
#>     checkmate::assert_number(eps, lower = 1e-10, null.ok = TRUE)
#>     numderiv <- sanitize_numderiv(numderiv)
#>     sanity_equivalence_p_adjust(equivalence, p_adjust)
#>     model <- sanitize_model(model = model, newdata = newdata, 
#>         wts = wts, vcov = vcov, by = by, calling_function = "comparisons", 
#>         ...)
#>     cross <- sanitize_cross(cross, variables, model)
#>     type <- sanitize_type(model = model, type = type, calling_function = "comparisons")
#>     sanity_comparison(comparison)
#>     if (inherits(model, c("mira", "amest"))) {
#>         out <- process_imputation(model, call_attr)
#>         return(out)
#>     }
#>     comparison_label <- transform_label <- NULL
#>     if (is.function(comparison)) {
#>         comparison_label <- deparse(substitute(comparison))
#>     }
#>     if (is.function(transform)) {
#>         transform_label <- deparse(substitute(transform))
#>         transform <- sanitize_transform(transform)
#>         names(transform) <- transform_label
#>     }
#>     else {
#>         transform <- sanitize_transform(transform)
#>         transform_label <- names(transform)
#>     }
#>     marginalmeans <- isTRUE(checkmate::check_choice(newdata, 
#>         choices = "marginalmeans"))
#>     newdata <- sanitize_newdata(model = model, newdata = newdata, 
#>         modeldata = modeldata, by = by, wts = wts)
#>     sanity_by(by, newdata)
#>     newdata <- dedup_newdata(model = model, newdata = newdata, 
#>         wts = wts, by = by, cross = cross, comparison = comparison)
#>     if (isFALSE(wts) && "marginaleffects_wts_internal" %in% colnames(newdata)) {
#>         wts <- "marginaleffects_wts_internal"
#>     }
#>     variables_list <- sanitize_variables(model = model, newdata = newdata, 
#>         modeldata = modeldata, variables = variables, cross = cross, 
#>         by = by, comparison = comparison, eps = eps)
#>     if (inherits(model, "lmerMod") && (isTRUE(hush(vcov %in% 
#>         c("satterthwaite", "kenward-roger"))))) {
#>         dv <- insight::find_response(model)
#>         if (!dv %in% colnames(newdata)) {
#>             newdata[[dv]] <- mean(insight::get_response(model))
#>         }
#>         if (!isTRUE(hush(is.infinite(df)))) {
#>             insight::format_error("The `df` argument is not supported when `vcov` is \"satterthwaite\" or \"kenward-roger\".")
#>         }
#>         df <- insight::get_df(model, type = vcov, data = newdata, 
#>             df_per_observation = TRUE)
#>     }
#>     vcov_false <- isFALSE(vcov)
#>     vcov.type <- get_vcov_label(vcov)
#>     vcov <- get_vcov(model, vcov = vcov, type = type, ...)
#>     predictors <- variables_list$conditional
#>     out <- inferences_dispatch(INF_FUN = comparisons, model = model, 
#>         newdata = newdata, vcov = vcov, variables = variables, 
#>         type = type, by = by, conf_level = conf_level, cross = cross, 
#>         comparison = comparison, transform = transform, wts = wts, 
#>         hypothesis = hypothesis, eps = eps, ...)
#>     if (!is.null(out)) {
#>         return(out)
#>     }
#>     tmp <- sanitize_hypothesis(hypothesis, ...)
#>     hypothesis <- tmp$hypothesis
#>     hypothesis_null <- tmp$hypothesis_null
#>     args <- list(model = model, newdata = newdata, variables = predictors, 
#>         cross = cross, marginalmeans = marginalmeans, modeldata = modeldata)
#>     dots[["modeldata"]] <- NULL
#>     args <- c(args, dots)
#>     contrast_data <- do.call("get_contrast_data", args)
#>     args <- list(model, newdata = newdata, variables = predictors, 
#>         type = type, original = contrast_data[["original"]], 
#>         hi = contrast_data[["hi"]], lo = contrast_data[["lo"]], 
#>         wts = contrast_data[["original"]][["marginaleffects_wts_internal"]], 
#>         by = by, marginalmeans = marginalmeans, cross = cross, 
#>         hypothesis = hypothesis, modeldata = modeldata)
#>     args <- c(args, dots)
#>     mfx <- do.call("get_contrasts", args)
#>     hyp_by <- attr(mfx, "hypothesis_function_by")
#>     if (!is.null(attr(mfx, "posterior_draws"))) {
#>         draws <- attr(mfx, "posterior_draws")
#>         J <- NULL
#>     }
#>     else if (!vcov_false && isTRUE(checkmate::check_matrix(vcov))) {
#>         idx <- intersect(colnames(mfx), c("group", "term", "contrast"))
#>         idx <- mfx[, (idx), drop = FALSE]
#>         args <- list(model, vcov = vcov, type = type, FUN = get_se_delta_contrasts, 
#>             newdata = newdata, index = idx, variables = predictors, 
#>             marginalmeans = marginalmeans, hypothesis = hypothesis, 
#>             hi = contrast_data$hi, lo = contrast_data$lo, original = contrast_data$original, 
#>             by = by, eps = eps, cross = cross, numderiv = numderiv)
#>         args <- c(args, dots)
#>         se <- do.call("get_se_delta", args)
#>         J <- attr(se, "jacobian")
#>         attr(se, "jacobian") <- NULL
#>         mfx$std.error <- as.numeric(se)
#>         draws <- NULL
#>     }
#>     else {
#>         J <- draws <- NULL
#>     }
#>     if ((is.null(by) || isFALSE(by)) && "rowid" %in% colnames(mfx)) {
#>         if ("rowid" %in% colnames(newdata)) {
#>             idx <- c("rowid", "rowidcf", "term", "contrast", 
#>                 "by", setdiff(colnames(contrast_data$original), 
#>                   colnames(mfx)))
#>             idx <- intersect(idx, colnames(contrast_data$original))
#>             tmp <- contrast_data$original[, ..idx, drop = FALSE]
#>             bycols <- intersect(colnames(tmp), colnames(mfx))
#>             idx <- duplicated(tmp, by = bycols)
#>             tmp <- tmp[!idx]
#>             mfx <- merge(mfx, tmp, all.x = TRUE, by = bycols, 
#>                 sort = FALSE)
#>         }
#>         else {
#>             idx <- setdiff(colnames(contrast_data$original), 
#>                 colnames(mfx))
#>             mfx <- data.table(mfx, contrast_data$original[, ..idx])
#>         }
#>     }
#>     mfx <- get_ci(mfx, conf_level = conf_level, df = df, draws = draws, 
#>         estimate = "estimate", null_hypothesis = hypothesis_null, 
#>         p_adjust = p_adjust, model = model)
#>     mfx <- sort_columns(mfx, newdata, by)
#>     attr(mfx, "posterior_draws") <- draws
#>     mfx <- equivalence(mfx, equivalence = equivalence, df = df, 
#>         ...)
#>     mfx <- backtransform(mfx, transform)
#>     if (any(!is.na(mfx[["marginaleffects_wts_internal"]]))) {
#>         marginaleffects_wts_internal <- mfx[["marginaleffects_wts_internal"]]
#>     }
#>     else {
#>         marginaleffects_wts_internal <- NULL
#>     }
#>     mfx[["marginaleffects_wts_internal"]] <- NULL
#>     out <- mfx
#>     data.table::setDF(out)
#>     out <- set_marginaleffects_attributes(out, get_marginaleffects_attributes(newdata, 
#>         include_regex = "^newdata.*class|explicit|matrix|levels"))
#>     attr(out, "newdata") <- newdata
#>     attr(out, "call") <- call_attr
#>     attr(out, "type") <- type
#>     attr(out, "model_type") <- class(model)[1]
#>     attr(out, "model") <- model
#>     attr(out, "variables") <- predictors
#>     attr(out, "jacobian") <- J
#>     attr(out, "vcov") <- vcov
#>     attr(out, "vcov.type") <- vcov.type
#>     attr(out, "weights") <- marginaleffects_wts_internal
#>     attr(out, "comparison") <- comparison
#>     attr(out, "transform") <- transform[[1]]
#>     attr(out, "comparison_label") <- comparison_label
#>     attr(out, "hypothesis_by") <- hyp_by
#>     attr(out, "transform_label") <- transform_label
#>     attr(out, "conf_level") <- conf_level
#>     attr(out, "by") <- by
#>     if (inherits(model, "brmsfit")) {
#>         insight::check_if_installed("brms")
#>         attr(out, "nchains") <- brms::nchains(model)
#>     }
#>     class(out) <- c("comparisons", class(out))
#>     return(out)
#> }, newdata = list(c(5.1, 4.9, 4.7, 4.6, 5, 5.4, 4.6, 5, 4.4, 
#> 4.9, 5.4, 4.8, 4.8, 4.3, 5.8, 5.7, 5.4, 5.1, 5.7, 5.1, 5.4, 5.1, 
#> 4.6, 5.1, 4.8, 5, 5, 5.2, 5.2, 4.7, 4.8, 5.4, 5.2, 5.5, 4.9, 
#> 5, 5.5, 4.9, 4.4, 5.1, 5, 4.5, 4.4, 5, 5.1, 4.8, 5.1, 4.6, 5.3, 
#> 5, 7, 6.4, 6.9, 5.5, 6.5, 5.7, 6.3, 4.9, 6.6, 5.2, 5, 5.9, 6, 
#> 6.1, 5.6, 6.7, 5.6, 5.8, 6.2, 5.6, 5.9, 6.1, 6.3, 6.1, 6.4, 6.6, 
#> 6.8, 6.7, 6, 5.7, 5.5, 5.5, 5.8, 6, 5.4, 6, 6.7, 6.3, 5.6, 5.5, 
#> 5.5, 6.1, 5.8, 5, 5.6, 5.7, 5.7, 6.2, 5.1, 5.7, 6.3, 5.8, 7.1, 
#> 6.3, 6.5, 7.6, 4.9, 7.3, 6.7, 7.2, 6.5, 6.4, 6.8, 5.7, 5.8, 6.4, 
#> 6.5, 7.7, 7.7, 6, 6.9, 5.6, 7.7, 6.3, 6.7, 7.2, 6.2, 6.1, 6.4, 
#> 7.2, 7.4, 7.9, 6.4, 6.3, 6.1, 7.7, 6.3, 6.4, 6, 6.9, 6.7, 6.9, 
#> 5.8, 6.8, 6.7, 6.7, 6.3, 6.5, 6.2, 5.9), c(0.2, 0.2, 0.2, 0.2, 
#> 0.2, 0.4, 0.3, 0.2, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 0.4, 0.4, 
#> 0.3, 0.3, 0.3, 0.2, 0.4, 0.2, 0.5, 0.2, 0.2, 0.4, 0.2, 0.2, 0.2, 
#> 0.2, 0.4, 0.1, 0.2, 0.2, 0.2, 0.2, 0.1, 0.2, 0.2, 0.3, 0.3, 0.2, 
#> 0.6, 0.4, 0.3, 0.2, 0.2, 0.2, 0.2, 1.4, 1.5, 1.5, 1.3, 1.5, 1.3, 
#> 1.6, 1, 1.3, 1.4, 1, 1.5, 1, 1.4, 1.3, 1.4, 1.5, 1, 1.5, 1.1, 
#> 1.8, 1.3, 1.5, 1.2, 1.3, 1.4, 1.4, 1.7, 1.5, 1, 1.1, 1, 1.2, 
#> 1.6, 1.5, 1.6, 1.5, 1.3, 1.3, 1.3, 1.2, 1.4, 1.2, 1, 1.3, 1.2, 
#> 1.3, 1.3, 1.1, 1.3, 2.5, 1.9, 2.1, 1.8, 2.2, 2.1, 1.7, 1.8, 1.8, 
#> 2.5, 2, 1.9, 2.1, 2, 2.4, 2.3, 1.8, 2.2, 2.3, 1.5, 2.3, 2, 2, 
#> 1.8, 2.1, 1.8, 1.8, 1.8, 2.1, 1.6, 1.9, 2, 2.2, 1.5, 1.4, 2.3, 
#> 2.4, 1.8, 1.8, 2.1, 2.4, 2.3, 1.9, 2.3, 2.5, 2.3, 1.9, 2, 2.3, 
#> 1.8), c(1.4, 1.4, 1.3, 1.5, 1.4, 1.7, 1.4, 1.5, 1.4, 1.5, 1.5, 
#> 1.6, 1.4, 1.1, 1.2, 1.5, 1.3, 1.4, 1.7, 1.5, 1.7, 1.5, 1, 1.7, 
#> 1.9, 1.6, 1.6, 1.5, 1.4, 1.6, 1.6, 1.5, 1.5, 1.4, 1.5, 1.2, 1.3, 
#> 1.4, 1.3, 1.5, 1.3, 1.3, 1.3, 1.6, 1.9, 1.4, 1.6, 1.4, 1.5, 1.4, 
#> 4.7, 4.5, 4.9, 4, 4.6, 4.5, 4.7, 3.3, 4.6, 3.9, 3.5, 4.2, 4, 
#> 4.7, 3.6, 4.4, 4.5, 4.1, 4.5, 3.9, 4.8, 4, 4.9, 4.7, 4.3, 4.4, 
#> 4.8, 5, 4.5, 3.5, 3.8, 3.7, 3.9, 5.1, 4.5, 4.5, 4.7, 4.4, 4.1, 
#> 4, 4.4, 4.6, 4, 3.3, 4.2, 4.2, 4.2, 4.3, 3, 4.1, 6, 5.1, 5.9, 
#> 5.6, 5.8, 6.6, 4.5, 6.3, 5.8, 6.1, 5.1, 5.3, 5.5, 5, 5.1, 5.3, 
#> 5.5, 6.7, 6.9, 5, 5.7, 4.9, 6.7, 4.9, 5.7, 6, 4.8, 4.9, 5.6, 
#> 5.8, 6.1, 6.4, 5.6, 5.1, 5.6, 6.1, 5.6, 5.5, 4.8, 5.4, 5.6, 5.1, 
#> 5.1, 5.9, 5.7, 5.2, 5, 5.2, 5.4, 5.1), c(1, 1, 1, 1, 1, 1, 1, 
#> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
#> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 
#> 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 
#> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 
#> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3), 1:150), vcov = c(0.0648561358708614, 
#> -0.0776075989705508, -0.0339713425276863, 0.0855221415040715, 
#> 0.088949330395953, 0.0170000905579961, -0.0776075989705508, 0.144241283857235, 
#> 0.0351858298835599, -0.117761642231515, -0.122089476746883, -0.0254853395735601, 
#> -0.0339713425276863, 0.0351858298835599, 0.0194905399614243, 
#> -0.0459657347451442, -0.0472861136127865, -0.00869677529341257, 
#> 0.0855221415040715, -0.117761642231515, -0.0459657347451442, 
#> 0.138337982006763, 0.15072216454488, 0.0227990421926966, 0.088949330395953, 
#> -0.122089476746883, -0.0472861136127865, 0.15072216454488, 0.179060079636581, 
#> 0.0216170189525216, 0.0170000905579961, -0.0254853395735601, 
#> -0.00869677529341257, 0.0227990421926966, 0.0216170189525216, 
#> 0.00542666426714662), variables = "Petal.Width", type = "response", 
#>     by = "Species", conf_level = 0.95, cross = FALSE, comparison = "difference", 
#>     transform = NULL, wts = FALSE, hypothesis = NULL, eps = NULL)
#> 
#> 
#> Bootstrap Statistics :
#>        original        bias    std. error
#> t1* -0.11025325 -0.0030423883   0.2900819
#> t2* -0.02006005 -0.0004180816   0.1672193
#> t3*  0.02158742  0.0007937141   0.1833126

Or we can extract the individual draws with the posterior_draws() function:

posterior_draws(est) |> head()
#>   drawid        draw        term contrast    Species    estimate predicted_lo predicted_hi predicted std.error   conf.low conf.high
#> 1      1 -0.46836678 Petal.Width mean(+1)     setosa -0.11025325     4.957514     4.845263  4.957514 0.2900819 -0.6425308 0.4921824
#> 2      1 -0.16414880 Petal.Width mean(+1) versicolor -0.02006005     6.327949     6.322072  6.327949 0.1672193 -0.3518707 0.3237374
#> 3      1 -0.02367360 Petal.Width mean(+1)  virginica  0.02158742     7.015513     7.051542  7.015513 0.1833126 -0.3674288 0.3745883
#> 4      2  0.08328491 Petal.Width mean(+1)     setosa -0.11025325     4.957514     4.845263  4.957514 0.2900819 -0.6425308 0.4921824
#> 5      2  0.16316822 Petal.Width mean(+1) versicolor -0.02006005     6.327949     6.322072  6.327949 0.1672193 -0.3518707 0.3237374
#> 6      2  0.20005501 Petal.Width mean(+1)  virginica  0.02158742     7.015513     7.051542  7.015513 0.1833126 -0.3674288 0.3745883

posterior_draws(est, shape = "DxP") |> dim()
#> [1] 500   3

rsample

As before, we can pass arguments to rsample::bootstraps() through inferences(). For example, for stratified resampling:

est <- avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
  inferences(method = "rsample", R = 100, strata = "Species")
est
#> 
#>         Term Contrast    Species Estimate  2.5 % 97.5 %
#>  Petal.Width mean(+1) setosa      -0.1103 -0.574  0.375
#>  Petal.Width mean(+1) versicolor  -0.0201 -0.299  0.262
#>  Petal.Width mean(+1) virginica    0.0216 -0.292  0.363
#> 
#> Columns: term, contrast, Species, estimate, predicted_lo, predicted_hi, predicted, conf.low, conf.high 
#> Type:  response

attr(est, "inferences")
#> # Bootstrap sampling using stratification with apparent sample 
#> # A tibble: 101 × 3
#>    splits           id           estimates       
#>    <list>           <chr>        <list>          
#>  1 <split [150/57]> Bootstrap001 <tibble [3 × 7]>
#>  2 <split [150/50]> Bootstrap002 <tibble [3 × 7]>
#>  3 <split [150/61]> Bootstrap003 <tibble [3 × 7]>
#>  4 <split [150/60]> Bootstrap004 <tibble [3 × 7]>
#>  5 <split [150/58]> Bootstrap005 <tibble [3 × 7]>
#>  6 <split [150/57]> Bootstrap006 <tibble [3 × 7]>
#>  7 <split [150/56]> Bootstrap007 <tibble [3 × 7]>
#>  8 <split [150/56]> Bootstrap008 <tibble [3 × 7]>
#>  9 <split [150/57]> Bootstrap009 <tibble [3 × 7]>
#> 10 <split [150/56]> Bootstrap010 <tibble [3 × 7]>
#> # ℹ 91 more rows

Or we can extract the individual draws with the posterior_draws() function:

posterior_draws(est) |> head()
#>   drawid        draw        term contrast    Species    estimate predicted_lo predicted_hi predicted   conf.low conf.high
#> 1      1  0.37776736 Petal.Width mean(+1)     setosa -0.11025325     4.957514     4.845263  4.957514 -0.5738371 0.3750435
#> 2      1  0.21404045 Petal.Width mean(+1) versicolor -0.02006005     6.327949     6.322072  6.327949 -0.2993074 0.2618256
#> 3      1  0.13843817 Petal.Width mean(+1)  virginica  0.02158742     7.015513     7.051542  7.015513 -0.2918573 0.3634466
#> 4      2  0.05088565 Petal.Width mean(+1)     setosa -0.11025325     4.957514     4.845263  4.957514 -0.5738371 0.3750435
#> 5      2 -0.08991371 Petal.Width mean(+1) versicolor -0.02006005     6.327949     6.322072  6.327949 -0.2993074 0.2618256
#> 6      2 -0.15492899 Petal.Width mean(+1)  virginica  0.02158742     7.015513     7.051542  7.015513 -0.2918573 0.3634466

posterior_draws(est, shape = "PxD") |> dim()
#> [1]   3 100

Fractional Weighted Bootstrap (aka Bayesian Bootstrap)

The fwb package implements fractional weighted bootstrap (aka Bayesian bootstrap):

“fwb implements the fractional weighted bootstrap (FWB), also known as the Bayesian bootstrap, following the treatment by Xu et al. (2020). The FWB involves generating sets of weights from a uniform Dirichlet distribution to be used in estimating statistics of interest, which yields a posterior distribution that can be interpreted in the same way the traditional (resampling-based) bootstrap distribution can be.” -Noah Greifer

The inferences() function makes it easy to apply this inference strategy to marginaleffects objects:

avg_comparisons(mod) |> inferences(method = "fwb")
#> 
#>          Term                        Contrast Estimate Std. Error  2.5 % 97.5 %
#>  Petal.Length mean(+1)                          0.8929     0.0769  0.736  1.053
#>  Petal.Width  mean(+1)                         -0.0362     0.1611 -0.337  0.290
#>  Species      mean(versicolor) - mean(setosa)  -1.4629     0.3248 -2.098 -0.829
#>  Species      mean(virginica) - mean(setosa)   -1.9842     0.3913 -2.762 -1.227
#> 
#> Columns: term, contrast, estimate, predicted_lo, predicted_hi, predicted, std.error, conf.low, conf.high 
#> Type:  response

Simulation-based inference

This simulation-based strategy to compute confidence intervals was described in Krinsky & Robb (1986) and popularized by King, Tomz, Wittenberg (2000). We proceed in 3 steps:

1. Draw R sets of simulated coefficients from a multivariate normal distribution with mean equal to the original model’s estimated coefficients and variance equal to the model’s variance-covariance matrix (classical, “HC3”, or other).
2. Use the R sets of coefficients to compute R sets of estimands: predictions, comparisons, or slopes.
3. Take quantiles of the resulting distribution of estimands to obtain a confidence interval and the standard deviation of simulated estimates to estimate the standard error.

Here are a few examples:

library(ggplot2)
library(ggdist)

avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
  inferences(method = "simulation")
#> 
#>         Term Contrast    Species Estimate  2.5 % 97.5 %
#>  Petal.Width mean(+1) setosa      -0.1103 -0.686  0.420
#>  Petal.Width mean(+1) versicolor  -0.0201 -0.341  0.298
#>  Petal.Width mean(+1) virginica    0.0216 -0.289  0.371
#> 
#> Columns: term, contrast, Species, estimate, predicted_lo, predicted_hi, predicted, conf.low, conf.high 
#> Type:  response

Since simulation based inference generates R estimates of the quantities of interest, we can treat them similarly to draws from the posterior distribution in bayesian models. For example, we can extract draws using the posterior_draws() function, and plot their distributions using packages likeggplot2 and ggdist:

avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
  inferences(method = "simulation") |>
  posterior_draws("rvar") |>
  ggplot(aes(y = Species, xdist = rvar)) +
  stat_slabinterval()

Multiple imputation and missing data

The same workflow and the same inferences function can be used to estimate models with multiple imputation for missing data.