library(marginaleffects)
mod <- lm(Sepal.Length ~ Petal.Width * Petal.Length + factor(Species), data = iris)16 Bootstrap & Simulation
marginaleffects offers an inferences() function to compute uncertainty estimates using the bootstrap and simulation-based inference.
WARNING: The inferences() function is experimental. It may be renamed, the user interface may change, or the functionality may migrate to arguments in other marginaleffects functions.
Consider a simple model:
We will compute uncertainty estimates around the output of comparisons(), but note that the same approach works with the predictions() and slopes() functions as well.
16.1 Delta method
The default strategy to compute standard errors and confidence intervals is the delta method. This is what we obtain by calling:
avg_comparisons(mod, by = "Species", variables = "Petal.Width")
#>
#> Species Estimate Std. Error z Pr(>|z|) S 2.5 % 97.5 %
#> setosa -0.1103 0.285 -0.387 0.699 0.5 -0.669 0.449
#> versicolor -0.0201 0.160 -0.125 0.900 0.2 -0.334 0.293
#> virginica 0.0216 0.169 0.128 0.898 0.2 -0.309 0.353
#>
#> Term: Petal.Width
#> Type: response
#> Comparison: mean(+1)
#> Columns: term, contrast, Species, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predictedSince this is the default method, we obtain the same results if we add the inferences() call in the chain:
avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
inferences(method = "delta")
#>
#> Species Estimate Std. Error z Pr(>|z|) S 2.5 % 97.5 %
#> setosa -0.1103 0.285 -0.387 0.699 0.5 -0.669 0.449
#> versicolor -0.0201 0.160 -0.125 0.900 0.2 -0.334 0.293
#> virginica 0.0216 0.169 0.128 0.898 0.2 -0.309 0.353
#>
#> Term: Petal.Width
#> Type: response
#> Comparison: mean(+1)
#> Columns: term, contrast, Species, estimate, std.error, statistic, p.value, s.value, conf.low, conf.high, predicted_lo, predicted_hi, predicted16.2 Bootstrap
marginaleffects supports three bootstrap frameworks in R: the well-established boot package, the newer rsample package, and the so-called “bayesian bootstrap” in fwb.
16.2.1 boot
avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
inferences(method = "boot")
#>
#> Species Estimate Std. Error 2.5 % 97.5 %
#> setosa -0.1103 0.261 -0.648 0.388
#> versicolor -0.0201 0.161 -0.321 0.310
#> virginica 0.0216 0.182 -0.347 0.419
#>
#> Term: Petal.Width
#> Type: response
#> Comparison: mean(+1)
#> Columns: term, contrast, Species, estimate, predicted_lo, predicted_hi, predicted, std.error, conf.low, conf.highAll unknown arguments that we feed to inferences() are pushed forward to boot::boot():
est <- avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
inferences(method = "boot", sim = "balanced", R = 500, conf_type = "bca")
est
#>
#> Species Estimate Std. Error 2.5 % 97.5 %
#> setosa -0.1103 0.270 -0.665 0.394
#> versicolor -0.0201 0.168 -0.342 0.321
#> virginica 0.0216 0.188 -0.360 0.369
#>
#> Term: Petal.Width
#> Type: response
#> Comparison: mean(+1)
#> Columns: term, contrast, Species, estimate, predicted_lo, predicted_hi, predicted, std.error, conf.low, conf.highWe can extract the original boot object from an attribute:
attr(est, "inferences")
#>
#> BALANCED BOOTSTRAP
#>
#>
#> Call:
#> (function (model, INF_FUN, ...)
#> {
#> conf_type <- attr(model, "inferences_conf_type")
#> checkmate::assert_choice(conf_type, choices = c("perc", "norm",
#> "basic", "bca"))
#> modcall <- insight::get_call(model)
#> modeldata <- get_modeldata(model, additional_variables = FALSE)
#> dots <- list(...)
#> dots[["vcov"]] <- FALSE
#> attr(model, "inferences_method") <- NULL
#> out <- do.call(INF_FUN, c(list(model), dots))
#> if (is.null(dots[["conf_level"]])) {
#> conf_level <- 0.95
#> }
#> else {
#> conf_level <- dots[["conf_level"]]
#> }
#> bootfun <- function(data, indices) {
#> d <- data[indices, , drop = FALSE]
#> modcall[["data"]] <- d
#> modboot <- eval(modcall)
#> modboot <- eval(modboot)
#> args <- c(list(modboot, modeldata = d), dots)
#> out <- do.call(INF_FUN, args)$estimate
#> return(out)
#> }
#> args <- list(data = modeldata, statistic = bootfun)
#> args <- c(args, attr(model, "inferences_dots"))
#> args <- args[unique(names(args))]
#> B <- do.call(boot::boot, args)
#> B$call <- match.call()
#> pr <- utils::capture.output(print(B))
#> pr <- pr[(grep("^Bootstrap Statistics :", pr) + 1):length(pr)]
#> pr <- gsub("std. error", "std.error", pr)
#> pr <- paste(pr, collapse = "\n")
#> pr <- utils::read.table(text = pr, header = TRUE)
#> out$std.error <- pr$std.error
#> ci_list <- lapply(seq_along(B$t0), boot::boot.ci, boot.out = B,
#> conf = conf_level, type = conf_type)
#> pos <- pmatch(conf_type, names(ci_list[[1]]))
#> if (conf_type == "norm") {
#> cols <- 2:3
#> }
#> else {
#> cols <- 4:5
#> }
#> ci <- lapply(ci_list, function(x) x[[pos]])
#> ci <- do.call("rbind", ci)[, cols]
#> if (is.matrix(ci)) {
#> out$conf.low <- ci[, 1]
#> out$conf.high <- ci[, 2]
#> }
#> else {
#> out$conf.low <- ci[1]
#> out$conf.high <- ci[2]
#> }
#> attr(out, "inferences") <- B
#> attr(out, "posterior_draws") <- t(B$t)
#> return(out)
#> })(model = list(c(3.78396410453894, -0.157380650291409, 0.854286226046438,
#> -1.46293310566858, -1.98423971100058, 0.0322348822561755), c(0.142485542022615,
#> -0.0575144579773967, -0.171441137727629, -0.443587778227166,
#> 0.0424855420226034, 0.204781851364483, -0.346289276464121, -0.0435877782271645,
#> -0.557514457977397, -0.154490610917879, 0.356412221772836, -0.329661098476932,
#> -0.168739639490674, -0.411486725209057, 1.01463218252214, 0.678217887154265,
#> 0.551653922944047, 0.153710723535879, 0.494523716318892, 0.0673150544635498,
#> 0.184265581273301, 0.0782178871542645, -0.013221176978328, -0.0849600135899262,
#> -0.587881059226235, -0.129661098476932, -0.108500130740626, 0.156412221772836,
#> 0.242485542022603, -0.429661098476932, -0.329661098476932, 0.378217887154265,
#> 0.145509389082121, 0.542485542022603, -0.143587778227164, 0.214632182522137,
#> 0.62855886227237, -0.0687396394906731, -0.47144113772763, 0.0564122217728351,
#> 0.140106392608208, -0.359893607391792, -0.47144113772763, -0.0873391630043204,
#> -0.268654184425299, -0.146289276464121, -0.0296610984769325,
#> -0.357514457977398, 0.256412221772835, 0.042485542022603, 0.672051123873723,
#> 0.253166504128602, 0.392111084356321, -0.201202445409389, 0.262902649185532,
#> -0.449298231899122, -0.0267735353888003, -0.189170005977571,
#> 0.361082610802931, -0.408416827162209, -0.266474227638094, 0.0239580689578126,
#> 0.290265218210599, -0.227948876126278, 0.257274183782397, 0.641875642235248,
#> -0.546833495871398, 0.00161310738033736, 0.053166504128602, -0.0179162100099074,
#> -0.516829096062078, 0.398797554590611, -0.207888915643679, -0.229124216863755,
#> 0.429940082696772, 0.541875642235248, 0.382109617753214, 0.101088477215358,
#> -0.146833495871398, 0.433525772361906, -0.0289417503570838, 0.0562215507013826,
#> 0.185250250939325, -0.689118350451328, -0.746833495871398, -0.145601127857536,
#> 0.372638794242462, 0.240320925398824, -0.190821602707335, -0.201202445409389,
#> -0.5612337914376, -0.138007370005769, 0.09595344246394, -0.0891700059775712,
#> -0.280440760005283, -0.182640174486829, -0.180440760005282, 0.229940082696771,
#> 0.282853926865501, -0.090821602707335, -0.715513357931099, -0.369916919883858,
#> 0.191096047245598, -0.325369702016127, -0.319664171555401, 0.045710412096514,
#> -0.723062154511679, 0.0360139881085848, -0.107831504837638, 0.0909992989002138,
#> 0.329381355194633, 0.0469765796495076, 0.259887838759361, -0.378743045749488,
#> -0.373425544491401, 0.0415908893829795, -0.0341388006053719,
#> 0.0476531581355767, -0.143891438994049, -0.0768461652547545,
#> 0.170220307288722, -0.38686744669361, 0.0593717703005697, 0.31324660785916,
#> -0.0245080569975196, 0.209706692340851, 0.304477509269916, 0.11324660785916,
#> -0.232310109119078, 0.398084828521244, 0.31455057778297, 0.534998567468207,
#> -0.234623578153396, 0.132889979802175, -0.516115825878858, 0.598849725194466,
#> -0.33925051622203, -0.134138800605372, 0.104477509269916, 0.452085786637802,
#> 0.0607494837779699, 0.727276180430108, -0.369916919883858, -0.115464983758407,
#> -0.0350513284250356, 0.434433534906543, 0.221636330349458, 0.237505756138754,
#> -0.251251756140585, -0.26921519496235), c(-71.5659253183152,
#> 8.26761414419049, 3.15185596602052, -1.28216612276567, -2.35695868745888,
#> 0.148835738547779, -0.340443902992748, -0.0297961938363055, -0.537559267311158,
#> -0.126979856156088, 0.370203806163695, -0.322033120361453, -0.134674631629569,
#> -0.357758958050013, 1.04691458573913, 0.66457113080326, 0.548771791848065,
#> 0.159556097007252, 0.483050211435928, 0.0673874684834769, 0.185729953113401,
#> 0.0645711308032596, 0.0313884387894284, -0.122309271919019, -0.598743899936893,
#> -0.122033120361453, -0.127529199719143, 0.170203806163695, 0.262440732688842,
#> -0.422033120361452, -0.322033120361453, 0.36457113080326, 0.173020143843912,
#> 0.562440732688841, -0.129796193836305, 0.246914585739135, 0.654677659213988,
#> -0.0346746316295687, -0.445322340786012, 0.0702038061636942,
#> 0.151724725531026, -0.348275274468974, -0.445322340786012, -0.133025279076833,
#> -0.303830191286351, -0.140443902992748, -0.0220331203614529,
#> -0.337559267311159, 0.270203806163694, 0.0624407326888415, 0.594789558050432,
#> 0.176856861428742, 0.311460651399507, -0.264180100742888, 0.185507808921433,
#> -0.521608343293153, -0.106472045222184, -0.219019640205961, 0.286906008196794,
#> -0.47387163788012, -0.3024007252343, -0.0490959810493318, 0.239146562194852,
#> -0.305210441949568, 0.201762493297324, 0.569041609576475, -0.623143138571258,
#> -0.0525439803193176, -0.023143138571258, -0.070340080938264,
#> -0.598877806997992, 0.335819899257112, -0.288539348600494, -0.303948838676952,
#> 0.361362953726953, 0.469041609576474, 0.303372207541751, 0.0192597836278147,
#> -0.223143138571258, 0.3975992747657, -0.0787178364254662, 0.0142181897373602,
#> 0.12848273341445, -0.77159506324593, -0.823143138571258, -0.22391053621031,
#> 0.294158756414124, 0.169877305216899, -0.255665749252941, -0.264180100742888,
#> -0.629286999142676, -0.213793091440887, 0.0369287869030251, -0.119019640205961,
#> -0.347151397762995, -0.246179106119825, -0.247151397762994, 0.161362953726953,
#> 0.254260119676912, -0.15566574925294, -0.678723653276397, -0.360583343049196,
#> 0.211187321696559, -0.315946525934337, -0.297752937425579, 0.0746124702030289,
#> -0.71259338476017, 0.0460443645433169, -0.0982348429407217, 0.130610274230017,
#> 0.339059047319985, 0.0572649359471646, 0.274944379692862, -0.369933390179567,
#> -0.362371391203293, 0.0563808814348639, -0.0248023674311454,
#> 0.084408364095088, -0.09646181450644, -0.067803852018609, 0.193170207449538,
#> -0.37892582767912, 0.0829380473128177, 0.322062583588008, -0.00693414930528918,
#> 0.219476840052894, 0.313206742091201, 0.122062583588008, -0.215994884806213,
#> 0.401524204301707, 0.328658051932604, 0.555960734814162, -0.216011004430172,
#> 0.140847095474082, -0.515882047438503, 0.629959533464212, -0.316043243678089,
#> -0.124802367431145, 0.113206742091201, 0.465883644191938, 0.0839567563219111,
#> 0.737986218427527, -0.360583343049196, -0.0884351295431258, -0.0067254357956353,
#> 0.447183549931195, 0.230492517452624, 0.248051484819537, -0.234421787061468,
#> -0.260225733418376), 6, c(4.95751445797738, 4.9575144579774,
#> 4.87144113772763, 5.04358777822717, 4.9575144579774, 5.19521814863552,
#> 4.94628927646412, 5.04358777822716, 4.9575144579774, 5.05449061091788,
#> 5.04358777822716, 5.12966109847693, 4.96873963949067, 4.71148672520906,
#> 4.78536781747786, 5.02178211284574, 4.84834607705595, 4.94628927646412,
#> 5.20547628368111, 5.03268494553645, 5.2157344187267, 5.02178211284574,
#> 4.61322117697833, 5.18496001358993, 5.38788105922623, 5.12966109847693,
#> 5.10850013074063, 5.04358777822716, 4.9575144579774, 5.12966109847693,
#> 5.12966109847693, 5.02178211284574, 5.05449061091788, 4.9575144579774,
#> 5.04358777822716, 4.78536781747786, 4.87144113772763, 4.96873963949067,
#> 4.87144113772763, 5.04358777822716, 4.85989360739179, 4.85989360739179,
#> 4.87144113772763, 5.08733916300432, 5.3686541844253, 4.94628927646412,
#> 5.12966109847693, 4.9575144579774, 5.04358777822716, 4.9575144579774,
#> 6.32794887612628, 6.1468334958714, 6.50788891564368, 5.70120244540939,
#> 6.23709735081447, 6.14929823189912, 6.3267735353888, 5.08917000597757,
#> 6.23891738919707, 5.60841682716221, 5.2664742276381, 5.87604193104219,
#> 5.7097347817894, 6.32794887612628, 5.3427258162176, 6.05812435776475,
#> 6.1468334958714, 5.79838689261966, 6.1468334958714, 5.61791621000991,
#> 6.41682909606208, 5.70120244540939, 6.50788891564368, 6.32912421686375,
#> 5.97005991730323, 6.05812435776475, 6.41789038224679, 6.59891152278464,
#> 6.1468334958714, 5.2664742276381, 5.52894175035708, 5.44377844929862,
#> 5.61474974906067, 6.68911835045133, 6.1468334958714, 6.14560112785754,
#> 6.32736120575754, 6.05967907460118, 5.79082160270733, 5.70120244540939,
#> 6.0612337914376, 6.23800737000577, 5.70404655753606, 5.08917000597757,
#> 5.88044076000528, 5.88264017448683, 5.88044076000528, 5.97005991730323,
#> 4.8171460731345, 5.79082160270733, 7.0155133579311, 6.16991691988386,
#> 6.9089039527544, 6.62536970201613, 6.8196641715554, 7.55428958790349,
#> 5.62306215451168, 7.26398601189142, 6.80783150483764, 7.10900070109979,
#> 6.17061864480537, 6.35302342035049, 6.54011216124064, 6.07874304574949,
#> 6.1734255444914, 6.35840911061702, 6.53413880060537, 7.65234684186442,
#> 7.84389143899405, 6.07684616525475, 6.72977969271128, 5.98686744669361,
#> 7.64062822969943, 5.98675339214084, 6.72450805699752, 6.99029330765915,
#> 5.89552249073008, 5.98675339214084, 6.63231010911908, 6.80191517147876,
#> 7.08544942221703, 7.36500143253179, 6.6346235781534, 6.16711002019782,
#> 6.61611582587886, 7.10115027480553, 6.63925051622203, 6.53413880060537,
#> 5.89552249073008, 6.4479142133622, 6.63925051622203, 6.17272381956989,
#> 6.16991691988386, 6.91546498375841, 6.73505132842504, 6.26556646509346,
#> 6.07836366965054, 6.26249424386125, 6.45125175614059, 6.16921519496235
#> ), c(0, 1, 2, 3, 3, 4), list(c(-12.2474487139159, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> 0.0816496580927726, 0.0816496580927726, 0.0816496580927726, 0.0816496580927726,
#> -14.6887734908898, 9.30429649857169, 0.0992979345242469, 0.0992979345242469,
#> 0.0992979345242469, 0.0778024888416311, 0.088550211682939, 0.0992979345242469,
#> 0.0992979345242469, 0.110045657365555, 0.0992979345242469, 0.0992979345242469,
#> 0.110045657365555, 0.110045657365555, 0.0992979345242469, 0.0778024888416311,
#> 0.0778024888416311, 0.088550211682939, 0.088550211682939, 0.088550211682939,
#> 0.0992979345242469, 0.0778024888416311, 0.0992979345242469, 0.0670547660003233,
#> 0.0992979345242469, 0.0992979345242469, 0.0778024888416311, 0.0992979345242469,
#> 0.0992979345242469, 0.0992979345242469, 0.0992979345242469, 0.0778024888416311,
#> 0.110045657365555, 0.0992979345242469, 0.0992979345242469, 0.0992979345242469,
#> 0.0992979345242469, 0.110045657365555, 0.0992979345242469, 0.0992979345242469,
#> 0.088550211682939, 0.088550211682939, 0.0992979345242469, 0.0563070431590154,
#> 0.0778024888416311, 0.088550211682939, 0.0992979345242469, 0.0992979345242469,
#> 0.0992979345242469, 0.0992979345242469, -0.0296747395714478,
#> -0.0404224624127557, -0.0404224624127557, -0.0189270167301399,
#> -0.0404224624127557, -0.0189270167301399, -0.0511701852540636,
#> 0.0133161517937838, -0.0189270167301399, -0.0296747395714478,
#> 0.0133161517937838, -0.0404224624127557, 0.0133161517937838,
#> -0.0296747395714478, -0.0189270167301399, -0.0296747395714478,
#> -0.0404224624127557, 0.0133161517937838, -0.0404224624127557,
#> 0.00256842895247588, -0.0726656309366794, -0.0189270167301399,
#> -0.0404224624127557, -0.008179293888832, -0.0189270167301399,
#> -0.0296747395714478, -0.0296747395714478, -0.0619179080953715,
#> -0.0404224624127557, 0.0133161517937838, 0.00256842895247588,
#> 0.0133161517937838, -0.008179293888832, -0.0511701852540636,
#> -0.0404224624127557, -0.0511701852540636, -0.0404224624127557,
#> -0.0189270167301399, -0.0189270167301399, -0.0189270167301399,
#> -0.008179293888832, -0.0296747395714478, -0.008179293888832,
#> 0.0133161517937838, -0.0189270167301399, -0.008179293888832,
#> -0.0189270167301399, -0.0189270167301399, 0.00256842895247588,
#> -0.0189270167301399, -0.147899690825835, -0.0834133537779873,
#> -0.104908799460603, -0.0726656309366794, -0.115656522301911,
#> -0.104908799460603, -0.0619179080953715, -0.0726656309366794,
#> -0.0726656309366794, -0.147899690825835, -0.0941610766192952,
#> -0.0834133537779873, -0.104908799460603, -0.0941610766192952,
#> -0.137151967984527, -0.126404245143219, -0.0726656309366794,
#> -0.115656522301911, -0.126404245143219, -0.0404224624127557,
#> -0.126404245143219, -0.0941610766192952, -0.0941610766192952,
#> -0.0726656309366794, -0.104908799460603, -0.0726656309366794,
#> -0.0726656309366794, -0.0726656309366794, -0.104908799460603,
#> -0.0511701852540636, -0.0834133537779873, -0.0941610766192952,
#> -0.115656522301911, -0.0404224624127557, -0.0296747395714478,
#> -0.126404245143219, -0.137151967984527, -0.0726656309366794,
#> -0.0726656309366794, -0.104908799460603, -0.137151967984527,
#> -0.126404245143219, -0.0834133537779873, -0.126404245143219,
#> -0.147899690825835, -0.126404245143219, -0.0834133537779873,
#> -0.0941610766192952, -0.126404245143219, -0.0726656309366794,
#> -46.0259122668959, 20.7480275407855, 5.81762435765699, 0.00153822106493907,
#> 0.0187273680177469, 0.0442240300146606, 0.0572594194454154, 0.00153822106493907,
#> 0.0187273680177469, -0.0369938303627295, 0.00153822106493907,
#> -0.0156509258878688, -0.0198046834099217, 0.0317627574485017,
#> 0.0531056619233624, 0.0786023239202762, 0.112980617825892, 0.0572594194454154,
#> 0.00569197858699198, 0.0400702724926076, -0.0328400728406765,
#> 0.0786023239202762, 0.087483955828978, 0.0827560814423291, -0.0672183667462921,
#> -0.0156509258878688, 0.0614131769674684, 0.00153822106493907,
#> 0.0187273680177469, -0.0156509258878688, -0.0156509258878688,
#> 0.0786023239202762, -0.0369938303627295, 0.0187273680177469,
#> 0.00153822106493907, 0.0531056619233624, 0.0359165149705546,
#> -0.0198046834099217, 0.0359165149705546, 0.00153822106493907,
#> 0.0744485663982231, 0.0744485663982231, 0.0359165149705546, 0.138477279822805,
#> 0.00984573610904502, 0.0572594194454154, -0.0156509258878688,
#> 0.0187273680177469, 0.00153822106493907, 0.0187273680177469,
#> -0.0861298642928878, -0.0132195189596035, -0.0819761067708348,
#> -0.00433788705090168, -0.0304086659124113, -0.0902836218149406,
#> -0.0090657614375506, 0.000389987335747224, -0.107472768767748,
#> 0.0513833113295746, -0.0339883065698684, 0.0383479218988198,
#> -0.119934041333907, -0.0861298642928878, 0.0644187007603295,
#> -0.0345624234344644, -0.0132195189596035, -0.137123188286715,
#> -0.0132195189596035, -0.064212842953431, 0.0508091944649788,
#> -0.00433788705090168, -0.0819761067708348, -0.163193967148225,
#> -0.055905327909325, -0.0345624234344644, -0.103319011245696,
#> -0.0221011508683054, -0.0132195189596035, -0.0339883065698684,
#> -0.0470236960006232, -0.068366600475484, -0.0256807915257625,
#> -0.0778223492487817, -0.0132195189596035, 0.025312532468065,
#> -0.0475978128652192, -0.0730944748621329, -0.0215270340037094,
#> -0.00433788705090168, -0.111626526289801, -0.0689407173400799,
#> -0.0428699384785703, 0.000389987335747224, -0.0387161809565173,
#> -0.0772482323841859, -0.0387161809565173, -0.055905327909325,
#> 0.0904894796218392, -0.0215270340037094, 0.114263791024965, 0.0377738050342239,
#> -0.0226752677329013, -0.0867039811574835, 0.0330459306475751,
#> -0.142999296402556, 0.0638445838957336, -0.207028009827138, -0.121082275063099,
#> 0.0970746440721574, 0.0763058564618925, 0.00339551112860831,
#> 0.0460813200783299, 0.0934950034147003, 0.230434062172567, 0.157523716839282,
#> -0.0695148342046758, -0.121656391927695, -0.117502634405642,
#> -0.0991652537236425, 0.0887671290280512, 0.110684150367508, -0.198720494783032,
#> 0.0336200475121709, 0.0117030261727143, -0.155460568968715, 0.0508091944649788,
#> 0.0336200475121709, 0.0288921731255222, -0.198146377918436, -0.134117664493854,
#> -0.147153053924609, 0.0674242245531907, -0.11635440067645, -0.240832186868158,
#> 0.0200105412168202, 0.144488327408528, -0.0695148342046758, 0.0508091944649788,
#> 0.0632704670311376, 0.144488327408528, 0.191902010744898, 0.0377738050342239,
#> 0.0543888351224356, 0.165831231883388, 0.17471286379209, 0.0549629519870317,
#> 0.0591167095090847, 0.140334569886475, -0.000758246393444546,
#> -4.08248290463863, 0.680689113282833, 1.8868601435992, 5.41384839674038,
#> 0.033392481153859, 0.0273909956668532, 0.0207108967780347, 0.0398463755754066,
#> 0.033392481153859, 0.0525279599512309, 0.0398463755754066, 0.0463002699969543,
#> 0.0460740655296833, 0.0267123822650405, 0.0204846923107638, 0.014483206823758,
#> 0.00157541798066278, 0.0207108967780347, 0.0400725800426776,
#> 0.0271647911995823, 0.0527541644185019, 0.014483206823758, 0.00757690346766855,
#> 0.0147094112910289, 0.0656619532615971, 0.0463002699969543, 0.0209371012453056,
#> 0.0398463755754066, 0.033392481153859, 0.0463002699969543, 0.0463002699969543,
#> 0.014483206823758, 0.0525279599512309, 0.033392481153859, 0.0398463755754066,
#> 0.0204846923107638, 0.0269385867323114, 0.0460740655296833, 0.0269385867323114,
#> 0.0398463755754066, 0.0142570023564871, 0.0142570023564871, 0.0269385867323114,
#> -0.00442606750634301, 0.0402987845099485, 0.0207108967780347,
#> 0.0463002699969543, 0.033392481153859, 0.0398463755754066, 0.033392481153859,
#> -0.0905195047068495, -0.116108877925769, -0.0902933002395785,
#> -0.123015181281858, -0.109654983504221, -0.0907457091741204,
#> -0.115882673458498, -0.130147689105219, -0.0842918147525728,
#> -0.14215066007923, -0.117239900262124, -0.135470561190412, -0.0849704281543855,
#> -0.0905195047068495, -0.148830758968049, -0.109881187971492,
#> -0.116108877925769, -0.0785165337328379, -0.116108877925769,
#> -0.104105906951757, -0.134791947788599, -0.123015181281858, -0.0902933002395785,
#> -0.0651563359552008, -0.103653498017216, -0.109881187971492,
#> -0.0840656102853019, -0.10920257456968, -0.116108877925769, -0.117239900262124,
#> -0.110559801373305, -0.104332111419028, -0.116787491327582, -0.0900670957723077,
#> -0.116108877925769, -0.128790462301593, -0.103201089082674, -0.097199603595668,
#> -0.116561286860311, -0.123015181281858, -0.0845180192198437,
#> -0.0969733991283971, -0.110333596906034, -0.130147689105219,
#> -0.110107392438763, -0.0974258080629389, -0.110107392438763,
#> -0.103653498017216, -0.162190956745686, -0.116561286860311, 0.03859518390109,
#> 0.0565996403621074, 0.0828676269828397, 0.10155069684567, 0.0637321481854677,
#> 0.128044887933673, 0.0432394425844703, 0.146727957796503, 0.114458485688765,
#> 0.0450490783226376, 0.0439180559862831, 0.0695074292052027, 0.0570520492966492,
#> 0.0374641615647355, -0.00680828151701418, 0.0187810917019054,
#> 0.0950968024241222, 0.121817197979396, 0.122043402446667, 0.100872083443857,
#> 0.0445966693880959, 0.0310102671431879, 0.147180366731045, 0.0563734358948365,
#> 0.0699598381397445, 0.12736627453186, 0.0499195414732889, 0.0563734358948365,
#> 0.0635059437181968, 0.139821654440414, 0.121138584577584, 0.127818683466402,
#> 0.0508243593423725, 0.107325977865405, 0.152277034348967, 0.0704122470742863,
#> 0.0254611905907239, 0.0950968024241222, 0.0499195414732889, 0.0505981548751016,
#> 0.0254611905907239, 0.00587330285881018, 0.0565996403621074,
#> 0.0575044582311911, 0.0192335006364472, 0.0123271972803578, 0.0501457459405598,
#> 0.0503719504078307, 0.0252349861234531, 0.0692812247379317, -4.08248290463863,
#> 4.44239210774059, -0.424722128220301, -3.4890454943047, 1.11564257129667,
#> 0.0809550703643557, 0.024695794624255, 0.0162883406803865, 0.00432538523132845,
#> -0.00408206871254012, 0.0162883406803865, 0.0282512961294445,
#> -0.0160450241615981, -0.0519338905087722, -0.0196005256667876,
#> 0.0570291594662397, 0.0331032485681237, 0.024695794624255, 0.0605846609714291,
#> 0.036658750073313, 0.0402142515785025, 0.0570291594662397, -0.0435264365649036,
#> 0.101325479757282, 0.0641401624766186, 0.0282512961294445, 0.0689921149152977,
#> 0.0162883406803865, 0.00432538523132845, 0.0282512961294445,
#> 0.0282512961294445, 0.0570291594662397, -0.00408206871254012,
#> 0.00432538523132845, 0.0162883406803865, -0.0196005256667876,
#> -0.00763757021772952, -0.0160450241615981, -0.00763757021772952,
#> 0.0162883406803865, 0.012732839175197, 0.012732839175197, -0.00763757021772952,
#> 0.109732933701151, 0.104880981262472, 0.024695794624255, 0.0282512961294445,
#> 0.00432538523132845, 0.0162883406803865, 0.00432538523132845,
#> 0.0688922571589244, 0.0653367556537349, 0.113188577449967, -0.0352188403774083,
#> 0.0772997111027929, 0.0245959368678818, 0.109633075944778, -0.180070756699594,
#> 0.0365588923169398, -0.0268113864335398, -0.156144845801478,
#> 0.0294478893065609, -0.0963300685561881, 0.0688922571589244,
#> -0.0830706621736404, 0.0330033908117504, 0.0653367556537349,
#> -0.0843671131071301, 0.0653367556537349, -0.0879226146123196,
#> 0.162336850179689, -0.0352188403774083, 0.113188577449967, 0.0281514383730712,
#> 0.000670025969765682, 0.0330033908117504, 0.0808552126079824,
#> 0.165892351684878, 0.0653367556537349, -0.156144845801478, -0.0998855700613776,
#> -0.132218934903362, -0.0675522052193929, 0.15748489774101, 0.0653367556537349,
#> 0.0857071650466615, 0.089262666551851, 0.0126329814188238, -0.0232558849283504,
#> -0.0352188403774083, -0.00773742797410282, 0.0569293017098664,
#> -0.0555892497703349, -0.180070756699594, -0.0112929294792922,
#> -0.0316633388722189, -0.0112929294792922, 0.000670025969765682,
#> -0.195589213653842, -0.0232558849283504, 0.126796326095687, -0.103092729303395,
#> 0.0333517330749229, -0.0636483614510311, 0.0417591870187914,
#> 0.117092421218329, -0.215611280783596, 0.020092326692375, -0.0397224505529149,
#> 0.138759281544745, -0.082722319910468, -0.0791668184052785, -0.0145000887213092,
#> -0.094685275359526, -0.00124068233876161, 0.00231481916642784,
#> -0.075611316900089, 0.149425786060314, 0.193722106351356, -0.196537322324159,
#> 0.0501666409626599, -0.106648230808584, 0.10868496727446, -0.147389049594437,
#> 0.00942582217680681, -0.0157965396547989, -0.159352005043495,
#> -0.147389049594437, -0.00253713327225124, -0.0804632693387682,
#> 0.0165368251871856, 0.0727961009272864, 0.0178332761206754, -0.184574366875101,
#> -0.145129999022737, 0.0980184627588919, 0.0585740949065285, -0.075611316900089,
#> -0.159352005043495, -0.0264630441703672, 0.0585740949065285,
#> -0.0216110917316882, -0.103092729303395, 0.074092551860776, 0.0909074597485132,
#> -0.00964813628263016, -0.115055684752453, -0.0707593644614099,
#> 0.0142777746154859, -0.123463138696321, -70.9625343450096, 56.3907660022668,
#> 3.08793638683961, -8.84664926411501, -4.44414200339705, 4.61722606476297,
#> 0.0342936864802481, -0.0102831456627637, -0.0400113570729295,
#> -0.0824223866626905, -0.0102831456627637, 0.0194450657474023,
#> -0.114316400626107, -0.209998442516358, -0.0994677798932615,
#> 0.13399533633709, 0.083202123729762, 0.0342936864802481, 0.116980913050993,
#> 0.0618560953371631, 0.0491732771575683, 0.13399533633709, -0.158924202713593,
#> 0.252596184837844, 0.1086296999779, 0.0194450657474023, 0.159391942640755,
#> -0.0102831456627637, -0.0400113570729295, 0.0194450657474023,
#> 0.0194450657474023, 0.13399533633709, -0.0824223866626905, -0.0400113570729295,
#> -0.0102831456627637, -0.0994677798932615, -0.0697395684830954,
#> -0.114316400626107, -0.0697395684830954, -0.0102831456627637,
#> 0.00673127762333315, 0.00673127762333315, -0.0697395684830954,
#> 0.299338819534107, 0.235581761551747, 0.0342936864802481, 0.0194450657474023,
#> -0.0400113570729295, -0.0102831456627637, -0.0400113570729295,
#> 0.0400912555211672, 0.0397792583812572, 0.046070371252877, -0.00407298704557909,
#> 0.0413520365991622, 0.025448929576454, 0.0457583741129669, -0.144868055440661,
#> 0.0313533129008607, 0.0101826093519207, -0.120064473472342, 0.0350609237275423,
#> -0.0580555185515465, 0.0400912555211672, -0.0276905203432057,
#> 0.0288755132076998, 0.0397792583812572, -0.0456537275673874,
#> 0.0397792583812572, -0.0502973298137991, 0.046500863262919, -0.00407298704557909,
#> 0.046070371252877, 0.0344241369293675, 0.0136401629276408, 0.0288755132076998,
#> 0.0438298362923229, 0.0403154527430766, 0.0397792583812572, -0.120064473472342,
#> -0.0605333182447075, -0.095260891504024, -0.0301373500918924,
#> 0.0433862767715832, 0.0397792583812572, 0.0469444227836587, 0.042924814817067,
#> 0.0195445462520475, 0.00183139627882739, -0.00407298704557909,
#> 0.010213579296395, 0.0363526747500114, -0.0220671642142349, -0.144868055440661,
#> 0.00773577960323411, -0.00592679245891999, 0.00773577960323411,
#> 0.0136401629276408, -0.142421225691974, 0.00183139627882739,
#> -0.153524218361304, 0.0622209129727486, -0.0408146856742592,
#> 0.0434274166806142, -0.0503829173625185, -0.12076894538546, 0.0904331761385364,
#> 0.00895501058768104, 0.0335781577969193, -0.173609465675907,
#> 0.0563912620556454, 0.0480400489825521, 0.00487346273214126,
#> 0.0656474966039946, 0.0330726583872315, 0.00739502488813094,
#> 0.048352046122462, -0.172673474256178, -0.244663250441498, 0.0839667384232578,
#> -0.0556195439442763, 0.0749037311523439, -0.0917084907179432,
#> 0.0778998227735477, -0.017970611471059, 0.0237288989132239, 0.0828244522153951,
#> 0.0778998227735477, -0.00654857436945884, 0.075558695376638,
#> -0.00868340697823551, -0.0639397870728954, -0.0232072380528166,
#> 0.0855395166411628, 0.110062071414045, -0.118634112776683, -0.0565245654195321,
#> 0.048352046122462, 0.0828244522153951, 0.0162954998337413, -0.0565245654195321,
#> 0.0389023093043347, 0.0622209129727486, -0.0871268283604801,
#> -0.0932684764174932, 0.0231486670962329, 0.0693113449678471,
#> 0.047135027507296, -0.00835861731997081, 0.0680505638898524),
#> c(1.08164965809277, 1.09929793452425, 1.03591651497055, 1.03984637557541,
#> 1.00432538523133, 1.18478854894442), 1:6, 1e-07, 6), 144,
#> list("contr.treatment"), list(c("setosa", "versicolor", "virginica"
#> )), lm(formula = Sepal.Length ~ Petal.Width * Petal.Length +
#> factor(Species), data = iris), Sepal.Length ~ Petal.Width *
#> Petal.Length + factor(Species), list(c(5.1, 4.9, 4.7,
#> 4.6, 5, 5.4, 4.6, 5, 4.4, 4.9, 5.4, 4.8, 4.8, 4.3, 5.8, 5.7,
#> 5.4, 5.1, 5.7, 5.1, 5.4, 5.1, 4.6, 5.1, 4.8, 5, 5, 5.2, 5.2,
#> 4.7, 4.8, 5.4, 5.2, 5.5, 4.9, 5, 5.5, 4.9, 4.4, 5.1, 5, 4.5,
#> 4.4, 5, 5.1, 4.8, 5.1, 4.6, 5.3, 5, 7, 6.4, 6.9, 5.5, 6.5,
#> 5.7, 6.3, 4.9, 6.6, 5.2, 5, 5.9, 6, 6.1, 5.6, 6.7, 5.6, 5.8,
#> 6.2, 5.6, 5.9, 6.1, 6.3, 6.1, 6.4, 6.6, 6.8, 6.7, 6, 5.7,
#> 5.5, 5.5, 5.8, 6, 5.4, 6, 6.7, 6.3, 5.6, 5.5, 5.5, 6.1, 5.8,
#> 5, 5.6, 5.7, 5.7, 6.2, 5.1, 5.7, 6.3, 5.8, 7.1, 6.3, 6.5,
#> 7.6, 4.9, 7.3, 6.7, 7.2, 6.5, 6.4, 6.8, 5.7, 5.8, 6.4, 6.5,
#> 7.7, 7.7, 6, 6.9, 5.6, 7.7, 6.3, 6.7, 7.2, 6.2, 6.1, 6.4,
#> 7.2, 7.4, 7.9, 6.4, 6.3, 6.1, 7.7, 6.3, 6.4, 6, 6.9, 6.7,
#> 6.9, 5.8, 6.8, 6.7, 6.7, 6.3, 6.5, 6.2, 5.9), c(0.2, 0.2,
#> 0.2, 0.2, 0.2, 0.4, 0.3, 0.2, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1,
#> 0.2, 0.4, 0.4, 0.3, 0.3, 0.3, 0.2, 0.4, 0.2, 0.5, 0.2, 0.2,
#> 0.4, 0.2, 0.2, 0.2, 0.2, 0.4, 0.1, 0.2, 0.2, 0.2, 0.2, 0.1,
#> 0.2, 0.2, 0.3, 0.3, 0.2, 0.6, 0.4, 0.3, 0.2, 0.2, 0.2, 0.2,
#> 1.4, 1.5, 1.5, 1.3, 1.5, 1.3, 1.6, 1, 1.3, 1.4, 1, 1.5, 1,
#> 1.4, 1.3, 1.4, 1.5, 1, 1.5, 1.1, 1.8, 1.3, 1.5, 1.2, 1.3,
#> 1.4, 1.4, 1.7, 1.5, 1, 1.1, 1, 1.2, 1.6, 1.5, 1.6, 1.5, 1.3,
#> 1.3, 1.3, 1.2, 1.4, 1.2, 1, 1.3, 1.2, 1.3, 1.3, 1.1, 1.3,
#> 2.5, 1.9, 2.1, 1.8, 2.2, 2.1, 1.7, 1.8, 1.8, 2.5, 2, 1.9,
#> 2.1, 2, 2.4, 2.3, 1.8, 2.2, 2.3, 1.5, 2.3, 2, 2, 1.8, 2.1,
#> 1.8, 1.8, 1.8, 2.1, 1.6, 1.9, 2, 2.2, 1.5, 1.4, 2.3, 2.4,
#> 1.8, 1.8, 2.1, 2.4, 2.3, 1.9, 2.3, 2.5, 2.3, 1.9, 2, 2.3,
#> 1.8), c(1.4, 1.4, 1.3, 1.5, 1.4, 1.7, 1.4, 1.5, 1.4, 1.5,
#> 1.5, 1.6, 1.4, 1.1, 1.2, 1.5, 1.3, 1.4, 1.7, 1.5, 1.7, 1.5,
#> 1, 1.7, 1.9, 1.6, 1.6, 1.5, 1.4, 1.6, 1.6, 1.5, 1.5, 1.4,
#> 1.5, 1.2, 1.3, 1.4, 1.3, 1.5, 1.3, 1.3, 1.3, 1.6, 1.9, 1.4,
#> 1.6, 1.4, 1.5, 1.4, 4.7, 4.5, 4.9, 4, 4.6, 4.5, 4.7, 3.3,
#> 4.6, 3.9, 3.5, 4.2, 4, 4.7, 3.6, 4.4, 4.5, 4.1, 4.5, 3.9,
#> 4.8, 4, 4.9, 4.7, 4.3, 4.4, 4.8, 5, 4.5, 3.5, 3.8, 3.7, 3.9,
#> 5.1, 4.5, 4.5, 4.7, 4.4, 4.1, 4, 4.4, 4.6, 4, 3.3, 4.2, 4.2,
#> 4.2, 4.3, 3, 4.1, 6, 5.1, 5.9, 5.6, 5.8, 6.6, 4.5, 6.3, 5.8,
#> 6.1, 5.1, 5.3, 5.5, 5, 5.1, 5.3, 5.5, 6.7, 6.9, 5, 5.7, 4.9,
#> 6.7, 4.9, 5.7, 6, 4.8, 4.9, 5.6, 5.8, 6.1, 6.4, 5.6, 5.1,
#> 5.6, 6.1, 5.6, 5.5, 4.8, 5.4, 5.6, 5.1, 5.1, 5.9, 5.7, 5.2,
#> 5, 5.2, 5.4, 5.1), c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
#> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
#> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3,
#> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
#> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
#> 3, 3, 3, 3, 3))), INF_FUN = function (model, newdata = NULL,
#> variables = NULL, comparison = "difference", type = NULL,
#> vcov = TRUE, by = FALSE, conf_level = 0.95, transform = NULL,
#> cross = FALSE, wts = FALSE, hypothesis = NULL, equivalence = NULL,
#> p_adjust = NULL, df = Inf, eps = NULL, numderiv = "fdforward",
#> ...)
#> {
#> dots <- list(...)
#> if ("transform_post" %in% names(dots)) {
#> transform <- dots[["transform_post"]]
#> insight::format_warning("The `transform_post` argument is deprecated. Use `transform` instead.")
#> }
#> if ("transform_pre" %in% names(dots)) {
#> comparison <- dots[["transform_pre"]]
#> insight::format_warning("The `transform_pre` argument is deprecated. Use `comparison` instead.")
#> }
#> scall <- rlang::enquo(newdata)
#> newdata <- sanitize_newdata_call(scall, newdata, model, by = by)
#> if (isTRUE(by)) {
#> modeldata <- get_modeldata(model, additional_variables = FALSE,
#> modeldata = dots[["modeldata"]], wts = wts)
#> }
#> else {
#> modeldata <- get_modeldata(model, additional_variables = by,
#> modeldata = dots[["modeldata"]], wts = wts)
#> }
#> call_attr <- c(list(name = "comparisons", model = model,
#> newdata = newdata, variables = variables, type = type,
#> vcov = vcov, by = by, conf_level = conf_level, comparison = comparison,
#> transform = transform, cross = cross, wts = wts, hypothesis = hypothesis,
#> equivalence = equivalence, p_adjust = p_adjust, df = df),
#> dots)
#> if ("modeldata" %in% names(dots)) {
#> call_attr[["modeldata"]] <- modeldata
#> }
#> call_attr <- do.call("call", call_attr)
#> bycols <- NULL
#> sanity_dots(model, ...)
#> sanity_df(df, newdata)
#> conf_level <- sanitize_conf_level(conf_level, ...)
#> checkmate::assert_number(eps, lower = 1e-10, null.ok = TRUE)
#> numderiv <- sanitize_numderiv(numderiv)
#> sanity_equivalence_p_adjust(equivalence, p_adjust)
#> model <- sanitize_model(model = model, newdata = newdata,
#> wts = wts, vcov = vcov, by = by, calling_function = "comparisons",
#> ...)
#> cross <- sanitize_cross(cross, variables, model)
#> type <- sanitize_type(model = model, type = type, calling_function = "comparisons")
#> sanity_comparison(comparison)
#> if (inherits(model, c("mira", "amest"))) {
#> out <- process_imputation(model, call_attr)
#> return(out)
#> }
#> comparison_label <- transform_label <- NULL
#> if (is.function(comparison)) {
#> comparison_label <- deparse(substitute(comparison))
#> }
#> if (is.function(transform)) {
#> transform_label <- deparse(substitute(transform))
#> transform <- sanitize_transform(transform)
#> names(transform) <- transform_label
#> }
#> else {
#> transform <- sanitize_transform(transform)
#> transform_label <- names(transform)
#> }
#> marginalmeans <- isTRUE(checkmate::check_choice(newdata,
#> choices = "marginalmeans"))
#> newdata <- sanitize_newdata(model = model, newdata = newdata,
#> modeldata = modeldata, by = by, wts = wts)
#> sanity_by(by, newdata)
#> newdata <- dedup_newdata(model = model, newdata = newdata,
#> wts = wts, by = by, cross = cross, comparison = comparison)
#> if (isFALSE(wts) && "marginaleffects_wts_internal" %in% colnames(newdata)) {
#> wts <- "marginaleffects_wts_internal"
#> }
#> variables_list <- sanitize_variables(model = model, newdata = newdata,
#> modeldata = modeldata, variables = variables, cross = cross,
#> by = by, comparison = comparison, eps = eps)
#> if (inherits(model, "lmerMod") && (isTRUE(hush(vcov %in%
#> c("satterthwaite", "kenward-roger"))))) {
#> dv <- insight::find_response(model)
#> if (!dv %in% colnames(newdata)) {
#> newdata[[dv]] <- mean(insight::get_response(model))
#> }
#> if (!isTRUE(hush(is.infinite(df)))) {
#> insight::format_error("The `df` argument is not supported when `vcov` is \"satterthwaite\" or \"kenward-roger\".")
#> }
#> df <- insight::get_df(model, type = vcov, data = newdata,
#> df_per_observation = TRUE)
#> }
#> vcov_false <- isFALSE(vcov)
#> vcov.type <- get_vcov_label(vcov)
#> vcov <- get_vcov(model, vcov = vcov, type = type, ...)
#> predictors <- variables_list$conditional
#> out <- inferences_dispatch(INF_FUN = comparisons, model = model,
#> newdata = newdata, vcov = vcov, variables = variables,
#> type = type, by = by, conf_level = conf_level, cross = cross,
#> comparison = comparison, transform = transform, wts = wts,
#> hypothesis = hypothesis, eps = eps, ...)
#> if (!is.null(out)) {
#> return(out)
#> }
#> tmp <- sanitize_hypothesis(hypothesis, ...)
#> hypothesis <- tmp$hypothesis
#> hypothesis_null <- tmp$hypothesis_null
#> args <- list(model = model, newdata = newdata, variables = predictors,
#> cross = cross, marginalmeans = marginalmeans, modeldata = modeldata)
#> dots[["modeldata"]] <- NULL
#> args <- c(args, dots)
#> contrast_data <- do.call("get_contrast_data", args)
#> args <- list(model, newdata = newdata, variables = predictors,
#> type = type, original = contrast_data[["original"]],
#> hi = contrast_data[["hi"]], lo = contrast_data[["lo"]],
#> wts = contrast_data[["original"]][["marginaleffects_wts_internal"]],
#> by = by, marginalmeans = marginalmeans, cross = cross,
#> hypothesis = hypothesis, modeldata = modeldata)
#> args <- c(args, dots)
#> mfx <- do.call("get_contrasts", args)
#> hyp_by <- attr(mfx, "hypothesis_function_by")
#> if (!is.null(attr(mfx, "posterior_draws"))) {
#> draws <- attr(mfx, "posterior_draws")
#> J <- NULL
#> }
#> else if (!vcov_false && isTRUE(checkmate::check_matrix(vcov))) {
#> idx <- intersect(colnames(mfx), c("group", "term", "contrast"))
#> idx <- mfx[, (idx), drop = FALSE]
#> args <- list(model, vcov = vcov, type = type, FUN = get_se_delta_contrasts,
#> newdata = newdata, index = idx, variables = predictors,
#> marginalmeans = marginalmeans, hypothesis = hypothesis,
#> hi = contrast_data$hi, lo = contrast_data$lo, original = contrast_data$original,
#> by = by, eps = eps, cross = cross, numderiv = numderiv)
#> args <- c(args, dots)
#> se <- do.call("get_se_delta", args)
#> J <- attr(se, "jacobian")
#> attr(se, "jacobian") <- NULL
#> mfx$std.error <- as.numeric(se)
#> draws <- NULL
#> }
#> else {
#> J <- draws <- NULL
#> }
#> if ((is.null(by) || isFALSE(by)) && "rowid" %in% colnames(mfx)) {
#> if ("rowid" %in% colnames(newdata)) {
#> idx <- c("rowid", "rowidcf", "term", "contrast",
#> "by", setdiff(colnames(contrast_data$original),
#> colnames(mfx)))
#> idx <- intersect(idx, colnames(contrast_data$original))
#> tmp <- contrast_data$original[, ..idx, drop = FALSE]
#> bycols <- intersect(colnames(tmp), colnames(mfx))
#> idx <- duplicated(tmp, by = bycols)
#> tmp <- tmp[!idx]
#> mfx <- merge(mfx, tmp, all.x = TRUE, by = bycols,
#> sort = FALSE)
#> }
#> else {
#> idx <- setdiff(colnames(contrast_data$original),
#> colnames(mfx))
#> mfx <- data.table(mfx, contrast_data$original[, ..idx])
#> }
#> }
#> mfx <- get_ci(mfx, conf_level = conf_level, df = df, draws = draws,
#> estimate = "estimate", null_hypothesis = hypothesis_null,
#> p_adjust = p_adjust, model = model)
#> mfx <- sort_columns(mfx, newdata, by)
#> attr(mfx, "posterior_draws") <- draws
#> mfx <- equivalence(mfx, equivalence = equivalence, df = df,
#> ...)
#> mfx <- backtransform(mfx, transform)
#> if (!all(is.na(mfx[["marginaleffects_wts_internal"]]))) {
#> marginaleffects_wts_internal <- mfx[["marginaleffects_wts_internal"]]
#> }
#> else {
#> marginaleffects_wts_internal <- NULL
#> }
#> mfx[["marginaleffects_wts_internal"]] <- NULL
#> out <- mfx
#> data.table::setDF(out)
#> out <- set_marginaleffects_attributes(out, get_marginaleffects_attributes(newdata,
#> include_regex = "^newdata.*class|explicit|matrix|levels"))
#> attr(out, "newdata") <- newdata
#> attr(out, "call") <- call_attr
#> attr(out, "type") <- type
#> attr(out, "model_type") <- class(model)[1]
#> attr(out, "model") <- model
#> attr(out, "variables") <- predictors
#> attr(out, "jacobian") <- J
#> attr(out, "vcov") <- vcov
#> attr(out, "vcov.type") <- vcov.type
#> attr(out, "weights") <- marginaleffects_wts_internal
#> attr(out, "comparison") <- comparison
#> attr(out, "transform") <- transform[[1]]
#> attr(out, "comparison_label") <- comparison_label
#> attr(out, "hypothesis_by") <- hyp_by
#> attr(out, "transform_label") <- transform_label
#> attr(out, "conf_level") <- conf_level
#> attr(out, "by") <- by
#> if (inherits(model, "brmsfit")) {
#> insight::check_if_installed("brms")
#> attr(out, "nchains") <- brms::nchains(model)
#> }
#> class(out) <- c("comparisons", class(out))
#> return(out)
#> }, newdata = list(c(5.1, 4.9, 4.7, 4.6, 5, 5.4, 4.6, 5, 4.4,
#> 4.9, 5.4, 4.8, 4.8, 4.3, 5.8, 5.7, 5.4, 5.1, 5.7, 5.1, 5.4, 5.1,
#> 4.6, 5.1, 4.8, 5, 5, 5.2, 5.2, 4.7, 4.8, 5.4, 5.2, 5.5, 4.9,
#> 5, 5.5, 4.9, 4.4, 5.1, 5, 4.5, 4.4, 5, 5.1, 4.8, 5.1, 4.6, 5.3,
#> 5, 7, 6.4, 6.9, 5.5, 6.5, 5.7, 6.3, 4.9, 6.6, 5.2, 5, 5.9, 6,
#> 6.1, 5.6, 6.7, 5.6, 5.8, 6.2, 5.6, 5.9, 6.1, 6.3, 6.1, 6.4, 6.6,
#> 6.8, 6.7, 6, 5.7, 5.5, 5.5, 5.8, 6, 5.4, 6, 6.7, 6.3, 5.6, 5.5,
#> 5.5, 6.1, 5.8, 5, 5.6, 5.7, 5.7, 6.2, 5.1, 5.7, 6.3, 5.8, 7.1,
#> 6.3, 6.5, 7.6, 4.9, 7.3, 6.7, 7.2, 6.5, 6.4, 6.8, 5.7, 5.8, 6.4,
#> 6.5, 7.7, 7.7, 6, 6.9, 5.6, 7.7, 6.3, 6.7, 7.2, 6.2, 6.1, 6.4,
#> 7.2, 7.4, 7.9, 6.4, 6.3, 6.1, 7.7, 6.3, 6.4, 6, 6.9, 6.7, 6.9,
#> 5.8, 6.8, 6.7, 6.7, 6.3, 6.5, 6.2, 5.9), c(0.2, 0.2, 0.2, 0.2,
#> 0.2, 0.4, 0.3, 0.2, 0.2, 0.1, 0.2, 0.2, 0.1, 0.1, 0.2, 0.4, 0.4,
#> 0.3, 0.3, 0.3, 0.2, 0.4, 0.2, 0.5, 0.2, 0.2, 0.4, 0.2, 0.2, 0.2,
#> 0.2, 0.4, 0.1, 0.2, 0.2, 0.2, 0.2, 0.1, 0.2, 0.2, 0.3, 0.3, 0.2,
#> 0.6, 0.4, 0.3, 0.2, 0.2, 0.2, 0.2, 1.4, 1.5, 1.5, 1.3, 1.5, 1.3,
#> 1.6, 1, 1.3, 1.4, 1, 1.5, 1, 1.4, 1.3, 1.4, 1.5, 1, 1.5, 1.1,
#> 1.8, 1.3, 1.5, 1.2, 1.3, 1.4, 1.4, 1.7, 1.5, 1, 1.1, 1, 1.2,
#> 1.6, 1.5, 1.6, 1.5, 1.3, 1.3, 1.3, 1.2, 1.4, 1.2, 1, 1.3, 1.2,
#> 1.3, 1.3, 1.1, 1.3, 2.5, 1.9, 2.1, 1.8, 2.2, 2.1, 1.7, 1.8, 1.8,
#> 2.5, 2, 1.9, 2.1, 2, 2.4, 2.3, 1.8, 2.2, 2.3, 1.5, 2.3, 2, 2,
#> 1.8, 2.1, 1.8, 1.8, 1.8, 2.1, 1.6, 1.9, 2, 2.2, 1.5, 1.4, 2.3,
#> 2.4, 1.8, 1.8, 2.1, 2.4, 2.3, 1.9, 2.3, 2.5, 2.3, 1.9, 2, 2.3,
#> 1.8), c(1.4, 1.4, 1.3, 1.5, 1.4, 1.7, 1.4, 1.5, 1.4, 1.5, 1.5,
#> 1.6, 1.4, 1.1, 1.2, 1.5, 1.3, 1.4, 1.7, 1.5, 1.7, 1.5, 1, 1.7,
#> 1.9, 1.6, 1.6, 1.5, 1.4, 1.6, 1.6, 1.5, 1.5, 1.4, 1.5, 1.2, 1.3,
#> 1.4, 1.3, 1.5, 1.3, 1.3, 1.3, 1.6, 1.9, 1.4, 1.6, 1.4, 1.5, 1.4,
#> 4.7, 4.5, 4.9, 4, 4.6, 4.5, 4.7, 3.3, 4.6, 3.9, 3.5, 4.2, 4,
#> 4.7, 3.6, 4.4, 4.5, 4.1, 4.5, 3.9, 4.8, 4, 4.9, 4.7, 4.3, 4.4,
#> 4.8, 5, 4.5, 3.5, 3.8, 3.7, 3.9, 5.1, 4.5, 4.5, 4.7, 4.4, 4.1,
#> 4, 4.4, 4.6, 4, 3.3, 4.2, 4.2, 4.2, 4.3, 3, 4.1, 6, 5.1, 5.9,
#> 5.6, 5.8, 6.6, 4.5, 6.3, 5.8, 6.1, 5.1, 5.3, 5.5, 5, 5.1, 5.3,
#> 5.5, 6.7, 6.9, 5, 5.7, 4.9, 6.7, 4.9, 5.7, 6, 4.8, 4.9, 5.6,
#> 5.8, 6.1, 6.4, 5.6, 5.1, 5.6, 6.1, 5.6, 5.5, 4.8, 5.4, 5.6, 5.1,
#> 5.1, 5.9, 5.7, 5.2, 5, 5.2, 5.4, 5.1), c(1, 1, 1, 1, 1, 1, 1,
#> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
#> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
#> 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
#> 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
#> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
#> 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3), 1:150), vcov = c(0.0648561358708614,
#> -0.0776075989705507, -0.0339713425276863, 0.0855221415040715,
#> 0.088949330395953, 0.0170000905579961, -0.0776075989705507, 0.144241283857235,
#> 0.0351858298835599, -0.117761642231515, -0.122089476746883, -0.0254853395735601,
#> -0.0339713425276863, 0.0351858298835599, 0.0194905399614243,
#> -0.0459657347451442, -0.0472861136127865, -0.00869677529341257,
#> 0.0855221415040715, -0.117761642231515, -0.0459657347451442,
#> 0.138337982006763, 0.15072216454488, 0.0227990421926966, 0.088949330395953,
#> -0.122089476746883, -0.0472861136127865, 0.15072216454488, 0.179060079636581,
#> 0.0216170189525216, 0.0170000905579961, -0.0254853395735601,
#> -0.00869677529341257, 0.0227990421926966, 0.0216170189525216,
#> 0.00542666426714662), variables = "Petal.Width", type = "response",
#> by = "Species", conf_level = 0.95, cross = FALSE, comparison = "difference",
#> transform = NULL, wts = FALSE, hypothesis = NULL, eps = NULL)
#>
#>
#> Bootstrap Statistics :
#> original bias std. error
#> t1* -0.11025325 0.0053204568 0.2697800
#> t2* -0.02006005 0.0021095913 0.1680026
#> t3* 0.02158742 0.0006269471 0.1880918Or we can extract the individual draws with the posterior_draws() function:
posterior_draws(est) |> head()
#> drawid draw term contrast Species estimate predicted_lo predicted_hi predicted std.error conf.low conf.high
#> 1 1 -0.52203326 Petal.Width mean(+1) setosa -0.11025325 4.957514 4.845263 4.957514 0.2697800 -0.6648142 0.3944026
#> 2 1 -0.15430494 Petal.Width mean(+1) versicolor -0.02006005 6.327949 6.322072 6.327949 0.1680026 -0.3422048 0.3214168
#> 3 1 0.01549669 Petal.Width mean(+1) virginica 0.02158742 7.015513 7.051542 7.015513 0.1880918 -0.3600655 0.3686412
#> 4 2 -0.01530060 Petal.Width mean(+1) setosa -0.11025325 4.957514 4.845263 4.957514 0.2697800 -0.6648142 0.3944026
#> 5 2 0.11934354 Petal.Width mean(+1) versicolor -0.02006005 6.327949 6.322072 6.327949 0.1680026 -0.3422048 0.3214168
#> 6 2 0.18151660 Petal.Width mean(+1) virginica 0.02158742 7.015513 7.051542 7.015513 0.1880918 -0.3600655 0.3686412
posterior_draws(est, shape = "DxP") |> dim()
#> [1] 500 3
16.2.2 rsample
As before, we can pass arguments to rsample::bootstraps() through inferences(). For example, for stratified resampling:
est <- avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
inferences(method = "rsample", R = 100, strata = "Species")
est
#>
#> Species Estimate 2.5 % 97.5 %
#> setosa -0.1103 -0.602 0.309
#> versicolor -0.0201 -0.320 0.261
#> virginica 0.0216 -0.377 0.360
#>
#> Term: Petal.Width
#> Type: response
#> Comparison: mean(+1)
#> Columns: term, contrast, Species, estimate, predicted_lo, predicted_hi, predicted, conf.low, conf.high
attr(est, "inferences")
#> # Bootstrap sampling using stratification with apparent sample
#> # A tibble: 101 × 3
#> splits id estimates
#> <list> <chr> <list>
#> 1 <split [150/62]> Bootstrap001 <tibble [3 × 7]>
#> 2 <split [150/56]> Bootstrap002 <tibble [3 × 7]>
#> 3 <split [150/54]> Bootstrap003 <tibble [3 × 7]>
#> 4 <split [150/58]> Bootstrap004 <tibble [3 × 7]>
#> 5 <split [150/60]> Bootstrap005 <tibble [3 × 7]>
#> 6 <split [150/61]> Bootstrap006 <tibble [3 × 7]>
#> 7 <split [150/52]> Bootstrap007 <tibble [3 × 7]>
#> 8 <split [150/53]> Bootstrap008 <tibble [3 × 7]>
#> 9 <split [150/57]> Bootstrap009 <tibble [3 × 7]>
#> 10 <split [150/57]> Bootstrap010 <tibble [3 × 7]>
#> # ℹ 91 more rowsOr we can extract the individual draws with the posterior_draws() function:
posterior_draws(est) |> head()
#> drawid draw term contrast Species estimate predicted_lo predicted_hi predicted conf.low conf.high
#> 1 1 -0.10345440 Petal.Width mean(+1) setosa -0.11025325 4.957514 4.845263 4.957514 -0.6017517 0.3086704
#> 2 1 0.12759420 Petal.Width mean(+1) versicolor -0.02006005 6.327949 6.322072 6.327949 -0.3197335 0.2613757
#> 3 1 0.23428283 Petal.Width mean(+1) virginica 0.02158742 7.015513 7.051542 7.015513 -0.3767238 0.3601881
#> 4 2 -0.67406786 Petal.Width mean(+1) setosa -0.11025325 4.957514 4.845263 4.957514 -0.6017517 0.3086704
#> 5 2 -0.08065901 Petal.Width mean(+1) versicolor -0.02006005 6.327949 6.322072 6.327949 -0.3197335 0.2613757
#> 6 2 0.19335251 Petal.Width mean(+1) virginica 0.02158742 7.015513 7.051542 7.015513 -0.3767238 0.3601881
posterior_draws(est, shape = "PxD") |> dim()
#> [1] 3 10016.2.3 Fractional Weighted Bootstrap (aka Bayesian Bootstrap)
The fwb package implements fractional weighted bootstrap (aka Bayesian bootstrap):
“fwb implements the fractional weighted bootstrap (FWB), also known as the Bayesian bootstrap, following the treatment by Xu et al. (2020). The FWB involves generating sets of weights from a uniform Dirichlet distribution to be used in estimating statistics of interest, which yields a posterior distribution that can be interpreted in the same way the traditional (resampling-based) bootstrap distribution can be.” -Noah Greifer
The inferences() function makes it easy to apply this inference strategy to marginaleffects objects:
avg_comparisons(mod) |> inferences(method = "fwb")
#>
#> Term Contrast Estimate Std. Error 2.5 % 97.5 %
#> Petal.Length mean(+1) 0.8929 0.0803 0.734 1.042
#> Petal.Width mean(+1) -0.0362 0.1536 -0.326 0.278
#> Species mean(versicolor) - mean(setosa) -1.4629 0.3262 -2.150 -0.836
#> Species mean(virginica) - mean(setosa) -1.9842 0.3922 -2.781 -1.236
#>
#> Type: response
#> Columns: term, contrast, estimate, predicted_lo, predicted_hi, predicted, std.error, conf.low, conf.high16.3 Simulation-based inference
This simulation-based strategy to compute confidence intervals was described in Krinsky & Robb (1986) and popularized by King, Tomz, Wittenberg (2000). We proceed in 3 steps:
- Draw
Rsets of simulated coefficients from a multivariate normal distribution with mean equal to the original model’s estimated coefficients and variance equal to the model’s variance-covariance matrix (classical, “HC3”, or other). - Use the
Rsets of coefficients to computeRsets of estimands: predictions, comparisons, or slopes. - Take quantiles of the resulting distribution of estimands to obtain a confidence interval and the standard deviation of simulated estimates to estimate the standard error.
Here are a few examples:
library(ggplot2)
library(ggdist)
avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
inferences(method = "simulation")
#>
#> Species Estimate 2.5 % 97.5 %
#> setosa -0.1103 -0.662 0.426
#> versicolor -0.0201 -0.328 0.297
#> virginica 0.0216 -0.303 0.357
#>
#> Term: Petal.Width
#> Type: response
#> Comparison: mean(+1)
#> Columns: term, contrast, Species, estimate, predicted_lo, predicted_hi, predicted, conf.low, conf.highSince simulation based inference generates R estimates of the quantities of interest, we can treat them similarly to draws from the posterior distribution in bayesian models. For example, we can extract draws using the posterior_draws() function, and plot their distributions using packages likeggplot2 and ggdist:
avg_comparisons(mod, by = "Species", variables = "Petal.Width") |>
inferences(method = "simulation") |>
posterior_draws("rvar") |>
ggplot(aes(y = Species, xdist = rvar)) +
stat_slabinterval()
16.4 Multiple imputation and missing data
The same workflow and the same inferences function can be used to estimate models with multiple imputation for missing data.