Performance

What to do when marginaleffects is slow?

Some options:

  1. Compute marginal effects and contrasts at the mean (or other representative value) instead of all observed rows of the original dataset: Use the newdata argument and the datagrid() function.
  2. Compute marginal effects for a subset of variables, paying special attention to exclude factor variables which can be particularly costly to process: Use the variables argument.
  3. Do not compute standard errors: Use the vcov = FALSE argument.

This simulation illustrates how computation time varies for a model with 25 regressors and 100,000 observations:

library(marginaleffects)

## simulate data and fit a large model
N <- 1e5
dat <- data.frame(matrix(rnorm(N * 26), ncol = 26))
mod <- lm(X1 ~ ., dat)

results <- bench::mark(
    # marginal effects at the mean; no standard error
    slopes(mod, vcov = FALSE, newdata = "mean"),
    # marginal effects at the mean
    slopes(mod, newdata = "mean"),
    # 1 variable; no standard error
    slopes(mod, vcov = FALSE, variables = "X3"),
    # 1 variable
    slopes(mod, variables = "X3"),
    # 26 variables; no standard error
    slopes(mod, vcov = FALSE),
    # 26 variables
    slopes(mod),
    iterations = 1, check = FALSE)

results[, c(1, 3, 5)]
# expression                                        median mem_alloc
# "slopes(mod, vcov = FALSE, newdata = \"mean\")" 194.98ms  306.19MB
# "slopes(mod, newdata = \"mean\")"               345.38ms  311.45MB
# "slopes(mod, vcov = FALSE, variables = \"X3\")" 197.51ms   649.6MB
# "slopes(mod, variables = \"X3\")"               742.05ms    1.27GB
# "slopes(mod, vcov = FALSE)"                        4.09s   13.87GB
# "slopes(mod)"                                     15.33s   26.83GB

The benchmarks above were conducted using the development version of marginaleffects on 2023-12-09.

Speed comparison

The slopes function is relatively fast. This simulation was conducted using the development version of the package on 2023-12-09:

library(margins)

N <- 1e3
dat <- data.frame(
    y = sample(0:1, N, replace = TRUE),
    x1 = rnorm(N),
    x2 = rnorm(N),
    x3 = rnorm(N),
    x4 = factor(sample(letters[1:5], N, replace = TRUE)))
mod <- glm(y ~ x1 + x2 + x3 + x4, data = dat, family = binomial)

marginaleffects can be 3 times faster and use 3 times less memory than margins when unit-level standard errors are not computed:

results <- bench::mark(
    slopes(mod, vcov = FALSE),
    margins(mod, unit_ses = FALSE),
    check = FALSE, relative = TRUE)
results[, c(1, 3, 5)]

# expression                     median mem_alloc
# <bch:expr>                      <dbl>     <dbl>
# slopes(mod, vcov = FALSE)        1         1   
# margins(mod, unit_ses = FALSE)   3.21      2.83

marginaleffects can be up to 1000x times faster and use 32x less memory than margins when unit-level standard errors are computed:

results <- bench::mark(
    slopes(mod, vcov = TRUE),
    margins(mod, unit_ses = TRUE),
    check = FALSE, relative = TRUE, iterations = 1)
results[, c(1, 3, 5)]
# expression                    median mem_alloc
#  <bch:expr>                     <dbl>     <dbl>
#  slopes(mod, vcov = TRUE)          1        1  
#  margins(mod, unit_ses = TRUE)  1161.      32.9

Models estimated on larger datasets (> 1000 observations) can be impossible to process using the margins package, because of memory and time constraints. In contrast, marginaleffects can work well on much larger datasets.

Note that, in some specific cases, marginaleffects will be considerably slower than packages like emmeans or modmarg. This is because these packages make extensive use of hard-coded analytical derivatives, or reimplement their own fast prediction functions.