DALEX::plot.model_performance() rejects list of model_performance objects

[agilebean]

The DALEX::plot.model_performance() has an extremely useful option that plots several model_performance objects as described in the book Explanatory Model Analysis:

plot(mr_lm, mr_rf, geom = "histogram")

This is extremely useful.

plot(xai.list.DALEX$gbm$performance.DALEX,
     xai.list.DALEX$rf$performance.DALEX,
     geom = "histogram")

shows

However, the plot function rejects a list of model objects:

xai.list.DALEX %>%
  map("performance.DALEX") %>% 
  plot(geom = "histogram")

The documentation for DALEX::plot.model_performance says about the input:
x a model to be explained

Big question:

Is it somehow possible to input a list of model_performance objects to plot()?
Either by extending DALEX::plot.model_performance, or by transforming the list of model_performance objects somehow?

As the model_performance objects are mostly assembled in a list,
would be very useful for lots of use cases!

[pbiecek]

@agilebean please try 6751ac9
now the plot function should work for lists (not only performance explanations but all other explanations as well).

[agilebean]

@pbiecek thanks so much for the commit so fast.

I tried it out now but unfortunately get this error:

Error in plot.model_performance(lm = list(residuals = list(predicted = c(1939.5536628618,  : 
  argument "x" is missing, with no default

However, I finally create a reprex:

libs <- c("DALEX", "tidyverse", "caret")
SEED <- 171

dataset <- diamonds %>%
  select(is.numeric) %>%
  sample_n(1000)

target <- dataset$price
features <- dataset %>% select(-price)

algorithm.labels <- c("lm", "knn", "gbm", "rf")

plan(multicore, workers = 8)

mlist <- algorithm.labels %>%
  future_map(
    ~ caret::train(
      method = .x,
      x = features,
      y = target,
      preProcess = c("center", "scale"),
      trControl = trainControl(method = "cv", number = 5)
    )
  ) %>%
  set_names(algorithm.labels)

xai.list <- mlist %>%
  future_map(
    ~.x %>%
      DALEX::explain(
      data = features,
      y = .x$trainingData$.outcome >= (target %>% quantile(.95)),
      label = paste(.x$method, " model"),
      colorize = TRUE
    ) %>%
      DALEX::model_performance(),
    .options = furrr_options(seed = SEED)
  )

xai.list %>% plot()

If plan(multicore) doesn't work for you, just replace it by plan(multisession).
Or just use map instead of future_map if you can wait.

[pbiecek]

Thank you,
it turns out that list with arguments should not be named,
I've just commited a hotfix, it works for the example above

Let me know if there are any issues with this fix

[agilebean]

@pbiecek thanks, i tested the hotfix.

xai.list %>% plot()
now shows the cumulative distribution of residuals - very nice:

However,
xai.list %>% plot(geom = "histogram")
or
xai.list %>% plot(geom = "boxplot")
throws
Error: $ operator is invalid for atomic vectors

[pbiecek]

sorry for this,
some names needs to be kept
this one: 95b6040
is working for both scenatios

[agilebean]

@pbiecek
great!, it works like a charm now!

xai.list.DALEX %>% plot(geom = "histogram")
plots

really nice work!