Setup

The source for all the repositories is located at https://carpentries.github.io/curriculum-feed/carpentries_lessons.json. To avoid having to pull from it multiple times I’m going to set up a folder:

library("fs")       # Filesystem navigation
library("jsonlite") # parsing JSON files
library("purrr")    # handling lists (JSON files)
library("dplyr")    # handling data frames and magic
library("tidyr")    # separating values
library("ggplot2")  # visualization
library("forcats")  # ordering factors
library("magrittr") # for the %T>% pipe I love so well
library("gh")       # accessing GitHub's API
library("polite")   # being respectful when downloading files
library("here")     # so I can always remember where I started
library("git2r")    # downloading github repositories
library("waldo")    # comparing objects
# Github packages
# remotes::install_github("carpentries/pegboard")
# remotes::install_github("fmichonneau/carpenter")
library("pegboard")  # parsing and analysis of carpentries episodes
library("carpenter") # create an issue

raw_data <- here::here("data", "raw")
raw_lessons <- fs::path(raw_data, "carpentries_lessons.json")
if (fs::file_exists(raw_lessons)) {
  
} else {
  fs::dir_create(raw_data)
  res <- download.file(
    url = "https://carpentries.github.io/curriculum-feed/carpentries_lessons.json",
    destfile = raw_lessons
  )
  if (res == 0L) {
    fs::file_chmod(raw_lessons, "a-wx")
  }
}

## NULL

lessons <- read_json(raw_lessons, simplifyVector = FALSE) %>%
  purrr::map_dfr(.f = dplyr::as_tibble) %>%
  dplyr::mutate_if(is.character, 
    ~dplyr::if_else(. == "NA", as.character(NA), .)) %>%
  dplyr::mutate(life_cycle = forcats::fct_relevel(life_cycle,
      "pre-alpha", "alpha", "beta", "stable", "on-hold", "deprecated"
    )) %>%
  dplyr::mutate(life_cycle = forcats::fct_explicit_na(life_cycle)) %>%
  print()

## # A tibble: 88 x 6
##    repository        curriculum   life_cycle language `_row` URL                
##    <chr>             <chr>        <fct>      <chr>    <chr>  <chr>              
##  1 capstone-novice-… <NA>         pre-alpha  en       1      <NA>               
##  2 lc-spreadsheets   lc           beta       en       2      https://github.com…
##  3 rr-automation     reproducibl… on-hold    en       3      https://github.com…
##  4 lesson-example    <NA>         stable     en       4      https://github.com…
##  5 git-novice-es     swc-es       stable     es       5      https://github.com…
##  6 shell-economics   dc-economics pre-alpha  en       6      https://github.com…
##  7 openrefine-ecolo… dc-ecology   stable     en       7      https://github.com…
##  8 python-humanitie… <NA>         pre-alpha  en       8      <NA>               
##  9 hg-novice         swc          stable     en       9      https://github.com…
## 10 spreadsheet-huma… <NA>         pre-alpha  en       10     <NA>               
## # … with 78 more rows

First look at available lessons

I’m going to take a look at all of the software carpentry lessons first. There are currently 88 available lessons, but they are in different stages of completion:

ggplot(lessons, aes(y = life_cycle, fill = curriculum)) +
  geom_bar(orientation = "y")

Let’s inspect the ones that are stable and then look at the ones that are lower down, first we should filter for those that are stable and actually have GitHub URLs:

stable <- lessons %>% filter(
  life_cycle == "stable", 
  curriculum != "instructor-training",
  !is.na(curriculum), # these are template repositories
  !is.na(URL)         # no GitHub URL is not very useful for me
)
stable

## # A tibble: 36 x 6
##    repository       curriculum life_cycle language `_row` URL                   
##    <chr>            <chr>      <fct>      <chr>    <chr>  <chr>                 
##  1 git-novice-es    swc-es     stable     es       5      https://github.com/sw…
##  2 openrefine-ecol… dc-ecology stable     en       7      https://github.com/da…
##  3 hg-novice        swc        stable     en       9      https://github.com/sw…
##  4 matlab-novice-i… swc        stable     en       11     https://github.com/sw…
##  5 r-novice-inflam… swc        stable     en       12     https://github.com/sw…
##  6 spreadsheet-eco… dc-ecology stable     en       13     https://github.com/da…
##  7 python-novice-g… swc        stable     en       16     https://github.com/sw…
##  8 lc-data-intro    lc         stable     en       19     https://github.com/Li…
##  9 R-ecology-lesson dc-ecology stable     en       20     https://github.com/da…
## 10 git-novice       swc        stable     en       24     https://github.com/sw…
## # … with 26 more rows

I end up with 36 repositories to play with:

ggplot(stable, aes(y = curriculum, fill = language)) +
  geom_bar(orientation = "y")

Inspection of repository features

I will use the {gh} package to inspect the features of each repository:

tags
directory structure
dependencies

I can use the GitHub API to get the contents of the repositories: https://developer.github.com/v3/repos/contents/#get-contents. The {gh} package allows me to write the responses to disk so that I don’t have to query every time I want to re-run the analysis. If I wanted to do this with fresh data, all I would need to do is to clear the data folder.

One of the tripping points here is that not all of the repositories will have _episodes_rmd/ directories, so I will need to walk over these with purrr::possibly(), a nice little failsafe function.

safegh <- purrr::possibly(gh::gh, otherwise = list(NA))

rmd_episodes <- function(owner, repo) {
  safegh <- purrr::slowly(
    f    = purrr::possibly(gh::gh, otherwise = list(NA)),
    rate = rate_delay(pause = 2)
  )
  OR <- glue::glue("{owner}--{repo}")
  safegh("/repos/:owner/:repo/contents/_episodes_rmd", 
    owner = owner, 
    repo = repo, 
    .destfile = here::here(fs::path("data", "rmd_JSON", OR, ext = "json"))
  )
}

vorhees <- purrr::possibly(jsonlite::read_json, otherwise = list())

lesson_has_rmd <- . %>%
  tidyr::separate(URL, into = c(NA, NA, NA, "user", NA), sep = "/", remove = FALSE) %>%
  mutate(user = purrr::walk2(.x = user, .y = repository, .f = rmd_episodes)) %>%
  mutate(JSON = purrr::map2(.x = user, .y = repository,
    .f = ~vorhees(
      here::here("data", "rmd_JSON", glue::glue("{.x}--{.y}.json"))
    )
  )) %>%
  filter(repository == "R-ecology-lesson" | lengths(JSON) > 2)
has_rmd <- stable %>% lesson_has_rmd
has_rmd_all <- lessons %>% 
  lesson_has_rmd %>% 
  filter(repository != "lesson-example") %>%
  arrange(life_cycle)

## Warning: Expected 5 pieces. Additional pieces discarded in 1 rows [6].

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--capstone-novice-spreadsheet-
## biblio.json': No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--python-humanities-
## lesson.json': No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--spreadsheet-humanities-
## lesson.json': No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--R-genomics.json': No such
## file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--geospatial-python.json': No
## such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--deep-learning_intro.json':
## No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--docker-introduction.json':
## No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--sql-humanities-lesson.json':
## No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--NA.json': No such file or
## directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--Data-Science-for-Docs.json':
## No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--gap-lesson.json': No such
## file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--git-Rstudio-course.json': No
## such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--jupyter-notebooks-
## intro.json': No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--shell-extras.json': No such
## file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--NA.json': No such file or
## directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--NA.json': No such file or
## directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/
## Documents/Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--python-packaging-
## publishing.json': No such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--NA.json': No such file or
## directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--jupyter_maps.json': No such
## file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--IoT_arduino_nano.json': No
## such file or directory

## Warning in open.connection(con, "rb"): cannot open file '/home/zhian/Documents/
## Carpentries/Git/zkamvar--postmaul/data/rmd_JSON/NA--OpenRefine-humanities-
## lesson.json': No such file or directory

After parsing, we find that we have RMarkdown files for a grand total of 16 lessons and 8 stable lessons. From here, we can grab these lessons to see if we can build them with our docker container.

Lessons that use R

Below is a table with all the lessons the use R in our curriculum. There are links within the table that take you further down this document to display a diff between output from R 3.6 and R 4.0. One issue with these diffs is that sometimes the ordering of messages shifts the entire diff, but for the most part, it’s clear where they came from.

has_rmd_all %>% 
  mutate(results = glue::glue("[results](#lesson-{repository})")) %>%
  select(URL, curriculum, life_cycle, language, results) %>%
  knitr::kable()

URL	curriculum	life_cycle	language	results
https://github.com/LibraryCarpentry/lc-r	lc	pre-alpha	en	results
https://github.com/datacarpentry/genomics-r-intro	dc-genomics	alpha	en	results
https://github.com/swcarpentry/r-novice-inflammation	swc	stable	en	results
https://github.com/datacarpentry/R-ecology-lesson	dc-ecology	stable	en	results
https://github.com/swcarpentry/r-novice-gapminder	swc	stable	en	results
https://github.com/datacarpentry/organization-geospatial	dc-geospatial	stable	en	results
https://github.com/swcarpentry/r-novice-gapminder-es	swc-es	stable	es	results
https://github.com/datacarpentry/r-intro-geospatial	dc-geospatial	stable	en	results
https://github.com/datacarpentry/r-raster-vector-geospatial	dc-geospatial	stable	en	results
https://github.com/datacarpentry/r-socialsci	dc-socsci	stable	en	results
https://github.com/datacarpentry/rr-automation	reproducible-science	on-hold	en	results
https://github.com/datacarpentry/rr-publication	reproducible-science	on-hold	en	results
https://github.com/datacarpentry/rr-intro	reproducible-science	on-hold	en	results
https://github.com/datacarpentry/rr-version-control	reproducible-science	on-hold	en	results
https://github.com/datacarpentry/rr-organization1	reproducible-science	on-hold	en	results
https://github.com/datacarpentry/rr-literate-programming	reproducible-science	on-hold	en	results

The command to use is:

docker run --rm -it -e USER=$(id -u) -e GROUP=$(id -g) -v ${PWD}:/home/rstudio rocker/verse:4.0.0 make -C /home/rstudio lesson-md

Let’s download these repos:

rmdpath <- path("data", "rmd-repos")
if (!dir_exists(rmdpath)) {
  dir_create(path = rmdpath)
}
g <- function(u, r, path = rmdpath) {
  URL <- glue::glue("https://github.com/{u}/{r}.git")
  path <- glue::glue("{path}/{u}--{r}R{c(3, 4)}")
  if (fs::dir_exists(path[[1]])) {
    git2r::pull(repo = path[[1]])
    git2r::pull(repo = path[[2]])
  } else {
    git2r::clone(URL, local_path = path[[1]])
    git2r::clone(URL, local_path = path[[2]])
  }
}
purrr::walk2(has_rmd_all$user, has_rmd_all$repository, g, rmdpath) 
standard_rmd <- has_rmd_all %>% filter(repository != "R-ecology-lesson")

Now all the repos have been downloaded, we can render the episodes under different versions of R and see how they go. Note that I have to use the geospatial R container in order to get things to work properly.

get_path <- function(u, r) {
  fs::path_abs(fs::path("data", "rmd-repos", glue::glue("{u}--{r}")))
}

run_docker <- function(the_path, R_VERSION, cmd = '-C /home/rstudio lesson-md') {
  make____it <- glue::glue("install2.r checkpoint && make -B -j 4 {cmd}")
  docker_run <- "docker run --rm -it -e USER=$(id -u) -e GROUP=$(id -g) -v {the_path}:/home/rstudio"
  contai_ner <- "rocker/geospatial:{R_VERSION} /bin/bash -c '{make____it}'"
  system(glue::glue("{glue::glue(docker_run)} {glue::glue(contai_ner)}"))
}

dockin <- function(u, r) {

  the_path <- get_path(u, r)

  # Run with R 3.6.3 ---------------------------
  r3_path <- glue::glue("{the_path}R3")
  if (params$build_md) run_docker(r3_path, "3.6.3")
  r3 <- try(Lesson$new(r3_path, rmd = FALSE))

  # Run with R 4.0.0 ---------------------------
  r4_path <- glue::glue("{the_path}R4")
  if (params$build_md) run_docker(r4_path, "4.0.0")
  r4 <- try(Lesson$new(r4_path, rmd = FALSE))

  list(r3, r4)

}

res <- purrr::map2(standard_rmd$user, standard_rmd$repository, dockin)

## Error: PCDATA invalid Char value 27 [9]
## Error: PCDATA invalid Char value 27 [9]

names(res) <- standard_rmd$repository

We can iterate and compare:

if (!dir_exists(path("data", "diffs"))) {
  dir_create(path("data", "diffs"))
}
cmpr <- function(lesson, name) {

  cat(glue::glue("\n## Lesson: {name}\n\n"))
  if (!inherits(lesson[[1]], "Lesson")) {
    cat("\n\n----ERRORED----\n\n")
    return(invisible(NULL))
  }

  o1 <- map(lesson[[1]]$episodes, ~setNames(xml2::xml_text(.x$output), xml2::xml_attr(.x$output, "sourcepos")))
  o2 <- map(lesson[[2]]$episodes, ~setNames(xml2::xml_text(.x$output), xml2::xml_attr(.x$output, "sourcepos")))

  for (i in seq_along(lesson[[1]]$episodes)) {
    episode <- glue::glue("{name}--{basename(lesson[[1]]$files[i])}")
    dpath <- path("data", "diffs", sub("md$", "diff", episode))
    diffcmd <- "git diff --no-index -- {lesson[[1]]$files[[i]]} {lesson[[2]]$files[[i]]} > {dpath}"
    system(glue::glue(diffcmd))
    cat(glue::glue("\n#### Episode: {episode}\n"))
    cat(glue::glue("\n\n[Link to full diff]({dpath})\n"))
    cat("\n```diff\n")
    print(waldo::compare(o2[[i]], o1[[i]], x_arg = "Rv4", y_arg = "Rv3"))
    cat("\n```\n")
  }
}

purrr::walk2(res, names(res), cmpr)

Files

analysis.md

Latest commit

History

analysis.md

File metadata and controls

Setup

First look at available lessons

Inspection of repository features

Lessons that use R

Lesson: lc-r

Episode: lc-r–00-before-we-start.md

Episode: lc-r–01-intro-to-r.md

Episode: lc-r–02-starting-with-data.md

Episode: lc-r–03-data-cleaning-and-transformation.md

Episode: lc-r–04-data-viz-ggplot.md

Lesson: genomics-r-intro

Episode: genomics-r-intro–01-introduction.md

Episode: genomics-r-intro–02-r-basics.md

Episode: genomics-r-intro–03-basics-factors-dataframes.md

Warning in diff_myers(args[[“a”]], args[[“b”]], max.diffs =

args[[“max.diffs”]], : Exceeded max.diffs: 29 vs 100 allowed. Diff is probably

suboptimal.

Warning in diff_myers(args[[“a”]], args[[“b”]], max.diffs =

args[[“max.diffs”]], : Exceeded max.diffs: 61 vs 100 allowed. Diff is probably

suboptimal.

Episode: genomics-r-intro–04-dplyr.md

Episode: genomics-r-intro–05-data-visualization.md

Episode: genomics-r-intro–XX-knitr-markdown.md

Lesson: r-novice-inflammation

Episode: r-novice-inflammation–01-starting-with-data.md

Episode: r-novice-inflammation–02-func-R.md

Episode: r-novice-inflammation–03-loops-R.md

Episode: r-novice-inflammation–04-cond.md

Episode: r-novice-inflammation–05-cmdline.md

Episode: r-novice-inflammation–06-best-practices-R.md

Episode: r-novice-inflammation–07-knitr-R.md

Episode: r-novice-inflammation–08-making-packages-R.md

Episode: r-novice-inflammation–09-supp-intro-rstudio.md

Episode: r-novice-inflammation–10-supp-addressing-data.md

Episode: r-novice-inflammation–11-supp-read-write-csv.md

Episode: r-novice-inflammation–12-supp-factors.md

Episode: r-novice-inflammation–13-supp-data-structures.md

Episode: r-novice-inflammation–14-supp-call-stack.md

Episode: r-novice-inflammation–15-supp-loops-in-depth.md

Lesson: r-novice-gapminder

Episode: r-novice-gapminder–01-rstudio-intro.md

Episode: r-novice-gapminder–02-project-intro.md

Episode: r-novice-gapminder–03-seeking-help.md

Episode: r-novice-gapminder–04-data-structures-part1.md

Warning in diff_myers(args[[“a”]], args[[“b”]], max.diffs =

args[[“max.diffs”]], : Exceeded max.diffs: 33 vs 100 allowed. Diff is probably

suboptimal.

Episode: r-novice-gapminder–05-data-structures-part2.md

Episode: r-novice-gapminder–06-data-subsetting.md

Episode: r-novice-gapminder–07-control-flow.md

Episode: r-novice-gapminder–08-plot-ggplot2.md

Episode: r-novice-gapminder–09-vectorization.md

Episode: r-novice-gapminder–10-functions.md

Episode: r-novice-gapminder–11-writing-data.md

Episode: r-novice-gapminder–12-plyr.md

Episode: r-novice-gapminder–13-dplyr.md

Episode: r-novice-gapminder–14-tidyr.md

Episode: r-novice-gapminder–15-knitr-markdown.md

Episode: r-novice-gapminder–16-wrap-up.md

Lesson: organization-geospatial

Episode: organization-geospatial–01-intro-raster-data.md

Episode: organization-geospatial–02-intro-vector-data.md

Episode: organization-geospatial–03-crs.md

Episode: organization-geospatial–04-geo-landscape.md

Lesson: r-novice-gapminder-es

Episode: r-novice-gapminder-es–01-rstudio-intro.md

Episode: r-novice-gapminder-es–02-project-intro.md

Episode: r-novice-gapminder-es–03-seeking-help.md

Episode: r-novice-gapminder-es–04-data-structures-part1.md

Warning in diff_myers(args[[“a”]], args[[“b”]], max.diffs =

args[[“max.diffs”]], : Exceeded max.diffs: 16 vs 100 allowed. Diff is probably

suboptimal.

Episode: r-novice-gapminder-es–05-data-structures-part2.md

Episode: r-novice-gapminder-es–06-data-subsetting.md

args[[“max.diffs”]], : Exceeded `max.diffs`: 29 vs 100 allowed. Diff is probably

args[[“max.diffs”]], : Exceeded `max.diffs`: 61 vs 100 allowed. Diff is probably

args[[“max.diffs”]], : Exceeded `max.diffs`: 33 vs 100 allowed. Diff is probably

args[[“max.diffs”]], : Exceeded `max.diffs`: 16 vs 100 allowed. Diff is probably

args[[“max.diffs”]], : Exceeded `max.diffs`: 29 vs 100 allowed. Diff is probably