Skip to content

Commit

Permalink
Initial implementation of a generic json_reader function and download…
Browse files Browse the repository at this point in the history
…_JSON method
  • Loading branch information
RichardMN committed Sep 16, 2021
1 parent aa85f36 commit abf4cdd
Show file tree
Hide file tree
Showing 29 changed files with 100 additions and 1 deletion.
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -121,4 +121,4 @@ Encoding: UTF-8
Language: en-gb
LazyData: true
Roxygen: list(markdown = TRUE)
RoxygenNote: 7.1.1
RoxygenNote: 7.1.2
1 change: 1 addition & 0 deletions NAMESPACE
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ importFrom(httr,GET)
importFrom(httr,POST)
importFrom(httr,content)
importFrom(httr,status_code)
importFrom(jsonlite,fromJSON)
importFrom(lifecycle,deprecate_warn)
importFrom(lifecycle,deprecated)
importFrom(lifecycle,is_present)
Expand Down
14 changes: 14 additions & 0 deletions R/shared-methods.R
Original file line number Diff line number Diff line change
Expand Up @@ -302,6 +302,20 @@ DataClass <- R6::R6Class(
)
},

#' @description Download raw data from `data_urls`, stores a named list
#' of the `data_url` name and the corresponding raw data table in
#' `data$raw`. Designed as a drop-in replacement for `download` so
#' it can be used in sub-classes.
#' @importFrom purrr map
download_JSON = function() {
if (length(self$data_urls) == 0) {
stop("No data to download as data_urls is empty")
}
self$data$raw <- map(self$data_urls, json_reader,
verbose = self$verbose
)
},

#' @description Cleans raw data (corrects format, converts column types,
#' etc). Works on raw data and so should be called after
#' \href{#method-download}{\code{download()}}
Expand Down
24 changes: 24 additions & 0 deletions R/utils.R
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,30 @@ csv_reader <- function(file, verbose = FALSE, guess_max = 1000, ...) {
return(tibble(data))
}

#' Custom JSON reading function
#'
#' @description Checks for use of memoise and then uses vroom::vroom.
#' @param file A URL or filepath to a JSON
#' @param ... extra parameters to be passed to jsonlite::fromJSON
#' @inheritParams message_verbose
#' @return A data table
#' @importFrom tibble tibble
#' @importFrom jsonlite fromJSON
#' @concept utility
json_reader <- function(file, verbose = FALSE, ...) {
if (verbose) {
message("Downloading data from ", file)
data <- fromJSON(file, ...)
} else {
data <- suppressWarnings(
suppressMessages(
fromJSON(file, ...)
)
)
}
return(tibble(data))
}

#' Wrapper for message
#'
#' @description A wrapper for `message` that only prints output when
Expand Down
1 change: 1 addition & 0 deletions man/Belgium.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/Brazil.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/Canada.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/Colombia.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/CountryDataClass.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/Covid19DataHub.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/Cuba.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions man/DataClass.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/ECDC.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/France.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions man/Germany.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit abf4cdd

Please sign in to comment.