roxygen2 6.1.0 discards latin1 packages

[bastistician]

The decision to soft-enforce Encoding: UTF-8 has been made some time ago (#649) and has recently become effective with the release of roxygen2 6.1.0. I understand that supporting just one package encoding makes roxygen2 much simpler to maintain. However, I would like to document three drawbacks of this change:

1. Writing R extensions discourages the use of an Encoding field in the DESCRIPTION when it is actually not needed (Is the requirement Encoding: UTF-8 at odds with WRE? #774)

Do not specify an encoding unless one is actually needed: doing so makes the package less portable. If a package has a specified encoding, you should run R CMD build etc in a locale using that encoding.

2. Who knows how long UTF-8 will actually remain the dominant encoding? Maybe some of its (future) derivatives will take over? Having a roxygen2 implementation which works with either encoding seems to be desirable in the long run.

3. roxygen2 cannot be used by "latin1" packages anymore.

To quantify the importance of the last point, I have checked how many "latin1" packages on CRAN have actually used roxygen2:

db <- tools::CRAN_package_db()
table(db$Encoding, exclude = NULL)
## latin1 latin2  UTF-8   <NA>
##    198      1   3116   9723

latin1pkgs <- subset(db, Encoding == "latin1")$Package

## how many of these "latin1" packages have used roxygen2?
## (pulling DESCRIPTION files from the github/cran mirror)
descdir <- file.path(tempdir(), "desc"); dir.create(descdir)
for (pkg in latin1pkgs) {
    if (file.exists(file.path(descdir, pkg)))
        next
    url <- paste0("https://raw.githubusercontent.com/cran/", pkg, "/master/DESCRIPTION")
    ## or from CRAN:
    ## url <- paste0("https://CRAN.R-project.org/web/packages/", pkg, "/DESCRIPTION")
    try(download.file(url = url, destfile = file.path(descdir, pkg)))
}
setwd(descdir)
roxy <- sapply(list.files(descdir), read.dcf, fields = "RoxygenNote")
table(roxy, exclude = NULL)
## 5.0.1 6.0.1  <NA>
##     5    21   171
affected_pkgs <- names(which(!is.na(roxy)))
affected_maintainers <- unique(subset(db, Package %in% affected_pkgs)$Maintainer)

This results in 26 packages of 22 different maintainers, who will either have to re-encode their package or stick to the previous version of roxygen2 (for now). The affected packages are listed below. I have also mentioned the maintainers (if I could find a corresponding GitHub username) to make them aware of this regression in roxygen2:

Barycenter, bibliometrix (@massimoaria), binomSamSize (@hoehleatsu), DFIT (@herulor), ecd, ExpDes (@EricBarba), ExpDes.pt (@EricBarba), FKF (@waternumbers), GFA, GFE, ggseqlogo (@omarwagih), INLABMA, InterfaceqPCR, ldhmm, lifecontingencies (@spedygiorgio), MatrixCorrelation (@khliland), microclass (@larssnip), mixlm (@khliland), PGRdup (@aravind-j), plsgenomics, RcmdrPlugin.NMBU (@khliland), REPPlab (@fischuu), surveillance (@bastistician), TeachingSampling (@psirusteam), VTrack (@RossDwyer), Xplortext

Are there any plans for roxygen2 to support non-UTF-8 packages again in the future? Is this even possible while still avoiding issue #592 on Windows?

[fischuu]

Thanks for letting us know, I'll change the used encoding in our next package update.

[hadley]

Unfortunately I just don't have the resources to support non UTF-8 encodings; it's a lot of work for just 26 packages (many of which I'm sure could be converted to UTF-8 without problems).