Package design document

[pratikunterwegs]

This discussion is to introduce and get feedback on @joshwlambert and my template and thoughts for Epiverse-TRACE package design documents.

It's fairly long, and the template is in the next comment.

These are initial ideas and we are very happy to get and update them and the template based on input. Thanks also to @jd-otero whose design doc for {iraca} was also referred to while drafting the template.

---

Design document ideas

We see package design documents as having at least two goals:

1. To communicate the context and motivation, design principles, package components, and in brief the technologies used in the package,
2. To communicate how to contribute to the package, or develop based on it.

These goals map only roughly to locations along a user-developer spectrum; both users and developers might benefit from both aspects.

We propose that Epiverse-TRACE packages should have two design documents:

1. A general, potentially initially internal comprehensive design document that meets both objectives above,
2. A more focused user-or-developer facing document that explains only key design and component decisions and reasoning, and how to contribute.
   The second document would be a subset of the first.

We intend that package design documents should be living documents, which are updated throughout development. Some sections might only be written once the package is in development and practical challenges are encountered and resolved. The document is intended to be a place to document (briefly) the design decisions taken to resolve these challenges. We mark some sections in the template document as P1 for phase 1 sections (can be written before package development), and P2 for phase 2 sections (can be written once the package is in development and basic use cases are working).

We also intend that the documents should be modular and reusable, such that sections can be usefully copied into documentation such as the package vignettes or the repository Readme.

Creation

We propose the following creation options for these documents:

• We encourage package developers to initially create the general document as a reference for themselves. The document can be created as a Google or Word document online or offline, as this format is convenient for rapid collective editing and comments from PIs, stakeholders, etc.
• We would request users to create the user-developer facing document in a markdown format (.Rmd or .qmd), so that it can be included in the package repo, presented as package vignette, and potentially combined into a larger collection (see below).

Storage

We propose the following storage and presentation options for the general and user-developer facing documents:

1. For the general design document, a shared online location such as a Google Drive folder;
2. For the user-developer facing document, in the package repository as a package design vignette;

We also propose the creation of an online collection of the user-developer facing (or even the general) design documents, as a book or website, in a repository called et_design_docs. Each package could have its own chapter or page. These chapters would be taken automatically from the package design vignettes via an automated workflow. Collecting the documents together would create a comprehensive reference for how Epiverse-TRACE thinks about R packages. This would link with Blueprints, while having a distinct focus.

[pratikunterwegs]

Design document template

Title: Design of the {packagename} package

Package title: The title taken from DESCRIPTION, or to be used there.
Example: {finalsize}: Calculate the Final Size of an Epidemic

Package description: The description taken from DESCRIPTION, or to be used there.
Example: {finalsize}: Calculate the final size of a susceptible-infectious-recovered epidemic in a population with demographic variation in contact patterns and susceptibility to disease, as discussed in Miller (2012) <doi:10.1007/s11538-012-9749-6>.

Use case

What is the current intended usage of the package;

• In which situations would it be useful,
• When during an epidemic outbreak or response does this situation arise (early, middle, late),
• Who is the intended user - analyst, developer, contributor etc.

Example from {epidemics}, which is used to model vaccine or intervention impacts, and is considered a 'late stage' package.

The expected use case is an analyst in a resource-limited setting who is asked to generate initial predictions for the trajectory of an epidemic. The analyst is required to also generate predictions for the implementation of non-pharmaceutical and pharmaceutical interventions at different stages of the outbreak. The analyst may be also asked to consider the effect of demography (age, or other factors), on the epidemic trajectory.

Package concept and motivation

• What is the motivation for building this package, how does it fit into a workflow, what does the package provide that is not already available, or what's different from existing implementations (ease-of-use, speed-boost etc.).
• What are the main concepts in this package, in a few lines. May be divided into sub-headings, with a few lines for each sub-heading. Be clear rather than concise. Examples of sub-headings:
  • {epiparameter}: A library of parameters; A class representing delay distributions
  • {epidemics}: A library of epidemic models; Classes to compose epidemic scenarios
  • {scenarios}: Convenient methods and interoperable classes to launch and compare epidemic models

Scope

Write a little bit about the scope of the package within the wider package and analytics ecosystem. Be open about where the intended use-case boundaries are, what features are not (currently) included, and reasons why.

Future development

What are the future directions of development - if a feature or method is not currently included, explain what it is, and when it might be added.

Limitations and disclaimer

What are the limitations of the package - what does it or will it not include. Example:

{epidemics} is not intended to hold models of vector-borne diseases, or models of co-infection of multiple diseases.

Potentially, add a disclaimer about package use for legal or other reasons.

Package design and components

What are the conceptual components of the package, these could be epidemiological or biological concepts represented by package classes or data structures. Figures are useful. See example figure for {epidemics}.

Design principles

What are the design principles - this section blends into the next section, Components.

• These could be user-facing principles that make usage easier, such as having a minimal number of functions, or multiple helper functions to aid with model parameterisation etc.
• These could also be developer-facing principles such as how to implement package concepts, e.g. one class per biological concept, or organisation into modules related to specific workflows. Example:

In {epidemics}, the unit of organisation is the population and the pathogen (population-based organisation), while in {iraca}, the unit of organisation is the individual human and vector (individual-based organisation)

Components

How are the package components actually designed, that is, how are the package concepts translated into code. This is initially intended to be light on technical details. Add the technical details in the next section, Implementation. Example:

In {epidemics} and {iraca}, the epidemiological units of organisation are represented as classes. Similarly, in {epiparameter}, each delay distribution is an object.

This is a good place to describe in general terms:

• Package classes and data structures,
• Models - describe the baseline model if this package is intended to hold simulations,
• Modules - describe related functions.

Implementation (P2)

Use this section to describe how the principles and components in the package are actually implemented in code. Example:

In {epidemics}, we intend models to be composable from four building blocks - population, infection, intervention, vaccination - and each of these is implemented as an S3 class with the following attributes...

Dependencies (P2)

What are the dependencies and design decisions around them,

• in relation to other epidemiology, public health, data science, documentation, testing or general infrastructure packages,
• in relation to packages and tasks upstream in the workflow or pipeline.

Package usage

How is the package intended to be used, with a very small pseudocode example for non-technical readers to understand.

Pseudocode example (P1)

Does not have to be detailed, the main functionality of the package explained in a few lines of pseudocode.

Use-case example(P2)

Later, an actual use-case can be shown - this is similar to the examples shown in package Readmes.

Package contribution (P2)

Write how users and developers should contribute to the package - this can be through pull requests or issues, but also through contribution functionality built-in.

Note: This is different from the CONTRIBUTING.md file in each package as this is quite general; rather it refers to a package-specific guide for contributing to packages that are explicitly intended to be collections of features. Two examples are {epiparameter} and {epidemics}, which are supposed to be libraries of parameters and models respectively, and where external contributions are actively encouraged.

[Bisaloo]

Thanks for your work on this! It'll be helpful to have a template to inform what kind of information needs to be included in this document and make sure we don't forget anything.

My main concern is having package information spread over multiple repositories. This complexifies maintenance and increased the risk that information in the online book becomes out of date. A solution might be to keep the source of this file in the package repo and to automatically create a book that collects all these vignettes via the GitHub API. This script would also automatically create the first section ("Title: Design of the {packagename} package") from the DESCRIPTION since it would likely be redundant in the vignette.

Comments by section:

• "Package concept and motivation": could this also include a comparison to existing packages? To make it clear why there was a need for this package in the ecosystem.
• "Limitations and disclaimer": isn't the limitations part redundant with the "scope" section?
• "Package usage": I think the scope of this document is growing too much and it would be good to keep it focused on design considerations to make sure it remains short and easy to digest. This section can be a good intro to the "Getting started" vignette
• "Package contribution": This is what the CONTRIBUTION.md is for. I have seen your Note but I disagree with it. The contributing guide should contain package-specific guidelines if applicable. Having this information split over multiple locations will hurt discoverability.

[pratikunterwegs]

Thanks, I hope the template is useful and I can get started by adding the filled in document for {epidemics} as a more fleshed-out reference.

Regarding the collection of these documents into a book, we intended what you describe, but this is not clear in the orignal post; I have edited it now. We would like there to be only one public-facing source, from which other parts can be taken manually (e.g. for the Readme), or which can be collated automatically.

"Package concept and motivation": could this also include a comparison to existing packages? To make it clear why there was a need for this package in the ecosystem.

"Limitations and disclaimer": isn't the limitations part redundant with the "scope" section?

Taking this together because these are in the same section. The overall section "Package concept and motivation" is intended to have a few sentences about the scientific and software context of the package, so referring to features not already present in existing packages. The "Scope" subsection gives space to provide more details of this context, including references and comparisons to existing packages. This leaves the "Limitations and disclaimer" to be more specific about which use cases are definitely not recommended.

I think these could be compressed into a single section, but it might be good for packages focused on public health to be clear with users when not to use them.

"Package usage": I think the scope of this document is growing too much and it would be good to keep it focused on design considerations to make sure it remains short and easy to digest. This section can be a good intro to the "Getting started" vignette

I think most packages at this stage have only basic "Get started" vignettes and Readmes, and writing this section can help inform what goes into a vignette. These documents are intended to be read by PIs and collaborators, potentially much before any vignettes are written, so it could help them get a sense of what typical usage of this package would look like. This is especially the case if the package is a terminal one that comes at the end of a pipeline or relies on other packages.

"Package contribution": This is what the CONTRIBUTION.md is for. I have seen your Note but I disagree with it. The contributing guide should contain package-specific guidelines if applicable. Having this information split over multiple locations will hurt discoverability.

Again, the same argument as above applies. The design document is intended to be written before the package infrastructure, and is supposed to be a place to workshop the eventual contents of locations such as CONTRIBUTION.md. There is inevitably going to be some duplication because not all of our design can or should happen on GitHub.

[Bisaloo]

I think three distinct concepts are melded together in this template, which makes it difficult to give feedback. Ultimately, these concepts should also remain distinct and addressed separately. You also make this clear distinction in your first message but the single template document proposed here blurs the lines:

• the initial "package blueprints" document: this is a technical document intended to be short-lived that informs the early development iterations. It is a vector of communication between the developers and the scientific experts involved in the package development. I don't believe this needs to be strictly standardized as this is mostly a discussion between the different parties and any unclear point can be addressed directly. If you strongly believe this should be more standard, the best route is probably to edit the pitcher template, which has strong overlap with what you are proposing both in terms of aims and content.
• the "design principles" document: this is a technical document intended to be long-lived that constitutes a record of why certain design choices have been made. It is a vector of communication between the current developers and the future developers (including future self). This would benefit from being standardized as the communication between the document authors and readers is asynchronous and unclear points cannot be addressed directly.
• the "Epiverse-TRACE packages online book": this is a community-building document intended to be long-lived that acts as a user-facing catalog of packages we are building. It is a vector of communication between the package authors (developers & scientific experts) and the users.

[pratikunterwegs]

I would frame it differently - this general template is intended to be a template for a single living document to which developers and contributors can iteratively make edits. This master document can be a Google or Word doc for easy editing. Having everything in one place can help to make it easier to see whether any proposed changes to the package concept would also affect the use case or implementation details.

Parts of this general document would then form what we communicate to other developers and to the community, as the package design vignettes and via the book. These would be in more general formats.

I would not see this as a use case for the Pitcher template, as these design documents are intended to be written after we have decided to accept an idea from Pitcher/Dicussions and develop it. This is better done with a smaller group, and not on Github; imo GH is not very good for collaborative editing, and is a real barrier for entry for non-technical people which we wouldn't want to have.

[TimTaylor]

I agree with @Bisaloo in that it feels like things are being melded together and I'm also wary of there being even more documents to track (in different locations). I'd focus on standardizing:

the "design principles" document: this is a technical document intended to be long-lived that constitutes a record of why certain design choices have been made. It is a vector of communication between the current developers and the future developers (including future self). This would benefit from being standardized as the communication between the document authors and readers is asynchronous and unclear points cannot be addressed directly.

Between "pitcher" and "design principles" there will be synchronous and asynchronous discussions with collaborators but I think trying to standardise these processes and ways of working is somewhat out of scope. There will likely be ephemeral documents for more collaborative discussions in the beginning but it is important to view these more as scratch pads, with the aim being for this initial information and design decisions to be transferred to the repository located design document.

[joshwlambert]

Hi all, thank you for all the feedback.

Moving forward with the primary task of having a design document in each Epiverse-TRACE package, we have decided on a few of action points.

1. @pratikunterwegs and I are proponents of all new packages having a detailed and comprehensive design document as outlined above, and we will both act as points of contact (helpers) to those that have questions around writing the document. However, we have decided, with the useful feedback of @Bisaloo and @TimTaylor, that this should not be a formally standardised or required document. Therefore, although we encourage developers to write this document, it will not be within the scope of this task.
2. Including a minimal design principles vignette (vignette is stressed as these must live within the R package and be rmarkdown (.Rmd) files) which includes design decisions during development, in each Epiverse-TRACE package will be the primary target of this task.

As part of this task we will:

• Look for good examples of design principles vignettes and document appropriate sections that can be included. This will not be an overly prescriptive list, but will hopefully guide developers of topics for inclusion.
• Arrange a call with every package maintainer within Epiverse-TRACE to discuss the design principles vignette, the benefits of such a vignette for both users and developers (internal and external to Epiverse-TRACE), and some guidance for sections to include in the vignette.

3. At present, we do not plan to create the design document book as a compilation of design documents. If the need or demand for this resource arises we will reconsider creating it, but that will be outlined in a separate task with a refined scope.

The intended final outcome from this task will be that a design principles vignette is present in every Epiverse-TRACE package. This vignette will not have length or formatting requirements; in it's minimal form it can be a couple of bullet points.

We hope that moving forward this will improve the contribution pathway for Epiverse-TRACE packages as part of co-creation and decentralising code ownership within the open-source community.

Please give feedback on this proposal. @pratikunterwegs and I will start working on it over the coming weeks.

[joshwlambert]

Progress on task:

Epiverse-TRACE packages have been reviewed and those containing a design vignette are: {episoap} and {simulist}.

Look for good examples of design principles vignettes and document appropriate sections that can be included. This will not be an overly prescriptive list, but will hopefully guide developers of topics for inclusion.

Good sections to include in the design principles vignette have been listed with suggested content in a design vignette template and is being suggested as an addition to the {packagetemplate} (epiverse-trace/packagetemplate#97).

Next steps:

Arrange calls with Epiverse-TRACE package maintainers to explain benefit of design vignette and advice all to add a short vignette to their packages.

[pratikunterwegs]

Closing this discussion as resolved for now by epiverse-trace/packagetemplate#97 and epiverse-trace/packagetemplate#99. Tagging @joshwlambert in case you feel it should stay open.