Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] introduce GeneratedBy to "core" BIDS #440

Merged
merged 4 commits into from
Feb 4, 2022
Merged

[ENH] introduce GeneratedBy to "core" BIDS #440

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
94112b6
BF: typo + added trailing period
yarikoptic Oct 11, 2021
2a638c6
[ENH] introduce GeneratedBy to "core" BIDS
yarikoptic Oct 11, 2021
d9060ae
Apply suggestions from code review by @tsalo and @sappelhoff
yarikoptic Oct 28, 2021
dfe4da5
Merge branch 'master' into enh-pipeline-description
effigies Jan 24, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 39 additions & 23 deletions src/03-modality-agnostic-files.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,22 @@ Every dataset MUST include this file with the following fields:
"EthicsApprovals": "OPTIONAL",
"ReferencesAndLinks": "OPTIONAL",
"DatasetDOI": "OPTIONAL",
"GeneratedBy": "RECOMMENDED",
"SourceDatasets": "RECOMMENDED",
Comment on lines +31 to +32
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RECOMMENDED will generally come with a validator warning on absence. That's fine with me, but want to make sure that's your intent. Also, this table is sorted by requirement level, so if these are RECOMMENDED they should come before the OPTIONAL fields.

}
) }}

Each object in the `GeneratedBy` array includes the following REQUIRED, RECOMMENDED
and OPTIONAL keys:

| **Key name** | **Requirement level** | **Data type** | **Description** |
|--------------|-----------------------|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Name | REQUIRED | [string][] | Name of the pipeline or process that generated the outputs. Use `"Manual"` to indicate the derivatives were generated by hand, or adjusted manually after an initial run of an automated pipeline. |
| Version | RECOMMENDED | [string][] | Version of the pipeline. |
| Description | OPTIONAL | [string][] | Plain-text description of the pipeline or process that generated the outputs. RECOMMENDED if `Name` is `"Manual"`. |
| CodeURL | OPTIONAL | [string][] | URL where the code used to generate the dataset may be found. |
| Container | OPTIONAL | [object][] | Used to specify the location and relevant attributes of software container image used to produce the dataset. Valid keys in this object include `Type`, `Tag` and [`URI`][uri] with [string][] values. |

Example:

```JSON
Expand All @@ -57,37 +70,45 @@ Example:
"Alzheimer A., & Kraepelin, E. (2015). Neural correlates of presenile dementia in humans. Journal of Neuroscientific Data, 2, 234001. doi:1920.8/jndata.2015.7"
],
"DatasetDOI": "doi:10.0.2.3/dfjj.10",
"HEDVersion": "8.0.0"
"HEDVersion": "8.0.0",
"GeneratedBy": [
{
"Name": "reproin",
"Version": "0.6.0",
"Container": {
"Type": "docker",
"Tag": "repronim/reproin:0.6.0"
}
}
],
"SourceDatasets": [
{
"URL": "s3://dicoms/studies/correlates",
"Version": "April 11 2011"
}
]
}
```

#### Derived dataset and pipeline description

As for any BIDS dataset, a `dataset_description.json` file MUST be found at the
top level of the a derived dataset:
`<dataset>/derivatives/<pipeline_name>/dataset_description.json`
top level of every derived dataset:
`<dataset>/derivatives/<pipeline_name>/dataset_description.json`.

In addition to the keys for raw BIDS datasets,
derived BIDS datasets include the following REQUIRED and RECOMMENDED
`dataset_description.json` keys:
In contrast to raw BIDS datasets, derived BIDS datasets MUST include a
`GeneratedBy` key:

{{ MACROS___make_metadata_table(
{
"GeneratedBy": "REQUIRED",
"SourceDatasets": "RECOMMENDED",
"GeneratedBy": "REQUIRED"
}
) }}

Each object in the `GeneratedBy` list includes the following REQUIRED, RECOMMENDED
and OPTIONAL keys:

| **Key name** | **Requirement level** | **Data type** | **Description** |
|--------------|-----------------------|---------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Name | REQUIRED | [string][] | Name of the pipeline or process that generated the outputs. Use `"Manual"` to indicate the derivatives were generated by hand, or adjusted manually after an initial run of an automated pipeline. |
| Version | RECOMMENDED | [string][] | Version of the pipeline. |
| Description | OPTIONAL | [string][] | Plain-text description of the pipeline or process that generated the outputs. RECOMMENDED if `Name` is `"Manual"`. |
| CodeURL | OPTIONAL | [string][] | URL where the code used to generate the derivatives may be found. |
| Container | OPTIONAL | [object][] | Used to specify the location and relevant attributes of software container image used to produce the derivative. Valid keys in this object include `Type`, `Tag` and [`URI`][uri] with [string][] values. |
If a derived dataset is stored as a subfolder of the raw dataset, then the `Name` field
of the first `GeneratedBy` object MUST be a substring of the derived dataset folder name.
That is, in a directory `<dataset>/derivatives/<pipeline>[-<variant>]/`, the first
`GeneratedBy` object should have a `Name` of `<pipeline>`.

Example:

Expand Down Expand Up @@ -120,11 +141,6 @@ Example:
}
```

If a derived dataset is stored as a subfolder of the raw dataset, then the `Name` field
of the first `GeneratedBy` object MUST be a substring of the derived dataset folder name.
That is, in a directory `<dataset>/derivatives/<pipeline>[-<variant>]/`, the first
`GeneratedBy` object should have a `Name` of `<pipeline>`.

### `README`

Every BIDS dataset SHOULD come with a free form text file (`README`) describing the dataset in more detail.
Expand Down
3 changes: 1 addition & 2 deletions src/schema/objects/metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -925,8 +925,7 @@ Funding:
GeneratedBy:
name: GeneratedBy
description: |
Used to specify provenance of the derived dataset.
See table below for contents of each object.
Used to specify provenance of the dataset.
type: array
minItems: 1
items:
Expand Down