Bloodpac QA update 1615481599

[themarcelor]

copying files from data.bloodpac.org to qa-bloodpac.planx-pla.net

[PlanXCyborg]

qa-bloodpac.planx-pla.net/manifest.json

Deployment changes

• fence
  • REQUIRES A FENCE DB MIGRATION, since with this update there is now an additional column in the authorization_code table "refresh_token_expires_in" (uc-cdis/Feat/refresh expires in fence#848)
  • Commons configured with RAS staging login must add the "discovery_url" fence config, because it now defaults to RAS prod (uc-cdis/Fix/ras url fence#824)
  • If the fence config already has [CLIENT_, USER_, SESSION_]ALLOWED_SCOPES fields, and GA4GH AAI integration is desired, then add ga4gh_passport_v1 to all those lists. If default config values are being used then no change required. (uc-cdis/PXP-6406 Feat/ga4gh scope fence#799)
• guppy
  • To deploy SearchFilters in a tab of the Explorer page, add an entry to data{/file}ExplorerConfig.filters.tabs.searchFields. NOTE: Do not add the same field to both searchFields and fields, this will cause strange behavior in the search field. Example config: (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "filters": { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "tabs": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "title": "Subject", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "searchFields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "subject_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "submitter_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "consent_codes" (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ], (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "fields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_type", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_format", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ... (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ] (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • } (uc-cdis/(PXP-6359) Add Search Filter guppy#99)

• portal
  • New optional config property, tierAccessLevel, in each tab of the explorerConfig block in the portal config. Used to enable index-scoped tiered-access, which is only available starting with Guppy server version 0.11.0. See README for more information. (uc-cdis/Feat/index scoped tiered access data-portal#802)
  • To deploy Discovery page: (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "featureFlags": { "discovery": true } must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • If discoveryConfig.features.authorization.enabled: true, "useArboristUI": true must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "discoveryConfig": <DiscoveryConfig> must be set in the portal config. DiscoveryConfig documentation can be found at https://github.com/uc-cdis/data-portal/blob/master/docs/portal_config.md (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • To upload data to Discovery page, data must be uploaded to environment's metadata-service instance with guid_type: "discovery_metadata". The metadata must all be under the top-level property gen3_discovery. (No documentation on this process yet, see this gist for right now https://gist.github.com/Avantol13/a53593837503fede2881f315198247c4) (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • Added new entry GWASApp for analysisTools for this new GWAS app for VA. Beaware not to confuse with the old "GWAS PoC" app, which bears the entry vaGWAS (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • antd modal now by default will use g3-color__base-blue as header background color, this is consistant with our color schemes for other UI components. If an env has customized color scheme, please override .ant-modal-header in gitops.css (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • The new feature of Portal requires Guppy v0.10.0 (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • Added new button types in the Portal config. See portal_config.md for an example. (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • At time of writing, qa-midrc is only env using the new feature flag. These changes should be deployed alongside each other to production, but nothing will break if done separately. (uc-cdis/(PXP-7097) Add optional alternate TopBar profile menu data-portal#759)
  • New value for GEN3_BUNDLE env var GEN3_BUNDLE=nct (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • New optional portal config variable injectDAPTag (default value is false). If set to true, will inject DAP tag script into page header (this is required for NCT PROD env only) (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • All envs should check if the dataExplorerConfig field in its portal config has ONLY arrangerConfig field but NO guppyConfig field (it is okay to have both). If so, please update dataExplorerConfig to use Guppy (uc-cdis/PXP-6409 Remove arranger data-portal#743)
  • A new Export to PFB button can be optionally enabled in the config.json. It is disabled by default. The button requires at least pelican:2020.10 release. (uc-cdis/Feat/export pfb to workspace data-portal#681)
  • Add GEN3_BUNDLE=covid19 as portal env var to enable covid bundle (uc-cdis/Feat/covid bundle data-portal#737)
  • Limited File PFB Export requires Pelican >= 0.5.0, Tube >= 0.4.1 (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export is enabled by setting fileExplorerConfig.enableLimitedFilePFBExport: { sourceNodeField: "source_node" } and by adding buttons of buttonType export-files (Export to Terra), export-files-to-pfb (Download PFB), or export-files-to-seven-bridges (Export to Seven Bridges) to fileExplorerConfig.buttons. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "fileExplorerConfig": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "buttons": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • .... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files-to-pfb", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to PFB", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to Terra", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enableLimitedFilePFBExport": {"sourceNodeField": "source_node"}, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires the source_node property to be ETL'd -- source_node should be added to props in the file section of etlMapping.yaml. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: mpingram_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • doc_type: file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • type: collector (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • root: None (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • category: data_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • props: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: object_id (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: md5sum (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: source_node (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires a new export-files job block to be added to the Sower config in manifest.json. The $ROOT_NODE environment variable must be set to "file", or the name of the Guppy file index if it isn't "file". For BioDataCatalyst, the EXTRA_NODES environment variable must be set to "". (This is for backwards compatibility: pelican-export includes reference_file by default on all BDCat PFB exports unless $EXTRA_NODES is set to an empty string). Example sower config: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "action": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "container": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "job-task", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "image": "quay.io/cdis/pelican-export:0.5.0", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "pull_policy": "Always", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "env": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "DICTIONARY_URL", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "dictionary_url" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "GEN3_HOSTNAME", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "hostname" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "ROOT_NODE", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "file" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "EXTRA_NODES", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumeMounts": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/pelican-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "config.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/peregrine-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "creds.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "cpu-limit": "1", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "memory-limit": "4Gi" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumes": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "pelicanservice-g3auto" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "peregrine-creds" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "restart_policy": "Never" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)

• ssjdispatcher
  • metadata-service must already be present or added to the "versions": { section of the cdis-manifest. After metadata-service has been confirmed to be in the cdis-manifest, the now-required Metadata Service creds for SSJDispatcher must be configured by running gen3 roll all with JENKINS_HOME not set. gen3 roll all calls gen3 kube-setup-ssjdispatcher, which now automatically configures the SSJDispatcher secret with Metadata Service creds. (uc-cdis/(PXP-6977): make mds creds required for service startup ssjdispatcher#32)

[PlanXCyborg]

qa-bloodpac.planx-pla.net/manifest.json

Deployment changes

• fence
  • REQUIRES A FENCE DB MIGRATION, since with this update there is now an additional column in the authorization_code table "refresh_token_expires_in" (uc-cdis/Feat/refresh expires in fence#848)
  • Commons configured with RAS staging login must add the "discovery_url" fence config, because it now defaults to RAS prod (uc-cdis/Fix/ras url fence#824)
  • If the fence config already has [CLIENT_, USER_, SESSION_]ALLOWED_SCOPES fields, and GA4GH AAI integration is desired, then add ga4gh_passport_v1 to all those lists. If default config values are being used then no change required. (uc-cdis/PXP-6406 Feat/ga4gh scope fence#799)
• guppy
  • To deploy SearchFilters in a tab of the Explorer page, add an entry to data{/file}ExplorerConfig.filters.tabs.searchFields. NOTE: Do not add the same field to both searchFields and fields, this will cause strange behavior in the search field. Example config: (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "filters": { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "tabs": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "title": "Subject", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "searchFields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "subject_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "submitter_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "consent_codes" (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ], (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "fields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_type", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_format", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ... (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ] (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • } (uc-cdis/(PXP-6359) Add Search Filter guppy#99)

• portal
  • New optional config property, tierAccessLevel, in each tab of the explorerConfig block in the portal config. Used to enable index-scoped tiered-access, which is only available starting with Guppy server version 0.11.0. See README for more information. (uc-cdis/Feat/index scoped tiered access data-portal#802)
  • To deploy Discovery page: (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "featureFlags": { "discovery": true } must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • If discoveryConfig.features.authorization.enabled: true, "useArboristUI": true must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "discoveryConfig": <DiscoveryConfig> must be set in the portal config. DiscoveryConfig documentation can be found at https://github.com/uc-cdis/data-portal/blob/master/docs/portal_config.md (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • To upload data to Discovery page, data must be uploaded to environment's metadata-service instance with guid_type: "discovery_metadata". The metadata must all be under the top-level property gen3_discovery. (No documentation on this process yet, see this gist for right now https://gist.github.com/Avantol13/a53593837503fede2881f315198247c4) (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • Added new entry GWASApp for analysisTools for this new GWAS app for VA. Beaware not to confuse with the old "GWAS PoC" app, which bears the entry vaGWAS (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • antd modal now by default will use g3-color__base-blue as header background color, this is consistant with our color schemes for other UI components. If an env has customized color scheme, please override .ant-modal-header in gitops.css (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • The new feature of Portal requires Guppy v0.10.0 (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • Added new button types in the Portal config. See portal_config.md for an example. (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • At time of writing, qa-midrc is only env using the new feature flag. These changes should be deployed alongside each other to production, but nothing will break if done separately. (uc-cdis/(PXP-7097) Add optional alternate TopBar profile menu data-portal#759)
  • New value for GEN3_BUNDLE env var GEN3_BUNDLE=nct (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • New optional portal config variable injectDAPTag (default value is false). If set to true, will inject DAP tag script into page header (this is required for NCT PROD env only) (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • All envs should check if the dataExplorerConfig field in its portal config has ONLY arrangerConfig field but NO guppyConfig field (it is okay to have both). If so, please update dataExplorerConfig to use Guppy (uc-cdis/PXP-6409 Remove arranger data-portal#743)
  • A new Export to PFB button can be optionally enabled in the config.json. It is disabled by default. The button requires at least pelican:2020.10 release. (uc-cdis/Feat/export pfb to workspace data-portal#681)
  • Add GEN3_BUNDLE=covid19 as portal env var to enable covid bundle (uc-cdis/Feat/covid bundle data-portal#737)
  • Limited File PFB Export requires Pelican >= 0.5.0, Tube >= 0.4.1 (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export is enabled by setting fileExplorerConfig.enableLimitedFilePFBExport: { sourceNodeField: "source_node" } and by adding buttons of buttonType export-files (Export to Terra), export-files-to-pfb (Download PFB), or export-files-to-seven-bridges (Export to Seven Bridges) to fileExplorerConfig.buttons. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "fileExplorerConfig": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "buttons": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • .... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files-to-pfb", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to PFB", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to Terra", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enableLimitedFilePFBExport": {"sourceNodeField": "source_node"}, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires the source_node property to be ETL'd -- source_node should be added to props in the file section of etlMapping.yaml. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: mpingram_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • doc_type: file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • type: collector (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • root: None (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • category: data_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • props: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: object_id (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: md5sum (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: source_node (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires a new export-files job block to be added to the Sower config in manifest.json. The $ROOT_NODE environment variable must be set to "file", or the name of the Guppy file index if it isn't "file". For BioDataCatalyst, the EXTRA_NODES environment variable must be set to "". (This is for backwards compatibility: pelican-export includes reference_file by default on all BDCat PFB exports unless $EXTRA_NODES is set to an empty string). Example sower config: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "action": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "container": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "job-task", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "image": "quay.io/cdis/pelican-export:0.5.0", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "pull_policy": "Always", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "env": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "DICTIONARY_URL", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "dictionary_url" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "GEN3_HOSTNAME", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "hostname" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "ROOT_NODE", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "file" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "EXTRA_NODES", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumeMounts": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/pelican-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "config.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/peregrine-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "creds.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "cpu-limit": "1", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "memory-limit": "4Gi" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumes": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "pelicanservice-g3auto" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "peregrine-creds" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "restart_policy": "Never" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)

• ssjdispatcher
  • metadata-service must already be present or added to the "versions": { section of the cdis-manifest. After metadata-service has been confirmed to be in the cdis-manifest, the now-required Metadata Service creds for SSJDispatcher must be configured by running gen3 roll all with JENKINS_HOME not set. gen3 roll all calls gen3 kube-setup-ssjdispatcher, which now automatically configures the SSJDispatcher secret with Metadata Service creds. (uc-cdis/(PXP-6977): make mds creds required for service startup ssjdispatcher#32)

[PlanXCyborg]

qa-bloodpac.planx-pla.net/manifest.json

Deployment changes

• fence
  • REQUIRES A FENCE DB MIGRATION, since with this update there is now an additional column in the authorization_code table "refresh_token_expires_in" (uc-cdis/Feat/refresh expires in fence#848)
  • Commons configured with RAS staging login must add the "discovery_url" fence config, because it now defaults to RAS prod (uc-cdis/Fix/ras url fence#824)
  • If the fence config already has [CLIENT_, USER_, SESSION_]ALLOWED_SCOPES fields, and GA4GH AAI integration is desired, then add ga4gh_passport_v1 to all those lists. If default config values are being used then no change required. (uc-cdis/PXP-6406 Feat/ga4gh scope fence#799)
• guppy
  • To deploy SearchFilters in a tab of the Explorer page, add an entry to data{/file}ExplorerConfig.filters.tabs.searchFields. NOTE: Do not add the same field to both searchFields and fields, this will cause strange behavior in the search field. Example config: (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "filters": { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "tabs": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "title": "Subject", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "searchFields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "subject_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "submitter_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "consent_codes" (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ], (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "fields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_type", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_format", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ... (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ] (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • } (uc-cdis/(PXP-6359) Add Search Filter guppy#99)

• portal
  • New optional config property, tierAccessLevel, in each tab of the explorerConfig block in the portal config. Used to enable index-scoped tiered-access, which is only available starting with Guppy server version 0.11.0. See README for more information. (uc-cdis/Feat/index scoped tiered access data-portal#802)
  • To deploy Discovery page: (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "featureFlags": { "discovery": true } must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • If discoveryConfig.features.authorization.enabled: true, "useArboristUI": true must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "discoveryConfig": <DiscoveryConfig> must be set in the portal config. DiscoveryConfig documentation can be found at https://github.com/uc-cdis/data-portal/blob/master/docs/portal_config.md (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • To upload data to Discovery page, data must be uploaded to environment's metadata-service instance with guid_type: "discovery_metadata". The metadata must all be under the top-level property gen3_discovery. (No documentation on this process yet, see this gist for right now https://gist.github.com/Avantol13/a53593837503fede2881f315198247c4) (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • Added new entry GWASApp for analysisTools for this new GWAS app for VA. Beaware not to confuse with the old "GWAS PoC" app, which bears the entry vaGWAS (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • antd modal now by default will use g3-color__base-blue as header background color, this is consistant with our color schemes for other UI components. If an env has customized color scheme, please override .ant-modal-header in gitops.css (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • The new feature of Portal requires Guppy v0.10.0 (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • Added new button types in the Portal config. See portal_config.md for an example. (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • At time of writing, qa-midrc is only env using the new feature flag. These changes should be deployed alongside each other to production, but nothing will break if done separately. (uc-cdis/(PXP-7097) Add optional alternate TopBar profile menu data-portal#759)
  • New value for GEN3_BUNDLE env var GEN3_BUNDLE=nct (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • New optional portal config variable injectDAPTag (default value is false). If set to true, will inject DAP tag script into page header (this is required for NCT PROD env only) (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • All envs should check if the dataExplorerConfig field in its portal config has ONLY arrangerConfig field but NO guppyConfig field (it is okay to have both). If so, please update dataExplorerConfig to use Guppy (uc-cdis/PXP-6409 Remove arranger data-portal#743)
  • A new Export to PFB button can be optionally enabled in the config.json. It is disabled by default. The button requires at least pelican:2020.10 release. (uc-cdis/Feat/export pfb to workspace data-portal#681)
  • Add GEN3_BUNDLE=covid19 as portal env var to enable covid bundle (uc-cdis/Feat/covid bundle data-portal#737)
  • Limited File PFB Export requires Pelican >= 0.5.0, Tube >= 0.4.1 (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export is enabled by setting fileExplorerConfig.enableLimitedFilePFBExport: { sourceNodeField: "source_node" } and by adding buttons of buttonType export-files (Export to Terra), export-files-to-pfb (Download PFB), or export-files-to-seven-bridges (Export to Seven Bridges) to fileExplorerConfig.buttons. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "fileExplorerConfig": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "buttons": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • .... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files-to-pfb", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to PFB", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to Terra", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enableLimitedFilePFBExport": {"sourceNodeField": "source_node"}, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires the source_node property to be ETL'd -- source_node should be added to props in the file section of etlMapping.yaml. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: mpingram_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • doc_type: file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • type: collector (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • root: None (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • category: data_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • props: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: object_id (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: md5sum (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: source_node (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires a new export-files job block to be added to the Sower config in manifest.json. The $ROOT_NODE environment variable must be set to "file", or the name of the Guppy file index if it isn't "file". For BioDataCatalyst, the EXTRA_NODES environment variable must be set to "". (This is for backwards compatibility: pelican-export includes reference_file by default on all BDCat PFB exports unless $EXTRA_NODES is set to an empty string). Example sower config: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "action": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "container": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "job-task", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "image": "quay.io/cdis/pelican-export:0.5.0", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "pull_policy": "Always", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "env": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "DICTIONARY_URL", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "dictionary_url" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "GEN3_HOSTNAME", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "hostname" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "ROOT_NODE", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "file" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "EXTRA_NODES", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumeMounts": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/pelican-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "config.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/peregrine-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "creds.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "cpu-limit": "1", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "memory-limit": "4Gi" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumes": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "pelicanservice-g3auto" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "peregrine-creds" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "restart_policy": "Never" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)

• ssjdispatcher
  • metadata-service must already be present or added to the "versions": { section of the cdis-manifest. After metadata-service has been confirmed to be in the cdis-manifest, the now-required Metadata Service creds for SSJDispatcher must be configured by running gen3 roll all with JENKINS_HOME not set. gen3 roll all calls gen3 kube-setup-ssjdispatcher, which now automatically configures the SSJDispatcher secret with Metadata Service creds. (uc-cdis/(PXP-6977): make mds creds required for service startup ssjdispatcher#32)

[PlanXCyborg]

qa-bloodpac.planx-pla.net/manifest.json

Deployment changes

• fence
  • REQUIRES A FENCE DB MIGRATION, since with this update there is now an additional column in the authorization_code table "refresh_token_expires_in" (uc-cdis/Feat/refresh expires in fence#848)
  • Commons configured with RAS staging login must add the "discovery_url" fence config, because it now defaults to RAS prod (uc-cdis/Fix/ras url fence#824)
  • If the fence config already has [CLIENT_, USER_, SESSION_]ALLOWED_SCOPES fields, and GA4GH AAI integration is desired, then add ga4gh_passport_v1 to all those lists. If default config values are being used then no change required. (uc-cdis/PXP-6406 Feat/ga4gh scope fence#799)
• guppy
  • To deploy SearchFilters in a tab of the Explorer page, add an entry to data{/file}ExplorerConfig.filters.tabs.searchFields. NOTE: Do not add the same field to both searchFields and fields, this will cause strange behavior in the search field. Example config: (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "filters": { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "tabs": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "title": "Subject", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "searchFields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "subject_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "submitter_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "consent_codes" (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ], (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "fields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_type", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_format", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ... (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ] (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • } (uc-cdis/(PXP-6359) Add Search Filter guppy#99)

• portal
  • New optional config property, tierAccessLevel, in each tab of the explorerConfig block in the portal config. Used to enable index-scoped tiered-access, which is only available starting with Guppy server version 0.11.0. See README for more information. (uc-cdis/Feat/index scoped tiered access data-portal#802)
  • To deploy Discovery page: (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "featureFlags": { "discovery": true } must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • If discoveryConfig.features.authorization.enabled: true, "useArboristUI": true must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "discoveryConfig": <DiscoveryConfig> must be set in the portal config. DiscoveryConfig documentation can be found at https://github.com/uc-cdis/data-portal/blob/master/docs/portal_config.md (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • To upload data to Discovery page, data must be uploaded to environment's metadata-service instance with guid_type: "discovery_metadata". The metadata must all be under the top-level property gen3_discovery. (No documentation on this process yet, see this gist for right now https://gist.github.com/Avantol13/a53593837503fede2881f315198247c4) (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • Added new entry GWASApp for analysisTools for this new GWAS app for VA. Beaware not to confuse with the old "GWAS PoC" app, which bears the entry vaGWAS (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • antd modal now by default will use g3-color__base-blue as header background color, this is consistant with our color schemes for other UI components. If an env has customized color scheme, please override .ant-modal-header in gitops.css (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • The new feature of Portal requires Guppy v0.10.0 (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • Added new button types in the Portal config. See portal_config.md for an example. (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • At time of writing, qa-midrc is only env using the new feature flag. These changes should be deployed alongside each other to production, but nothing will break if done separately. (uc-cdis/(PXP-7097) Add optional alternate TopBar profile menu data-portal#759)
  • New value for GEN3_BUNDLE env var GEN3_BUNDLE=nct (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • New optional portal config variable injectDAPTag (default value is false). If set to true, will inject DAP tag script into page header (this is required for NCT PROD env only) (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • All envs should check if the dataExplorerConfig field in its portal config has ONLY arrangerConfig field but NO guppyConfig field (it is okay to have both). If so, please update dataExplorerConfig to use Guppy (uc-cdis/PXP-6409 Remove arranger data-portal#743)
  • A new Export to PFB button can be optionally enabled in the config.json. It is disabled by default. The button requires at least pelican:2020.10 release. (uc-cdis/Feat/export pfb to workspace data-portal#681)
  • Add GEN3_BUNDLE=covid19 as portal env var to enable covid bundle (uc-cdis/Feat/covid bundle data-portal#737)
  • Limited File PFB Export requires Pelican >= 0.5.0, Tube >= 0.4.1 (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export is enabled by setting fileExplorerConfig.enableLimitedFilePFBExport: { sourceNodeField: "source_node" } and by adding buttons of buttonType export-files (Export to Terra), export-files-to-pfb (Download PFB), or export-files-to-seven-bridges (Export to Seven Bridges) to fileExplorerConfig.buttons. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "fileExplorerConfig": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "buttons": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • .... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files-to-pfb", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to PFB", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to Terra", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enableLimitedFilePFBExport": {"sourceNodeField": "source_node"}, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires the source_node property to be ETL'd -- source_node should be added to props in the file section of etlMapping.yaml. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: mpingram_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • doc_type: file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • type: collector (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • root: None (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • category: data_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • props: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: object_id (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: md5sum (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: source_node (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires a new export-files job block to be added to the Sower config in manifest.json. The $ROOT_NODE environment variable must be set to "file", or the name of the Guppy file index if it isn't "file". For BioDataCatalyst, the EXTRA_NODES environment variable must be set to "". (This is for backwards compatibility: pelican-export includes reference_file by default on all BDCat PFB exports unless $EXTRA_NODES is set to an empty string). Example sower config: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "action": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "container": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "job-task", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "image": "quay.io/cdis/pelican-export:0.5.0", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "pull_policy": "Always", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "env": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "DICTIONARY_URL", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "dictionary_url" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "GEN3_HOSTNAME", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "hostname" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "ROOT_NODE", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "file" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "EXTRA_NODES", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumeMounts": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/pelican-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "config.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/peregrine-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "creds.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "cpu-limit": "1", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "memory-limit": "4Gi" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumes": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "pelicanservice-g3auto" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "peregrine-creds" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "restart_policy": "Never" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)

• ssjdispatcher
  • metadata-service must already be present or added to the "versions": { section of the cdis-manifest. After metadata-service has been confirmed to be in the cdis-manifest, the now-required Metadata Service creds for SSJDispatcher must be configured by running gen3 roll all with JENKINS_HOME not set. gen3 roll all calls gen3 kube-setup-ssjdispatcher, which now automatically configures the SSJDispatcher secret with Metadata Service creds. (uc-cdis/(PXP-6977): make mds creds required for service startup ssjdispatcher#32)

[PlanXCyborg]

qa-bloodpac.planx-pla.net/manifest.json

Deployment changes

• fence
  • REQUIRES A FENCE DB MIGRATION, since with this update there is now an additional column in the authorization_code table "refresh_token_expires_in" (uc-cdis/Feat/refresh expires in fence#848)
  • Commons configured with RAS staging login must add the "discovery_url" fence config, because it now defaults to RAS prod (uc-cdis/Fix/ras url fence#824)
  • If the fence config already has [CLIENT_, USER_, SESSION_]ALLOWED_SCOPES fields, and GA4GH AAI integration is desired, then add ga4gh_passport_v1 to all those lists. If default config values are being used then no change required. (uc-cdis/PXP-6406 Feat/ga4gh scope fence#799)
• guppy
  • To deploy SearchFilters in a tab of the Explorer page, add an entry to data{/file}ExplorerConfig.filters.tabs.searchFields. NOTE: Do not add the same field to both searchFields and fields, this will cause strange behavior in the search field. Example config: (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "filters": { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "tabs": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "title": "Subject", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "searchFields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "subject_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "submitter_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "consent_codes" (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ], (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "fields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_type", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_format", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ... (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ] (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • } (uc-cdis/(PXP-6359) Add Search Filter guppy#99)

• portal
  • New optional config property, tierAccessLevel, in each tab of the explorerConfig block in the portal config. Used to enable index-scoped tiered-access, which is only available starting with Guppy server version 0.11.0. See README for more information. (uc-cdis/Feat/index scoped tiered access data-portal#802)
  • To deploy Discovery page: (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "featureFlags": { "discovery": true } must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • If discoveryConfig.features.authorization.enabled: true, "useArboristUI": true must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "discoveryConfig": <DiscoveryConfig> must be set in the portal config. DiscoveryConfig documentation can be found at https://github.com/uc-cdis/data-portal/blob/master/docs/portal_config.md (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • To upload data to Discovery page, data must be uploaded to environment's metadata-service instance with guid_type: "discovery_metadata". The metadata must all be under the top-level property gen3_discovery. (No documentation on this process yet, see this gist for right now https://gist.github.com/Avantol13/a53593837503fede2881f315198247c4) (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • Added new entry GWASApp for analysisTools for this new GWAS app for VA. Beaware not to confuse with the old "GWAS PoC" app, which bears the entry vaGWAS (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • antd modal now by default will use g3-color__base-blue as header background color, this is consistant with our color schemes for other UI components. If an env has customized color scheme, please override .ant-modal-header in gitops.css (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • The new feature of Portal requires Guppy v0.10.0 (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • Added new button types in the Portal config. See portal_config.md for an example. (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • At time of writing, qa-midrc is only env using the new feature flag. These changes should be deployed alongside each other to production, but nothing will break if done separately. (uc-cdis/(PXP-7097) Add optional alternate TopBar profile menu data-portal#759)
  • New value for GEN3_BUNDLE env var GEN3_BUNDLE=nct (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • New optional portal config variable injectDAPTag (default value is false). If set to true, will inject DAP tag script into page header (this is required for NCT PROD env only) (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • All envs should check if the dataExplorerConfig field in its portal config has ONLY arrangerConfig field but NO guppyConfig field (it is okay to have both). If so, please update dataExplorerConfig to use Guppy (uc-cdis/PXP-6409 Remove arranger data-portal#743)
  • A new Export to PFB button can be optionally enabled in the config.json. It is disabled by default. The button requires at least pelican:2020.10 release. (uc-cdis/Feat/export pfb to workspace data-portal#681)
  • Add GEN3_BUNDLE=covid19 as portal env var to enable covid bundle (uc-cdis/Feat/covid bundle data-portal#737)
  • Limited File PFB Export requires Pelican >= 0.5.0, Tube >= 0.4.1 (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export is enabled by setting fileExplorerConfig.enableLimitedFilePFBExport: { sourceNodeField: "source_node" } and by adding buttons of buttonType export-files (Export to Terra), export-files-to-pfb (Download PFB), or export-files-to-seven-bridges (Export to Seven Bridges) to fileExplorerConfig.buttons. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "fileExplorerConfig": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "buttons": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • .... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files-to-pfb", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to PFB", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to Terra", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enableLimitedFilePFBExport": {"sourceNodeField": "source_node"}, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires the source_node property to be ETL'd -- source_node should be added to props in the file section of etlMapping.yaml. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: mpingram_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • doc_type: file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • type: collector (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • root: None (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • category: data_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • props: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: object_id (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: md5sum (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: source_node (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires a new export-files job block to be added to the Sower config in manifest.json. The $ROOT_NODE environment variable must be set to "file", or the name of the Guppy file index if it isn't "file". For BioDataCatalyst, the EXTRA_NODES environment variable must be set to "". (This is for backwards compatibility: pelican-export includes reference_file by default on all BDCat PFB exports unless $EXTRA_NODES is set to an empty string). Example sower config: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "action": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "container": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "job-task", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "image": "quay.io/cdis/pelican-export:0.5.0", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "pull_policy": "Always", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "env": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "DICTIONARY_URL", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "dictionary_url" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "GEN3_HOSTNAME", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "hostname" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "ROOT_NODE", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "file" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "EXTRA_NODES", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumeMounts": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/pelican-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "config.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/peregrine-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "creds.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "cpu-limit": "1", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "memory-limit": "4Gi" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumes": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "pelicanservice-g3auto" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "peregrine-creds" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "restart_policy": "Never" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)

• ssjdispatcher
  • metadata-service must already be present or added to the "versions": { section of the cdis-manifest. After metadata-service has been confirmed to be in the cdis-manifest, the now-required Metadata Service creds for SSJDispatcher must be configured by running gen3 roll all with JENKINS_HOME not set. gen3 roll all calls gen3 kube-setup-ssjdispatcher, which now automatically configures the SSJDispatcher secret with Metadata Service creds. (uc-cdis/(PXP-6977): make mds creds required for service startup ssjdispatcher#32)

[PlanXCyborg]

qa-bloodpac.planx-pla.net/manifest.json

Deployment changes

• fence
  • REQUIRES A FENCE DB MIGRATION, since with this update there is now an additional column in the authorization_code table "refresh_token_expires_in" (uc-cdis/Feat/refresh expires in fence#848)
  • Commons configured with RAS staging login must add the "discovery_url" fence config, because it now defaults to RAS prod (uc-cdis/Fix/ras url fence#824)
  • If the fence config already has [CLIENT_, USER_, SESSION_]ALLOWED_SCOPES fields, and GA4GH AAI integration is desired, then add ga4gh_passport_v1 to all those lists. If default config values are being used then no change required. (uc-cdis/PXP-6406 Feat/ga4gh scope fence#799)
• guppy
  • To deploy SearchFilters in a tab of the Explorer page, add an entry to data{/file}ExplorerConfig.filters.tabs.searchFields. NOTE: Do not add the same field to both searchFields and fields, this will cause strange behavior in the search field. Example config: (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "filters": { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "tabs": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • { (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "title": "Subject", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "searchFields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "subject_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "submitter_id", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "consent_codes" (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ], (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "fields": [ (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_type", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • "data_format", (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ... (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • ] (uc-cdis/(PXP-6359) Add Search Filter guppy#99)
  • } (uc-cdis/(PXP-6359) Add Search Filter guppy#99)

• portal
  • New optional config property, tierAccessLevel, in each tab of the explorerConfig block in the portal config. Used to enable index-scoped tiered-access, which is only available starting with Guppy server version 0.11.0. See README for more information. (uc-cdis/Feat/index scoped tiered access data-portal#802)
  • To deploy Discovery page: (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "featureFlags": { "discovery": true } must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • If discoveryConfig.features.authorization.enabled: true, "useArboristUI": true must be set in the portal config. (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • "discoveryConfig": <DiscoveryConfig> must be set in the portal config. DiscoveryConfig documentation can be found at https://github.com/uc-cdis/data-portal/blob/master/docs/portal_config.md (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • To upload data to Discovery page, data must be uploaded to environment's metadata-service instance with guid_type: "discovery_metadata". The metadata must all be under the top-level property gen3_discovery. (No documentation on this process yet, see this gist for right now https://gist.github.com/Avantol13/a53593837503fede2881f315198247c4) (uc-cdis/(PXP-7402) Discovery page data-portal#803)
  • Added new entry GWASApp for analysisTools for this new GWAS app for VA. Beaware not to confuse with the old "GWAS PoC" app, which bears the entry vaGWAS (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • antd modal now by default will use g3-color__base-blue as header background color, this is consistant with our color schemes for other UI components. If an env has customized color scheme, please override .ant-modal-header in gitops.css (uc-cdis/PXP-7296 PXP-7180 Feat/va gwas app data-portal#777)
  • The new feature of Portal requires Guppy v0.10.0 (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • Added new button types in the Portal config. See portal_config.md for an example. (uc-cdis/(PXP-6540): Feat/explorer download formats data-portal#780)
  • At time of writing, qa-midrc is only env using the new feature flag. These changes should be deployed alongside each other to production, but nothing will break if done separately. (uc-cdis/(PXP-7097) Add optional alternate TopBar profile menu data-portal#759)
  • New value for GEN3_BUNDLE env var GEN3_BUNDLE=nct (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • New optional portal config variable injectDAPTag (default value is false). If set to true, will inject DAP tag script into page header (this is required for NCT PROD env only) (uc-cdis/PXP-6895 NIAID CT bundle data-portal#742)
  • All envs should check if the dataExplorerConfig field in its portal config has ONLY arrangerConfig field but NO guppyConfig field (it is okay to have both). If so, please update dataExplorerConfig to use Guppy (uc-cdis/PXP-6409 Remove arranger data-portal#743)
  • A new Export to PFB button can be optionally enabled in the config.json. It is disabled by default. The button requires at least pelican:2020.10 release. (uc-cdis/Feat/export pfb to workspace data-portal#681)
  • Add GEN3_BUNDLE=covid19 as portal env var to enable covid bundle (uc-cdis/Feat/covid bundle data-portal#737)
  • Limited File PFB Export requires Pelican >= 0.5.0, Tube >= 0.4.1 (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export is enabled by setting fileExplorerConfig.enableLimitedFilePFBExport: { sourceNodeField: "source_node" } and by adding buttons of buttonType export-files (Export to Terra), export-files-to-pfb (Download PFB), or export-files-to-seven-bridges (Export to Seven Bridges) to fileExplorerConfig.buttons. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "fileExplorerConfig": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "buttons": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • .... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files-to-pfb", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to PFB", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enabled": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "type": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "title": "Export All to Terra", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "rightIcon": "external-link", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "tooltipText": "You have not selected any cases to export. Please use the checkboxes on the left to select specific cases you would like to export." (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "enableLimitedFilePFBExport": {"sourceNodeField": "source_node"}, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires the source_node property to be ETL'd -- source_node should be added to props in the file section of etlMapping.yaml. Example: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: mpingram_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • doc_type: file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • type: collector (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • root: None (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • category: data_file (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • props: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: object_id (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: md5sum (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ... (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • name: source_node (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • Limited File PFB Export requires a new export-files job block to be added to the Sower config in manifest.json. The $ROOT_NODE environment variable must be set to "file", or the name of the Guppy file index if it isn't "file". For BioDataCatalyst, the EXTRA_NODES environment variable must be set to "". (This is for backwards compatibility: pelican-export includes reference_file by default on all BDCat PFB exports unless $EXTRA_NODES is set to an empty string). Example sower config: (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "action": "export-files", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "container": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "job-task", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "image": "quay.io/cdis/pelican-export:0.5.0", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "pull_policy": "Always", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "env": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "DICTIONARY_URL", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "dictionary_url" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "GEN3_HOSTNAME", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "valueFrom": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "configMapKeyRef": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "manifest-global", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "key": "hostname" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "ROOT_NODE", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "file" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "EXTRA_NODES", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "value": "" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumeMounts": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/pelican-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "config.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "readOnly": true, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "mountPath": "/peregrine-creds.json", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "subPath": "creds.json" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "cpu-limit": "1", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "memory-limit": "4Gi" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "volumes": [ (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "pelican-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "pelicanservice-g3auto" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "name": "peregrine-creds-volume", (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secret": { (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "secretName": "peregrine-creds" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • } (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • ], (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • "restart_policy": "Never" (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)
  • }, (uc-cdis/(PXP-6544) Limited PFB Export from the Files Tab data-portal#729)

• ssjdispatcher
  • metadata-service must already be present or added to the "versions": { section of the cdis-manifest. After metadata-service has been confirmed to be in the cdis-manifest, the now-required Metadata Service creds for SSJDispatcher must be configured by running gen3 roll all with JENKINS_HOME not set. gen3 roll all calls gen3 kube-setup-ssjdispatcher, which now automatically configures the SSJDispatcher secret with Metadata Service creds. (uc-cdis/(PXP-6977): make mds creds required for service startup ssjdispatcher#32)