BALSAMIC priorities and roadmap Q4 2020

[hassanfa]

Long issue lists don't get done, and they are too focused to be defined as a project. Some issues are also part of other projects. This issue defines roadmap and priorities, with an aim for the next release. Hotfixes and emergency fixes are not included. If issues are missing, please create them.

Next release by end of October, 2020

New features:

• Feat: Start analysis from VCF Start analysis from VCF #408
• Feat: Start analysis from BAM/CRAM: Deliver BAM files as CRAM for WGS runs #303
• Feat: WGS CRAM conversion: Deliver BAM files as CRAM for WGS runs #303
• Feat: QC model for TGA, WES, and WGA: Create QC model for approving an analysis #394 Progress: added values for HSMETRICS_QC_CHECK #401
• Refactor: Merge workflows into one: One workflow instead of three workflow for QC, WGS, and WES #412 Done in One workflow for align&qc and variant calling #428
• Feat: Update GATK4: Update to GATK4 #278
• Feat: Filter for tumor-normal Germline variant uploaded to Scout for paired analysis #435 done in PR feat, add AF_ratio to Vardict filters #441
• Feat: QC for UMI: done in PR Features: Adds qc rules to UMI workflow #431
• Feat: quite mode for snakemake: Quiet mode #289 . Feature added in refactor reference, remove run reference #438

Documentation:

• Doc: Snakemake rule standards Add description of how to add new tools to Snakemake workflow(s) to documentation #406 Done in Feat/snakemakedoc #489
• Doc: BALSAMIC tutorial Step by step guide to setup BALSAMIC and up and running. #400
• Doc: Bioinfo tool description: Include detailed Bioinfo tool descriptions, links and citations in BALSAMIC report #339

Fixes:

• Increase memory for VarDict and Mutect2: 4 cases failed on "vardict_tumor_only" and "manta_tumor_only" #429 and [ Failed Analysis ]BALSAMIC.daringmidge.mutect2_tumor_normal #370 and Failed VarDict och Mutect2 on Java memory #427 Done in PR: Refactor/vardict muect2 resources #440
• Increase assigned cores for CNVKit TIME OUT on CNV-kit when ordering 90X WGS #476 Done in PR increase cnvkit queue time request from 8hour, 10 core to 10hour 10 core #482
• Fixed a bug to not run UMI quality trim for WGS applications [Bug] Do not run UMI quality trim for WGS applications #473 done in PR Disable UMI trim option for WGS #486

Refactor:

• Refactor: Merge run reference and config reference done in PR refactor reference, remove run reference #438
• Refactor: Align rules done in PR Refactor rules: align rules #434
• Refactor: Disable any variant caller done in PR Refactor/disable multiple variant callers #433
• Refactor: fix params to global constants and cleanup of UMI rule names: Features: Adds qc rules to UMI workflow #431
• Refactor: changed wildcard {sample} -> {case_name} in UMI workflow. Uses vep.somatic rule from balsamic workflow. Done in Refactor/umi wildcards #514

Items finished by September, 2020

New features:

• Feat: disable/enable variant callers: Disable/enable variant callers through CLI #411 done in Option to disable variant caller #417
• Feat: Reference hg38: Reference genome hg38 #301 done in feat, hg38 reference files #407

Fixes:

• Fix: cyvcf2 and cython missing package cyvcf2 and Cython not in requirements.txt #409 done in adding cyvcf2 and cython to requirements #413

Changelog 6.0.0 September 2020:

Added

• UMI workflow using Sentieon tools. Analysis run available via balsamic run analysis --help command. PR UMI-workflow using Sentieon tools #359
• VCFutils to create VCF from flat text file. This is for internal purpose to generate validation VCF. PR Feature: Adding VCFutils to balsamic plugins #349
• Download option for hg38 (not validated) PR feat, hg38 reference files #407
• Option to disable variant callers for WES runs. PR Option to disable variant caller #417

Fixed

• Missing cyvcf2 dependency, and changed conda environment for base environment PR adding cyvcf2 and cython to requirements #413
• Missing numpy dependency PR Release 6.0.0 #426

Changed

• COSMIC db for hg19 updated to v90 PR feat, hg38 reference files #407
• Fastp trimming is now a two-pass trimming and adapter trimming is always enabled. This might affect coverage slightly PR Feat/hard qc fastq #422
• All containers start with a clean environment Add sentieon env vars to config file to get in rules instead #425
• All Sentieon environment variables are now added to config when workflow executes Add sentieon env vars to config file to get in rules instead #425
• Branching model will be changed to gitflow

Issues to select from for the next release:

Features (new):

• Feat: Check tumor samples coming from same individual: Check that tumor samples come from the same individual #300
• Feat: Somatic variant caller filters: Somatic variant caller specific filters #117
• Feat: Annotate VCF from other sources than provided by VEP: Filter variants through COSMIC and cBioPortal #382
• Feat: Add tests to Docker container: Add tests to Docker container #299
• Feat: Custom annotation: Custom annotation addition (e.g. LoqusDB) #285
• Feat: Roh call and LoH: Add rhocall / LoH as for RD #277
• Feat: GFFToolkits: GFFToolkit for extracting exons #113
• Feat: Variant normalization: vt peek, decompose, normalize (or via bcftools or rtg tools) #108
• Feat: TMB calculation TMB calculation based on AF (a range of AF) #104 and Tumor mutational burden #51
• Feat: Fusioncatcher Fusioncatcher #102
• Feat: Tumor purity for WGS: Investigate Accurity for WGS applications and tumor purity #101
• Feat: Tumor purity and clones: Tumor heterogeneity calculations for WGS applications using: Theta2, phylowgs, or Hatchet #100
• Feat: Ti/Tv report and VCF statistics: Ti/Tv transition to transversion calculations #93
• Feat: panel of normals for Mutect2: Panel of normals for Mutect2 #90
• Feat: Second aligner for noise suppression: Second aligner to reduce artifacts risen from mapping algorithms #89
• Feat: Downsample options: Downsampling fq files for QC and Alignment workflow #84
• Feat: Plot functionality for coverage: Add plot functionality to coverage report script #75
• Feat: Selected SV visualization or Circos visualization: Structural variant visualization #55
• Feat: observed vs expected plots: Observed vs Expected variant frequency given a set of individual samples (N>=10) #25

Doc:

Fixes and refactors:

• Refactor: filter string distribution for variant callers Filter string distribution for each variant caller #146
• Refactor: tmp_dir workflow wide create util for creating tmp_dirs for each process on request per rule #397
• Refactor: use Python pathlib: Refactor os.path with pathlib.Path #377
• Refactor: decouple environments from single container: Decouple Docker containers and separate them #326
• Refactor: sample names in merged bam files: unique sample names to final bam #243
• Refactor: improve run time for Strelka: Strelka call regions for targeted analysis to reduce runtime. #118

[hassanfa]

This is mostly done. Will create a new one for 2021.