NNI 2021 May~June Iteration Planning

[kvartet]

This is the plan for the iteration in May~June, it's a 6 weeks iteration. It has been tracked in the v2.3 Project.

Release Plan for v2.3

• Release manager: @ultmaster
• Feature freeze date: 5.21
• Code freeze and demo date: 5.27
• Branch cut and next release planning date: 6.4
• Release date: 6.9

---

Retiarii & NAS

• P0 - More high-level APIs for users to easily express model space @ultmaster (PR [Retiarii] New API: Repeat and Cell #3481)
• P0 - TensorFlow support: graph converter and support inline mutation API @liuzhe-lz
• P0 - NAS visualization improvements: refactor graph representation @98may 5.14
• P1 - Pure-python execution engine @ultmaster 5.7
• P1 - Support RL strategy @ultmaster
• P1 - Further improve the logic of graph generation and code generation @QuanluZhang 5.20
  • Further integrated with TorchScript
  • Bug fix (list: Graph converter does not work for nested ModuleList #3573)
• new NAS algorithm: Blockwise DNAS FBNet
• P1 - Support one system-level optimization: CSE optimization to deduplicate computation

Model Compression

• P0 - Auto shape inference @J-shang taking from @zheng-ningxin
• P1 - Multi-compressor @J-shang 4.30
• P1 - Auto compressor @J-shang
• P1 - collaboration with nnmeter @zheng-ningxin
• P1 - module compression pipeline @colorjam
• P1 - Support any tensorflow algorithm @liuzhe-lz 4.30
• Add LSQ quantizer
• P2 - Add exclude config to AGP pruner (discussion needed)
• P2 - Dependency-aware pruner with schedule (discussion needed)

Training service & NNI manager && nnictl

• P0 - Add test for shared storage (PR Add shared storage integration test #3455) @acured 4.30
• P0 - Make reuse default (needs check) @SparkSnail
• P1 - Update documentation and pipeline to v2 @SparkSnail
• P1 - Remove launching tensorboard from nnictl @SparkSnail
• P1 - Add the output of subprocess (Add output of subprocess in trial #3141) @acured
• fix v2 aml config and set version check as a warning
• improve NNI manager logging
• P2 - Hybrid kubeflow support (migrate kubeflow to environment service) @SparkSnail
• P2 - Mask confidential config fields (depends on prev items) @liuzhe-lz
• P2 - Update training service interface @liuzhe-lz (design review)
• P2 - nnictl experiment export improvements @J-shang

WebUI

• P0 - Make selected trials consistent in detail table @Lijiaoa 5.7
• P1 - Improve search parameters on trial detail page @Lijiaoa
• P1 - set base url for the web report (Is there any option to set base url for the web report? #2771) @acured
• add trial stdout button on local mode
• P2 - Show dict intermediate result in the dialog of single trial intermediate result @Lijiaoa

UX Design

• P1 - NAS visualization
  • UX Design - concept design @QuanluZhang / @scarlett2018
  • UX Design - formal design @yangou1988
• P2 - Adaptive UI (height) for high resolution, current window for high resolution need improvement @yangou1988 @Lijiaoa
• Support experiment resume/view/stop on webUI
• Experiment/Project Management: tag/description

Hyper-parameter tuning

• P1 - HPO benchmark @xiaowu0162
• P1 - DNGO tuner @98may
• P2 - Improve hyperband, PBT with shared storage (dependency on shared storage implementation) @J-shang

Pipeline

• P0 - add pipeline badge on readme @ultmaster
• P0 - add aml pipeline test @SparkSnail (Support aml pipeline #3477 draft pr)
• P0 - add hybrid pipeline test @SparkSnail (Support hybrid pipeline #3478 draft pr)

Demo

• v2.2 example and illustration for new features @kvartet
• Add new demos in more details @kvartet

[ultmaster]

Endgame

Tests

NAS

• Repeat and Cell in Retiarii high-level APIs ([Retiarii] New API: Repeat and Cell #3481) Quanlu
• Pure-python execution engine ([Retiarii] Pure-python execution engine #3605) Quanlu
• Policy-based RL strategy ([Retiarii] Policy-based RL Strategy #3650) Quanlu
• Nested ModuleList and a special case in list append in Retiarii (Support nested ModuleList and fix an issue in list append #3652) Yuge
• Blockwise DNAS FBNet (Request for add NAS algorithm: Blockwise DNAS FBNet #3532) Yuge
• Fix experiment_working_directory in Retiarii config (fix retiarii config experiment_working_directory #3607) Yuge

Model compression

• Auto compression ([Model Compression] auto compression #3631) Bin
• Model compression e2e example ([Model Compression] Update api of iterative pruners #3507) ([#3507 follow up] update doc #3688) Ning
• Tensorflow slim pruner (Add TensorFlow slim pruner #3614) Ning
• LSQ quantizer (Add LSQ quantizer #3503) Bin
• Fix a bug in mask conflict (fix the bug in line no.336 in mask_conflict.py #3629) Ning
• Fix a bug in model speedup shape inference (fix the bug in nni/compression/pytorch/speedup/infer_shape.py, line no. 625-626 #3588) Ning

Training service and Rest

• Support for 3rd-party training service (Support 3rd-party training service #3662) Shinai
• Track trial stdout (add trial stdout button on local mode #3690) (fix no log from subprocess on trial #3653) Yuge
• Prefix URL in RestFUL API (Support hybrid and V2 config pipieline #3648) ([webui] add basename in router #3625) Zhe (Hao)
• Fix an issue in AML V2 config (Fix aml v2 config #3655) Zhe
• Fix syntax error on Windows (Fix syntax error on windows #3634) Ning
• Report version check failure as warning (Set version check as warning #3654) Yiru
• Fix an issue that causes missing log and metric on Windows when using PyTorch with multithread (Fix on log and metric issue on Windows when using pytorch with multithread #3604)
• Improve NNI manager logging (Improve NNI manager logging #3624) Shinai
• Delete redundant code in training service (Delete redundant code #3526)

HPO

• DNGO tuner (DNGO tuner #3479) Yuge
• HPO benchmark (HPO Benchmark #3644) Ayan

WebUI

• Improved search parameters on trial detail page (Improve search parameters on trial detail page #3651) Yuge
• Make selected trials consistent after auto-refresh in detail table (Make selected trials consistent across auto-refresh in detail table #3597) Yuge

nnictl

• Delete tensorboard on nnictl (delete tensorboard on nnictl #3613)

Examples

• Add notebook for tabular data classification in AML (Add notebook for tabular data classification in AML #3599) Quanlu

Pipeline and tests

• Hybrid and V2 config pipeline (Support hybrid and V2 config pipieline #3648)
• AML pipeline (Support aml pipeline #3477)
• Unpin pytorch_lightning<1.2 (Unpin pytorch_lightning<1.2 #3598)
• Fix an issue in documentation of Hybrid training service (Fix Hybrid mode doc #3600)
• Pin yargs<17.0 (Pin yargs<17.0 #3608)
• Update issue bug report template (update issue bug report template #3501)
• Add pipeline badge in README (Add pipeline badge to README #3589)

Documentation and example work

• Promote Retiarii docs and deprecate old NAS framework
• Convert all v1 examples to v2

[kvartet]

closed as v2.3 has been released.