4855 lazy resampling impl -- Compose #5860

wyli · 2023-01-17T00:38:04Z

part of #4855

upgrade #4911 to use the latest dev API

Description

e.g., for a sequence of spatial transforms

xforms = [
    mt.LoadImageD(keys, ensure_channel_first=True),
    mt.Orientationd(keys, "RAS"),
    mt.SpacingD(keys, (1.5, 1.5, 1.5)),
    mt.CenterScaleCropD(keys, roi_scale=0.9),
    # mt.CropForegroundD(keys, source_key="seg", k_divisible=5),
    mt.RandRotateD(keys, prob=1.0, range_y=np.pi / 2, range_x=np.pi / 3),
    mt.RandSpatialCropD(keys, roi_size=(76, 87, 73)),
    mt.RandScaleCropD(keys, roi_scale=0.9),
    mt.Resized(keys, (30, 40, 60)),
    # mt.NormalizeIntensityd(keys),
    mt.ZoomD(keys, 1.3, keep_size=False),
    mt.FlipD(keys),
    mt.Rotate90D(keys),
    mt.RandAffined(keys),
    mt.ResizeWithPadOrCropd(keys, spatial_size=(32, 43, 54)),
    mt.DivisiblePadD(keys, k=3),
]
lazy_kwargs = dict(mode=("bilinear", 0), padding_mode=("border", "nearest"), dtype=(torch.float32, torch.uint8))
xform = mt.Compose(xforms, lazy_evaluation=True, overrides=lazy_kwargs, override_keys=keys)
xform.set_random_state(0)

lazy_evaluation=True preserves more details

compared with the regular compose

Types of changes

Non-breaking change (fix or new feature that would not break existing functionality).
Breaking change (fix or new feature that would cause existing functionality to change).
New tests added to cover the changes.
Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
In-line docstrings updated.
Documentation updated, tested make html command in the docs/ folder.

Signed-off-by: Wenqi Li <[email protected]>

…-impl Signed-off-by: Wenqi Li <[email protected]>

Signed-off-by: Wenqi Li <[email protected]>

…-impl Signed-off-by: Wenqi Li <[email protected]>

tests/test_integration_lazy_samples.py

Signed-off-by: Ben Murray <[email protected]>

monai/transforms/compose.py

Nic-Ma

Looks good to me, looking forward to more tests and some bundle benchmarks before merging.

Thanks.

monai/transforms/lazy/functional.py

tests/test_integration_lazy_samples.py

monai/transforms/compose.py

wyli · 2023-03-23T12:49:14Z

thanks, I'm working on this branch to address the review comments.

atbenmurray · 2023-03-23T12:57:44Z

thanks, I'm working on this branch to address the review comments.

Ahh, @wyli I was in the process of adding the missing keywords to the overrides namespace, but if you are already doing it I'll leave it

wyli · 2023-03-23T13:07:29Z

thanks, I'm working on this branch to address the review comments.

Ahh, @wyli I was in the process of adding the missing keywords to the overrides namespace, but if you are already doing it I'll leave it

sure, I'll make minor usability fixes and merge it soon..

Signed-off-by: Wenqi Li <[email protected]>

wyli · 2023-03-23T18:15:28Z

/build

Signed-off-by: Wenqi Li <[email protected]>

wyli · 2023-03-23T18:29:21Z

/build

part of Project-MONAI#4855 upgrade Project-MONAI#4911 to use the latest dev API ### Description Example usage: for a sequence of spatial transforms ```py xforms = [ mt.LoadImageD(keys, ensure_channel_first=True), mt.Orientationd(keys, "RAS"), mt.SpacingD(keys, (1.5, 1.5, 1.5)), mt.CenterScaleCropD(keys, roi_scale=0.9), # mt.CropForegroundD(keys, source_key="seg", k_divisible=5), mt.RandRotateD(keys, prob=1.0, range_y=np.pi / 2, range_x=np.pi / 3), mt.RandSpatialCropD(keys, roi_size=(76, 87, 73)), mt.RandScaleCropD(keys, roi_scale=0.9), mt.Resized(keys, (30, 40, 60)), # mt.NormalizeIntensityd(keys), mt.ZoomD(keys, 1.3, keep_size=False), mt.FlipD(keys), mt.Rotate90D(keys), mt.RandAffined(keys), mt.ResizeWithPadOrCropd(keys, spatial_size=(32, 43, 54)), mt.DivisiblePadD(keys, k=3), ] lazy_kwargs = dict(mode=("bilinear", 0), padding_mode=("border", "nearest"), dtype=(torch.float32, torch.uint8)) xform = mt.Compose(xforms, lazy_evaluation=True, overrides=lazy_kwargs, override_keys=keys) xform.set_random_state(0) ``` lazy_evaluation=True preserves more details ![Screenshot 2023-01-17 at 00 31 40](https://user-images.githubusercontent.com/831580/212784981-ea39833b-54ab-42fb-bc03-38b012281857.png) compared with the regular compose ![Screenshot 2023-01-17 at 00 31 43](https://user-images.githubusercontent.com/831580/212785016-ba3be8ff-f17f-47b4-8025-cd351a637a82.png) ### Types of changes  - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [ ] In-line docstrings updated. - [ ] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: Wenqi Li <[email protected]> Signed-off-by: Yiheng Wang <[email protected]> Signed-off-by: KumoLiu <[email protected]> Signed-off-by: Ben Murray <[email protected]> Co-authored-by: Ben Murray <[email protected]> Co-authored-by: binliu <[email protected]> Co-authored-by: Yiheng Wang <[email protected]> Co-authored-by: YunLiu <[email protected]> Co-authored-by: Yiheng Wang <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: KumoLiu <[email protected]>

wyli mentioned this pull request Jan 17, 2023

[Prototype] 4855 compose with lazy resampling #4911

Closed

12 tasks

wyli force-pushed the 4855-lazy-resampling-impl branch from 690d075 to bd21ea0 Compare January 17, 2023 15:24

wyli added this to the Lazy resampling (+ cache-based dataset refactoring) [P1 v1.2] milestone Jan 17, 2023

wyli force-pushed the 4855-lazy-resampling-impl branch 3 times, most recently from 5e58297 to 42844e8 Compare January 24, 2023 00:38

wyli added 9 commits January 25, 2023 16:01

smaller copy

3083442

Signed-off-by: Wenqi Li <[email protected]>

fixes tests

1e15fd1

Signed-off-by: Wenqi Li <[email protected]>

fixe tests

88ab8ed

Signed-off-by: Wenqi Li <[email protected]>

fixes push transform

79f368e

Signed-off-by: Wenqi Li <[email protected]>

one of

edcd20f

Signed-off-by: Wenqi Li <[email protected]>

push transform tensor

256267b

Signed-off-by: Wenqi Li <[email protected]>

refactoring push transform

bd33d93

Signed-off-by: Wenqi Li <[email protected]>

fixes tests

2c040be

Signed-off-by: Wenqi Li <[email protected]>

default false lazy

7699634

Signed-off-by: Wenqi Li <[email protected]>

wyli force-pushed the 4855-lazy-resampling-impl branch from 45631e0 to 1232a6b Compare January 25, 2023 18:38

fixes tests

aa0389e

Signed-off-by: Wenqi Li <[email protected]>

wyli force-pushed the 4855-lazy-resampling-impl branch from 1232a6b to aa0389e Compare January 25, 2023 18:51

wyli added 3 commits January 25, 2023 21:08

update one_of/random_order tests

0fbbd78

Signed-off-by: Wenqi Li <[email protected]>

evaluate cases

a2a60f1

Signed-off-by: Wenqi Li <[email protected]>

no inplace meta change

fce044b

Signed-off-by: Wenqi Li <[email protected]>

wyli force-pushed the 4855-lazy-resampling-impl branch from 9790148 to fce044b Compare January 26, 2023 00:52

wyli added 8 commits January 26, 2023 01:40

update samples

d6d8e9b

Signed-off-by: Wenqi Li <[email protected]>

multisample cropping

4b062ef

Signed-off-by: Wenqi Li <[email protected]>

fixes tests

c00b2f6

Signed-off-by: Wenqi Li <[email protected]>

Merge remote-tracking branch 'upstream/dev' into 4855-lazy-resampling…

1f63bbc

…-impl Signed-off-by: Wenqi Li <[email protected]>

style fix

526f1f7

Signed-off-by: Wenqi Li <[email protected]>

fixes style

a61d3ef

Signed-off-by: Wenqi Li <[email protected]>

Merge remote-tracking branch 'upstream/dev' into 4855-lazy-resampling…

93784f4

…-impl Signed-off-by: Wenqi Li <[email protected]>

Merge remote-tracking branch 'upstream/dev' into 4855-lazy-resampling…

937744c

…-impl Signed-off-by: Wenqi Li <[email protected]>

atbenmurray mentioned this pull request Mar 22, 2023

Improve Compose encapsulation #6224

Merged

7 tasks

Nic-Ma reviewed Mar 22, 2023

View reviewed changes

tests/test_integration_lazy_samples.py Outdated Show resolved Hide resolved

wyli and others added 2 commits March 22, 2023 15:45

Merge branch 'dev' into 4855-lazy-resampling-impl

349dc09

Fix for non-trivial mappings between override_keys and _keys (#21)

40932e0

Signed-off-by: Ben Murray <[email protected]>

KumoLiu reviewed Mar 23, 2023

View reviewed changes

monai/transforms/compose.py Outdated Show resolved Hide resolved

wyli commented Mar 23, 2023

View reviewed changes

monai/transforms/compose.py Outdated Show resolved Hide resolved

wyli commented Mar 23, 2023

View reviewed changes

monai/transforms/compose.py Show resolved Hide resolved

Nic-Ma approved these changes Mar 23, 2023

View reviewed changes

monai/transforms/lazy/functional.py Outdated Show resolved Hide resolved

tests/test_integration_lazy_samples.py Outdated Show resolved Hide resolved

monai/transforms/compose.py Outdated Show resolved Hide resolved

Merge branch 'dev' into 4855-lazy-resampling-impl

17e4134

wyli and others added 7 commits March 23, 2023 14:22

Merge branch 'dev' into 4855-lazy-resampling-impl

12821a5

update integration tests

9d1fe50

Signed-off-by: Wenqi Li <[email protected]>

update device

cc73d6d

Signed-off-by: Wenqi Li <[email protected]>

update

b8674bf

Signed-off-by: Wenqi Li <[email protected]>

update

91adea2

Signed-off-by: Wenqi Li <[email protected]>

update based on comments

732ecd0

Signed-off-by: Wenqi Li <[email protected]>

update based on comments

28b60b9

Signed-off-by: Wenqi Li <[email protected]>

wyli enabled auto-merge (squash) March 23, 2023 18:17

update default flag

c7ec452

Signed-off-by: Wenqi Li <[email protected]>

wyli disabled auto-merge March 23, 2023 18:30

wyli changed the title ~~4855 lazy resampling impl~~ 4855 lazy resampling impl -- Compose Mar 23, 2023

wyli merged commit b87375f into Project-MONAI:dev Mar 23, 2023

wyli added this to the Lazy resampling (+ cache-based dataset refactoring) [P1 v1.2] milestone Mar 23, 2023

atbenmurray mentioned this pull request Apr 28, 2023

Lazy eval refactor #6257

Closed

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

4855 lazy resampling impl -- Compose #5860

4855 lazy resampling impl -- Compose #5860

wyli commented Jan 17, 2023 •

edited

Loading

Nic-Ma left a comment

wyli commented Mar 23, 2023

atbenmurray commented Mar 23, 2023

wyli commented Mar 23, 2023

wyli commented Mar 23, 2023

wyli commented Mar 23, 2023

4855 lazy resampling impl -- Compose #5860

4855 lazy resampling impl -- Compose #5860

Conversation

wyli commented Jan 17, 2023 • edited Loading

Description

Types of changes

Nic-Ma left a comment

Choose a reason for hiding this comment

wyli commented Mar 23, 2023

atbenmurray commented Mar 23, 2023

wyli commented Mar 23, 2023

wyli commented Mar 23, 2023

wyli commented Mar 23, 2023

wyli commented Jan 17, 2023 •

edited

Loading