-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MetaTensor
: collate; decollate; dataset; dataloader; out=; indexing and iterating across batches
#4137
MetaTensor
: collate; decollate; dataset; dataloader; out=; indexing and iterating across batches
#4137
Conversation
Signed-off-by: Richard Brown <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @rijobro ,
Thanks for the quick update.
I am wondering whether it's possible to give a warning if user doesn't use our list_data_collate
? Otherwise, I feel it is easily ignored and lead to unknown errors..
Thanks.
/build |
Signed-off-by: Richard Brown <[email protected]>
@ericspod @wyli @Nic-Ma I upgraded the logic so that a dl = DataLoader(ds)
batch = next(iter(dl))
batch[0] # should only return the 0th image and 0th metadata
batch[:, -1] # should return all metadata
batch[..., -1] # should return all metadata
next(iter(batch)) # should return 0th image and 0th metadata. |
Signed-off-by: Richard Brown <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
Signed-off-by: Richard Brown <[email protected]>
@Nic-Ma could you run |
MetaTensor
: collate , decollate, dataset, dataloader, out=MetaTensor
: collate; decollate; dataset; dataloader; out=; indexing and iterating across batches
Hi @rijobro , Thanks for your update. Thanks. |
/build |
From the following code snippet, the first is good (uses our collation fn). The middle case can have a warning. But how would we do the last case? Or are you just hoping for a warning in the middle case? Edit: import torch
from monai.data.meta_tensor import MetaTensor
from monai.data.dataset import Dataset
from monai.data.dataloader import DataLoader
from torch.utils.data import DataLoader as TorchDataLoader
data = [MetaTensor(torch.rand((1, 20, 20))) for _ in range(5)]
ds = Dataset(data)
# good
dl = DataLoader(ds)
next(iter(dl))
# can report warning
dl_no_collate = DataLoader(ds, collate_fn=lambda x: x)
next(iter(dl_no_collate))
# can't report warning
torch_dl = TorchDataLoader(ds)
next(iter(torch_dl)) |
Hi @rijobro , Yeah, I was asking about the last case, I understand it's hard to give a warning there. Just worried about user experience. Thanks. |
I don't have any better ideas at the moment, but will certainly reflect on it. The
|
Add padding to filter to ensure same size after anti-aliasing Use replicate padding insteadof zero padding to avoid artifacts for non-zero boundary Reuse GaussianSmooth 4073 Enhance DynUNet doc-strings (Project-MONAI#4102) * Fix doc strings error Signed-off-by: Yiheng Wang <[email protected]> * remove duplicate places Signed-off-by: Yiheng Wang <[email protected]> 4105 drops pt16 support (Project-MONAI#4106) * update sys req Signed-off-by: Wenqi Li <[email protected]> * temp test Signed-off-by: Wenqi Li <[email protected]> * update code for torch>=1.7 Signed-off-by: Wenqi Li <[email protected]> * temp tests Signed-off-by: Wenqi Li <[email protected]> * fixes tests Signed-off-by: Wenqi Li <[email protected]> * autofix Signed-off-by: Wenqi Li <[email protected]> * fixes import Signed-off-by: Wenqi Li <[email protected]> * clear cache Signed-off-by: Wenqi Li <[email protected]> * update based on comments Signed-off-by: Wenqi Li <[email protected]> * remove temp cmd Signed-off-by: Wenqi Li <[email protected]> Make `pixelshuffle` scriptable (Project-MONAI#4109) * Update the existing functionality to comply with the `torchscript.jit.script` function. Signed-off-by: Ramon Emiliani <[email protected]> meta tensor (Project-MONAI#4077) * meta tensor Signed-off-by: Richard Brown <[email protected]> 4084 Add kwargs for `Tensor.to()` in engines (Project-MONAI#4112) * [DLMED] add kwargs for to() API Signed-off-by: Nic Ma <[email protected]> * [MONAI] python code formatting Signed-off-by: monai-bot <[email protected]> * [DLMED] fix typo Signed-off-by: Nic Ma <[email protected]> * [DLMED] fix flake8 Signed-off-by: Nic Ma <[email protected]> * [DLMED] update according to comments Signed-off-by: Nic Ma <[email protected]> Co-authored-by: monai-bot <[email protected]> fixes pytorch version tests (Project-MONAI#4127) Signed-off-by: Wenqi Li <[email protected]> update meta tensor api (Project-MONAI#4131) * update meta tensor api Signed-off-by: Wenqi Li <[email protected]> * update based on comments Signed-off-by: Wenqi Li <[email protected]> runtests.sh isort (Project-MONAI#4134) Signed-off-by: Richard Brown <[email protected]> update citation (Project-MONAI#4133) Signed-off-by: Wenqi Li <[email protected]> `ToMetaTensor` and `FromMetaTensor` transforms (Project-MONAI#4115) to and from meta no skip if before pytorch 1.7 (Project-MONAI#4139) * no skip if before pytorch 1.7 Signed-off-by: Richard Brown <[email protected]> * fix Signed-off-by: Richard Brown <[email protected]> * fix Signed-off-by: Richard Brown <[email protected]> [DLMED] fix file name in meta (Project-MONAI#4145) Signed-off-by: Nic Ma <[email protected]> 4116 Add support for advanced args of AMP (Project-MONAI#4132) * [DLMED] fix typo in bundle scripts Signed-off-by: Nic Ma <[email protected]> * [DLMED] add support for AMP args Signed-off-by: Nic Ma <[email protected]> * [MONAI] python code formatting Signed-off-by: monai-bot <[email protected]> * [DLMED] fix flake8 Signed-off-by: Nic Ma <[email protected]> Co-authored-by: monai-bot <[email protected]> New wsireader (Project-MONAI#4147) `MetaTensor`: collate; decollate; dataset; dataloader; out=; indexing and iterating across batches (Project-MONAI#4137) `MetaTensor`: collate; decollate; dataset; dataloader; out=; indexing and iterating across batches (Project-MONAI#4137)
Add collation and decollation for
MetaTensor
. Also add support forout=
used as a kwarg, e.g.,torch.add(a,b,out=c)
.Non-breaking, so going into
dev
.Status
Ready
Types of changes