CutMix and Mosaic Augmentation

[awsaf49]

📚 Documentation

CutMix and Mosaic Augmentations are pretty good augmentation when it comes to achieve better score. It would be great if these two are included in albumentations.

[ternaus]

Could you please provide links to the description of Mosaic Augmentation?

[awsaf49]

Could you please provide links to the description of Mosaic Augmentation?

I think it was even mentioned in YOLOv4 paper. Here are some references:
https://www.kaggle.com/nvnnghia/awesome-augmentation
https://github.com/klauspa/Yolov4-tensorflow

[chamorajg]

Is someone working on this issue ? Can I pick this up and create a PR for this request ?

[Dipet]

Yes, you can try to implement this.

[senarvi]

I guess what makes it difficult is that you would need to implement an augmentation that takes multiple images as input. Is that even possible in Albumentations?

[mulcyber]

The BasicTransform (and all other transforms) take only 1 image a input and don't manage iterables themselves.
So I don't think it's possible without changing how albumentations or putting them as a different kind of object.

We would need to create an "BasicIterableTransform" than takes an iterable as input. But it wouldn't work with tools like torch.utils.data.Dataset without modification, need a new compose class, etc.

I agree it's a shame they're not included. It's super useful in detection problems, even though it requires quite a few modifications in albumentations and changes how the transforms would be used.

Any maintainer can tell us if it's the kind of features (namely Transforms taking iterables) that would be accepted/merged or it's just too different to how the library works now?

[dav-ell]

I've been working on a custom augment that performs mosaic by loading in 3 new images with whatever image is currently in the pipeline, but I got stuck at trying to return the new set of bounding boxes in the final moasic-ed image. Has anyone made progress on this?

[dav-ell]

I was able to create a custom version that fit my needs, but I couldn't get it to work with the albumentations pipeline. I ended up doing this (where self.mosaic :

# in my datamodule

self.mosaic = Mosaic(img_files, bbox_files, p=0.9) # requires all image and label files to create random mosaics

def transform(image, bboxes, class_labels, **params):
    """A wrapper for the albumentations pipeline"""
    image, bboxes, class_labels = self.mosaic(image, bboxes, class_labels)
    return self.transform(image=image, bboxes=bboxes, class_labels=class_labels, **params)

# create dataset, pass transform

That way my dataset could still call transform() with the usual albumentations API. However, it'd be nice to not have to use workarounds like this.

[i-aki-y]

I made a PR to add Mosaic augmentation with as few changes to the existing implementation as possible #1147.
Although some remaining works are to be done, please check it out if you are interested in this feature.
I welcome any comments and feedback.

[innat]

cc. @awsaf49
Cutmix is available officially in keras, HERE. The Mosaic one will be soon.

[awsaf49]

cc. @awsaf49
Cutmix is available officially in keras, HERE. The Mosaic one will be soon.

Thank you 😀

[ccl-private]

Hi @dav-ell
How to write albumentations pipeline wrapper? Can you provide your whole code?

[mikel-brostrom]

Any updates on Mosaic for Albumentations?

[ternaus]

Working on it.

[ankandrew]

Having Mosaic augmentation available in Albumentations would be great 🙏

[Buckler89]

@ternaus Any update?

[ternaus]

Postponed it till I start training object detection model myself. There are different ways to implement the transform, not sure which will be the most convenient.

But if someone want to write an implementation and create a pull request - would be happy to review and discuss.