Gaussian Blur layer implementation #143

artu1999 · 2022-02-20T17:29:27Z

implemented Gaussian Blur preprocessing layer using the gaussian blur implemented in tensorflow additions (Reference)

layer = GaussianBlur(kernel_size=(10,7), sigma=1)

layer = GaussianBlur(kernel_size=(10,7), sigma=2)

artu1999 · 2022-02-20T17:47:36Z

I have just realised the email I have for author wasn't matching the email on the CLA. In case no changes are required, let me know if I should revert and resubmit.

bhack · 2022-02-20T17:59:31Z

@qlzh727 Do you know if we have an update from the TF team on tensorflow/tensorflow#39050?

LukeWood · 2022-02-21T02:20:13Z

FYI @artu1999 we'll need you to sign the CLA to contribute code. Thanks

LukeWood · 2022-02-21T02:21:07Z

keras_cv/layers/preprocessing/gaussian_blur.py

+            training = backend.learning_phase()
+
+        def _blur(image, kernel_size, sigma):
+            image = tfa.image.gaussian_filter2d(


we don't rely on TFA in this repo. We will need to ship our own implementation. Gaussian blurs can actually be implemented through convolution (if tf.image doesn't have anything we can use). Perhaps check the SimCLR repo for a reference implementation

LukeWood · 2022-02-21T02:21:14Z

keras_cv/layers/preprocessing/gaussian_blur.py

+        Returns:
+            images: Blurred input images, same as input.
+        """
+        if training is None:


assume training is a python bool.

LukeWood · 2022-02-21T02:21:18Z

keras_cv/layers/preprocessing/gaussian_blur.py

+        Returns:
+            images: Blurred input images, same as input.
+        """
+        if training is None:


assume training is a python bool.

LukeWood · 2022-02-21T02:22:06Z

Thanks for the PR . I've left some early comments.

artu1999 · 2022-02-21T11:08:50Z

@LukeWood I have realised I used the wrong author name, I will make sure to use the correct one for the next commit. In terms of the next steps:

Is there any desire of making the Blur random? the implementation in simCLR already does that (Reference). Examples where Random Blur is used:

https://arxiv.org/pdf/2009.00104.pdf

https://arxiv.org/pdf/2006.10955.pdf

I think it might be something very useful as other libraries don't seem to support this.

Will start readapting the code so it won't rely on tfa.
Include training in docstring

LukeWood · 2022-02-21T17:57:59Z

@LukeWood I have realised I used the wrong author name, I will make sure to use the correct one for the next commit. In terms of the next steps:

Is there any desire of making the Blur random? the implementation in simCLR already does that (Reference). Examples where Random Blur is used:

https://arxiv.org/pdf/2009.00104.pdf

https://arxiv.org/pdf/2006.10955.pdf

I think it might be something very useful as other libraries don't seem to support this.

Will start readapting the code so it won't rely on tfa.

Include training in docstring

I'd be in favor of support for random behavior. For example, we could allow filter_size to support ranges as well as sigma to support ranges, then randomly sample within the range.

bhack · 2022-02-21T20:40:52Z

I'd be in favor of support for random behavior. For example, we could allow filter_size to support ranges as well as sigma to support ranges, then randomly sample within the range.

@LukeWood Are we going to always be random inside the batch? Cause I think that this sometimes is going already to complicate the vectorizzation a bit. Do we have any reference about the convergence improvement with in-batch randomization?
If we could randomize only between the batches instead I think that a Random could be more a policy class then something that we need to be supported with every args at preprocessing layer level.

LukeWood · 2022-02-21T20:46:46Z

I'd be in favor of support for random behavior. For example, we could allow filter_size to support ranges as well as sigma to support ranges, then randomly sample within the range.

@LukeWood Are we going to always be random inside the batch? Cause I think that this sometimes is going already to complicate the vectorizzation a bit. Do we have any reference about the convergence improvement with in-batch randomization? If we could randomize only between the batches instead I think that a Random could be more a policy class then something that we need to be supported with every args at preprocessing layer level.

We are leaning towards guiding users towards implementing:
augment_image

Then relying on XLA to perform optimization. This may be slightly less performant but the entire JAX ecosystem relies on this, so it should actually be fine. We want to minimize dependence between samples so this is optimal.

bhack · 2022-02-21T20:57:00Z

Yes but also if you want to use XLA or vectorized_map does it make sense to have the preprocessing layer itself Random? Isn't better to have a Random policy class that handle the randomization of the args for the underline preprocessing layer?
In that case this could enable more complex policies still controlling the preprocessing args.

artu1999 · 2022-02-22T11:14:26Z

Having a higher level Random Blur layer might also be useful in case we want to give users the possibility of applying random MotionBlur and MedianBlur as well. Possibly, we could use an argument to specify which type of random blur the user wants to apply or even apply a random type of Blur for each picture in the batch.

@joker-eph

As we are discussed in keras-team#143 (comment) this is going to fail (check the CI) cause: ``` ValueError: Input "maxval" of op 'RandomUniformInt' expected to be loop invariant. ``` As I've mentioned in the thread we really need to understand if we want to have randomness inside the batch or between the batches and what kind of impact we have between the computing overhead, contributing speed/code readability and network convergence. Also I don't know if @joker-eph or @qlzh727 could expose use a little bit the pro and cons of `jit_comile` a function vs using the `vectorized_map` or if they are orthogonal. With many CV transformations we cannot compile the function as the underline `tf.raw_ops.ImageProjectiveTransformV3 ` op isn't supported by XLA.

qlzh727 · 2022-02-22T19:04:11Z

@qlzh727 Do you know if we have an update from the TF team on tensorflow/tensorflow#39050?

I don't think there is any active development on that issue at the moment.

LukeWood · 2022-03-10T19:04:13Z

Yes but also if you want to use XLA or vectorized_map does it make sense to have the preprocessing layer itself Random? Isn't better to have a Random policy class that handle the randomization of the args for the underline preprocessing layer? In that case this could enable more complex policies still controlling the preprocessing args.

This sounds really heavy handed. I think we just want the layers to handle randomization themselves

bhack · 2022-03-11T16:43:11Z

This sounds really heavy handed. I think we just want the layers to handle randomization themselves

I still believe that I will be harder to remove the implicit randomization over the (hype)parameters space especially as we don't expose the non randomized ops in the internal API.

LukeWood · 2022-03-21T20:49:33Z

@artu1999 any update here?

artu1999 · 2022-03-21T20:55:31Z

@artu1999 any update here?

Sorry for the inactivity, got quite busy with my final university project. I am going to make a commit either tomorrow or the day after with the new code, I just need to fix a couple of things here and there.

artu1999 · 2022-03-22T12:28:34Z

Also, quick question: would you be happy with using keras.engine.base_layer.BaseRandomLayer as the class to inherit from or would you prefer to stick with keras.layers.Layer?

LukeWood · 2022-03-22T18:16:09Z

Hey @artu1999 - Please use BaseImageAugmentationLayer. Please see RandomShear as an example. We will be moving all of our KPL layers to use this:

https://github.com/keras-team/keras-cv/blob/master/keras_cv/layers/preprocessing/random_shear.py#L21

Basically, you write a function augment_image to augment a single image and it handles vectorization and input format processing for you :)

artu1999 · 2022-03-22T20:50:33Z

Hey @artu1999 - Please use BaseImageAugmentationLayer. Please see RandomShear as an example. We will be moving all of our KPL layers to use this:

https://github.com/keras-team/keras-cv/blob/master/keras_cv/layers/preprocessing/random_shear.py#L21

Basically, you write a function augment_image to augment a single image and it handles vectorization and input format processing for you :)

Will use that layer then, thanks!

LukeWood · 2022-04-01T17:12:49Z

Closing for your other PR. Thanks @artu1999 for starting this process

* Fix discrepancy with softmax when axis=None numpy and jax handled this as an all reduce option tf as a reduction on the last dimension * Fix format

Add Gaussian Blur layer implementation

93f10db

bhack mentioned this pull request Feb 20, 2022

tf.nn.depthwise_conv2d with rank=1 kernels (separable filters) tensorflow/tensorflow#39050

Closed

LukeWood reviewed Feb 21, 2022

View reviewed changes

bhack mentioned this pull request Feb 22, 2022

[Canary test] GridMask: Vectorized function call on the batch dimension #146

Closed

bhack mentioned this pull request Mar 11, 2022

tfa.image.gaussian_filter2d doesn't support tensor inputs for "filter_shape" and "sigma" args tensorflow/addons#2617

Open

bhack mentioned this pull request Mar 23, 2022

Gaussian Blur in regularization #199

Closed

artu1999 mentioned this pull request Apr 1, 2022

Gaussian blur #251

Closed

LukeWood closed this Apr 1, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Gaussian Blur layer implementation #143

Gaussian Blur layer implementation #143

artu1999 commented Feb 20, 2022

artu1999 commented Feb 20, 2022 •

edited

Loading

bhack commented Feb 20, 2022

LukeWood commented Feb 21, 2022

LukeWood Feb 21, 2022

LukeWood Feb 21, 2022

LukeWood Feb 21, 2022

LukeWood commented Feb 21, 2022

artu1999 commented Feb 21, 2022

LukeWood commented Feb 21, 2022

bhack commented Feb 21, 2022

LukeWood commented Feb 21, 2022

bhack commented Feb 21, 2022 •

edited

Loading

artu1999 commented Feb 22, 2022 •

edited

Loading

qlzh727 commented Feb 22, 2022

LukeWood commented Mar 10, 2022

bhack commented Mar 11, 2022

LukeWood commented Mar 21, 2022

artu1999 commented Mar 21, 2022

artu1999 commented Mar 22, 2022

LukeWood commented Mar 22, 2022

artu1999 commented Mar 22, 2022

LukeWood commented Apr 1, 2022

Gaussian Blur layer implementation #143

Gaussian Blur layer implementation #143

Conversation

artu1999 commented Feb 20, 2022

artu1999 commented Feb 20, 2022 • edited Loading

bhack commented Feb 20, 2022

LukeWood commented Feb 21, 2022

LukeWood Feb 21, 2022

Choose a reason for hiding this comment

LukeWood Feb 21, 2022

Choose a reason for hiding this comment

LukeWood Feb 21, 2022

Choose a reason for hiding this comment

LukeWood commented Feb 21, 2022

artu1999 commented Feb 21, 2022

LukeWood commented Feb 21, 2022

bhack commented Feb 21, 2022

LukeWood commented Feb 21, 2022

bhack commented Feb 21, 2022 • edited Loading

artu1999 commented Feb 22, 2022 • edited Loading

qlzh727 commented Feb 22, 2022

LukeWood commented Mar 10, 2022

bhack commented Mar 11, 2022

LukeWood commented Mar 21, 2022

artu1999 commented Mar 21, 2022

artu1999 commented Mar 22, 2022

LukeWood commented Mar 22, 2022

artu1999 commented Mar 22, 2022

LukeWood commented Apr 1, 2022

artu1999 commented Feb 20, 2022 •

edited

Loading

bhack commented Feb 21, 2022 •

edited

Loading

artu1999 commented Feb 22, 2022 •

edited

Loading