Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Input dropout #5636

Merged
merged 12 commits into from
Apr 21, 2020
Merged

Input dropout #5636

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
2c8f974
add dropout before ffnn
Ghostvv Apr 15, 2020
948ccc8
update changelog
Ghostvv Apr 15, 2020
faed41d
Merge branch '1.9.x' into input-dropout
Ghostvv Apr 16, 2020
13d1193
add pass training to input dropout
Ghostvv Apr 16, 2020
73213bf
Update changelog/5636.bugfix.rst
Ghostvv Apr 16, 2020
4a919ea
Merge branch 'master' into input-dropout
Ghostvv Apr 17, 2020
cd48a26
Merge branch 'master' into input-dropout
Ghostvv Apr 20, 2020
b179c51
add dense input dropout as a parameter
Ghostvv Apr 20, 2020
104987a
only apply input dropout to text
Ghostvv Apr 21, 2020
7917324
update changelog
Ghostvv Apr 21, 2020
754b932
fix number reference for changelog for preveous PR
Ghostvv Apr 21, 2020
0c60176
fix defaults in docs
Ghostvv Apr 21, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog/5636.bugfix.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add dropout before FFNN after sparse for dense layer in ``DIETClassifier``
6 changes: 5 additions & 1 deletion rasa/nlu/classifiers/diet_classifier.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1077,6 +1077,9 @@ def _prepare_input_layers(self, name: Text) -> None:
self._tf_layers[f"sparse_dropout.{name}"] = layers.SparseDropout(
rate=self.config[DROP_RATE]
)
self._tf_layers[f"dropout.{name}"] = tf.keras.layers.Dropout(
rate=self.config[DROP_RATE]
)
self._prepare_sparse_dense_layers(
self.data_signature[f"{name}_features"],
name,
Expand Down Expand Up @@ -1185,7 +1188,8 @@ def _combine_sparse_dense_features(
else:
dense_features.append(f)

return tf.concat(dense_features, axis=-1) * mask
outputs = tf.concat(dense_features, axis=-1) * mask
return self._tf_layers[f"dropout.{name}"](outputs, self._training)

def _features_as_seq_ids(
self, features: List[Union[np.ndarray, tf.Tensor, tf.SparseTensor]], name: Text
Expand Down