Corrected the epsilon value (#665)

* Corrected the epsilon value * Corrected the epsilon value * Corrected the epsilon value * Corrected the epsilon value * Corrected the epsilon value * Update bert_backbone.py
keras-team · Jan 17, 2023 · 99edccf · 99edccf
1 parent 544aca0
commit 99edccf
Showing 1 changed file with 1 addition and 0 deletions.
diff --git a/keras_nlp/models/bert/bert_backbone.py b/keras_nlp/models/bert/bert_backbone.py
@@ -163,6 +163,7 @@ def __init__(
                     x, approximate=True
                 ),
                 dropout=dropout,
+                layer_norm_epsilon=1e-12,
                 kernel_initializer=bert_kernel_initializer(),
                 name=f"transformer_layer_{i}",
             )(x, padding_mask=padding_mask)