Skip to content

Commit

Permalink
Update word_piece_tokenizer.py (#617)
Browse files Browse the repository at this point in the history
* Update word_piece_tokenizer.py

* Revert "Update word_piece_tokenizer.py"

This reverts commit fcad4b8.

* Update word_piece_tokenizer.py
  • Loading branch information
ADITYADAS1999 authored Jan 4, 2023
1 parent 340a5cc commit 7774071
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion keras_nlp/tokenizers/word_piece_tokenizer.py
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@

def pretokenize(
text,
lowercase=True,
lowercase=False,
strip_accents=True,
split=True,
split_on_cjk=True,
Expand Down

0 comments on commit 7774071

Please sign in to comment.