Add an add_prefix_space Arg in RobertaPreprocessor

[abheesht17]

RoBERTa uses the BPE tokeniser. If you see the vocabulary (https://huggingface.co/roberta-base/blob/main/vocab.json), most tokens have this character at the beginning: "Ġ". This is because whitespace is considered a part of the token. Setting add_prefix_space to True will add a space at the beginning of the input sequences, so that the token is not mapped to UNK.

Have a look here for more details: https://huggingface.co/docs/transformers/model_doc/roberta#transformers.RobertaTokenizer.

[chenmoneygithub]

@abheesht17 Thanks for reporting the issue!

I think the current BytePairTokenizer implementation is respecting whitespace, does it fail in your test case? The corresponding code is here: https://github.com/keras-team/keras-nlp/blob/master/keras_nlp/tokenizers/byte_pair_tokenizer.py#L46

[shivance]

Is this issue still open ? @chenmoneygithub @mattdangerw @jbischof

[mattdangerw]

Sure is! Would you like to take it on?

I think we can add this add_prefix_space argument to the base level BytePairTokenizer class. This will allow us to use it from all the models that need it (e.g. gpt2 and roberta).