Add an add_prefix_space Arg in BytePairTokenizer

[shivance]

Closes #436

cc : @abheesht17

[abheesht17]

Will this work 🤔? We are passing this argument in this function, but not actually doing anything with it inside the function.

I was thinking of doing something along these lines in the tokenize() function:

    def tokenize(self, inputs):
        if not isinstance(inputs, (tf.Tensor, tf.RaggedTensor)):
            inputs = tf.convert_to_tensor(inputs)
        if self.add_prefix_space:
            inputs = tf.strings.join([" ", inputs])
        ...

Trial:

>>> import tensorflow as tf
>>> inputs = tf.constant(["add space at the beginning of every string", "we can use tf.strings.join(...)"])
>>> inputs
<tf.Tensor: shape=(2,), dtype=string, numpy=
array([b' add space at the beginning of every string',
       b' we can use tf.strings.join(...)'], dtype=object)>

CC: @mattdangerw

[shivance]

Not sure why code format check fails

Ran format script locally

[abheesht17]

Not sure why code format check fails

Ran format script locally

Upgrade black version.

pip install --upgrade black

[abheesht17]

@shivance, please add a unit test as well

[shivance]

Thanks @abheesht17 , just found #708

[shivance]

cc: @jbischof
Ready for review

[chenmoneygithub]

Thanks for the PR! Also thanks Abi for the careful review!

[shivance]

Yes , definitely Thanks @abheesht17 !!

[mattdangerw]

This looks good to me! One minor comment on the docstring.

[mattdangerw]

Thank you!!