Updated Reformer to use caching during generation

[guillaume-be]

What does this PR do?

The current reformer implementation supports caching of buckets and states, but this is not used during generation. Running a generation example in debugging mode, such as

from transformers import ReformerModelWithLMHead, ReformerTokenizer

model = ReformerModelWithLMHead.from_pretrained("google/reformer-crime-and-punishment").cuda()
tok = ReformerTokenizer.from_pretrained("google/reformer-crime-and-punishment")
output = tok.decode(
    model.generate(tok.encode("Notwithstanding", return_tensors="pt").cuda(),
                   do_sample=True,
                   temperature=0.7,
                   max_length=100,
                   use_cache=True)[0])

One can see that the past_buckets_states passed to the attention are always None (at

transformers/src/transformers/modeling_reformer.py

Line 365 in 504ff7b

past_buckets_states=None,

)

This is because the name of the past states for the reformer are neither past_key_values or mems.
This PR adds the name of the past states to the generation past allocation.

Generally, it may make sense to harmonize the past value for all models, so that the generate function generalizes better

Who can review?

Text Generation: @patrickvonplaten, @TevenLeScao
Reformer: @patrickvonplaten

[patrickvonplaten]

Great catch!

[patrickvonplaten]

Let's merge that quickly so that I can integrate it into https://github.com/huggingface/transformers/pull/6949/files#diff-b7601d397d5d60326ce61a9c91beaa2afa026014141052b32b07e1d044fbbe17

[patrickvonplaten]

Actually, we would have to add in two spots of this generate version. Considering that we will merge the big generate refactor today, I just added your fix quickly here: 12b54ec

Mentioned your PR at the fix - hope it's ok for you to close this PR to avoid any more merge conflicts.

Thanks a lot!