Exported beam search model consumes a lot of more memory

[HaoboGu]

Hello, I used convert_beam_search.py in latest master to convert my gpt2 model to beam search gpt2 model. I found that loading converted beam search model consumes more memory than raw gpt2 model(approximately doubled). Here is the screen shot for loading my gpt2 model before and after the convension:

I also tried using tiny-gpt2 on huggingface model hub, the memory consumption also increases a lot(30MB -> 43MB) after converting tiny-gpt2 to beam search model:

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS Monterey
• ONNX Runtime installed from (source or binary): ort-nightly
• ONNX Runtime version: 1.12.0
• Python version: 3.7.9

[tianleiwu]

I can reproduce the issue. Let me investigate the cause.

[tianleiwu]

@HaoboGu, please git pull the master branch, and re-generate the onnx model with convert_generation.py.

[HaoboGu]

install from source is quite slow.. is ort-nightly which contains this fix released?