Is it possible to convert a normal Pytorch checkpoint to a deepspeed-compatible checkpoint?

[richarddwang]

My goal is to use deepspeed zero inference on bloomz model, or even any model on huggingface hub.

Specifically, I would like to download model weights from huggingface hub repo (e.g. bloomz), and convert them to deepspeed compatible checkpoints through a script. (basically a reverse one of zero_to_fp32.py?)

Are there existing scripts for this or similar purpose?
Or is it possible to write one? (then where should I start?)

[tjruwase]

This should work out of the box, please see https://github.com/huggingface/transformers-bloom-inference/blob/main/bloom-inference-scripts/bloom-ds-zero-inference.py

[richarddwang]

Didn't notice the script is also compatible to zero inference. Thank you! You saved me a lot of time.