Compiling Chronos-Bolt Models into ONNX

[Yashil]

Hello Chronos Community,

I'm currently working with the Chronos-Bolt models and aiming to export them to the ONNX format for deployment. However, I'm encountering issues during the export process related to unsupported operators and type errors.

Has anyone successfully exported a Chronos-Bolt model to ONNX? If so, could you share your experience or provide guidance on how to achieve this?

Any help or pointers would be greatly appreciated!

Thank you, Yashil.

[canerturkmen]

Hi @Yashil thanks for reaching out. I have a PoC here which I put together a while back. Would be great to hear if it works for you.

[Yashil]

Thanks a lot, @canerturkmen Your script worked perfectly and helped me compile the model into ONNX format. It was a huge help.

I did notice the following three TracerWarnings:

1. `...\chronos\chronos_bolt.py:270: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if context.shape[-1] > self.chronos_config.context_length:`
2. `...\chronos\chronos_bolt.py:58: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if length % self.patch_size != 0:`
3. `...\Lib\site-packages\transformers\modeling_utils.py:1017: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
  if causal_mask.shape[1] < attention_mask.shape[1]:`

The first two are coming from chronos_bolt.py, I suppose one can work around them (possibly by restructuring the if-statements to avoid directly casting tensors to Python booleans). The third warning is in the transformers library, which might be trickier to fix, so I’ll dig around in the Transformers repo and documentation for any known issues or workarounds.