Support for Non-RoPE Models?

[mishalydev]

Hi,

First of all, thank you for the great work on this well-structured and accessible codebase—it’s been a pleasure to explore!

I’m trying to adapt your code for a non-RoPE model, but I’ve encountered issues when running it in different sections of the pipeline. I wanted to check:

• Is support for non-RoPE models currently implemented?
• If not, do you have any suggestions or pointers on how I might go about making the necessary modifications?

Thank you in advance for your help!