[Misc] Use RoPE cache for MRoPE (vllm-project#8396)

Signed-off-by: Alvant <[email protected]>
compressa-ai · Oct 26, 2024 · bb48ba7 · bb48ba7
1 parent 5e69db4
commit bb48ba7
Showing 1 changed file with 1 addition and 1 deletion.
diff --git a/vllm/model_executor/layers/rotary_embedding.py b/vllm/model_executor/layers/rotary_embedding.py
@@ -990,7 +990,7 @@ def get_rope(
                 base, is_neox_style, dtype, short_factor, long_factor,
                 **extra_kwargs)
         elif scaling_type == "mrope":
-            return MRotaryEmbedding(
+            rotary_emb = MRotaryEmbedding(
                 head_size,
                 rotary_dim,
                 max_position,