Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add a subtle fix for gemma 2 conversions
Gemma 2 will use different normalization constants for the query depending of the model size. 9b = head_dim 27b = hidden_dim / num_query_heads We need to slightly tweak our config conversion to account for this.
- Loading branch information