New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix make_mlp_unpacked with lora #1209

Open

jambayk wants to merge 2 commits into main from jambayk/mlp-unpacked-lora

+4 −4

jambayk commented Jan 30, 2025 •

edited

Loading

The check was done on the mlp module instead of the gate_up_linear module so it always returned False.
Need to access lora_B from gate_up_linear.
Code referred to k_proj and v_proj


          check gate_up_linear

cac2f48

apsonawane previously approved these changes

View reviewed changes


          fix unpack logic

ba0f867

jambayk dismissed apsonawane’s stale review via

ba0f867

January 30, 2025 23:54

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet