Skip to content

Commit

Permalink
recognize group_size ==0/None as channelwise quant
Browse files Browse the repository at this point in the history
  • Loading branch information
Michael Gschwind committed Apr 9, 2024
1 parent 99c1116 commit 9bdfe0a
Showing 1 changed file with 2 additions and 0 deletions.
2 changes: 2 additions & 0 deletions quantize.py
Original file line number Diff line number Diff line change
Expand Up @@ -583,6 +583,8 @@ def create_quantized_state_dict(self):
if isinstance(mod, torch.nn.Linear):
assert not mod.bias
in_features = mod.in_features
if group_size is None or group_size == 0:
group_size = in_features
# print("in features:", in_features, " out features:", out_features)
# assert out_features % 8 == 0, "require out_features % 8 == 0"
# print(f"linear: {fqn}, in={in_features}, out={out_features}")
Expand Down

0 comments on commit 9bdfe0a

Please sign in to comment.