-
Notifications
You must be signed in to change notification settings - Fork 84
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Test Fix] Add Quantization then finetune tests #964
Conversation
👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review. |
/ready |
tests/llmcompressor/transformers/finetune/test_oneshot_then_finetune.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
As per offline conversation, lets decompress before running finetune
and verify if we still need the skipif
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. please fix quality
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need some clarification on params_quantized
, otherwise LGTM!
else 0 | ||
num_params = 0 | ||
for name, layer in get_quantized_layers(self.module): | ||
num_param_weight = torch.numel( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if getattr(layer, "weight", None) is not None:
num_params += torch.numel(layer.weight)
if getattr(layer, "bias", None) is not None:
num_params += torch.numel(layer.bias)
Contingent on merge of huggingface/transformers#34719^ has been merged not yet released
SUMMARY:
Add test to
Model must be run_compressed = False to run
Note:
CompressedTensorsConfig
. See https://github.com/vllm-project/llm-compressor/pull/964/files#diff-e480ed475c0a5b2beb4052c1dd2aca671999634ace41a5ea017fdff1ce68be0bR130-R135Also fix a bug where in log_sparsification, the layer name is not being recognized so fails. Here nothting is being sparsified, so num params is set to zero
TEST PLAN:
ran the test using transformers main
must pass tests/llmcompressor/transformers/finetune/test_oneshot_then_finetune.py