Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Test Fix] Add Quantization then finetune tests #964

Merged
merged 27 commits into from
Jan 23, 2025
Merged

Conversation

horheynm
Copy link
Collaborator

@horheynm horheynm commented Dec 9, 2024

Contingent on merge of huggingface/transformers#34719
^ has been merged not yet released

SUMMARY:
Add test to

  • Given a model, oneshot quantize, then run ptq - training.
    Model must be run_compressed = False to run

Note:

Also fix a bug where in log_sparsification, the layer name is not being recognized so fails. Here nothting is being sparsified, so num params is set to zero

TEST PLAN:
ran the test using transformers main
must pass tests/llmcompressor/transformers/finetune/test_oneshot_then_finetune.py

Copy link

github-actions bot commented Dec 9, 2024

👋 Hi! Thank you for contributing to llm-compressor. Please add the ready label when the PR is ready for review.

@horheynm
Copy link
Collaborator Author

/ready

@dsikka dsikka marked this pull request as draft December 12, 2024 17:01
@horheynm horheynm changed the title Add Quantization then finetune tests [Test Fix] Add Quantization then finetune tests Dec 16, 2024
@horheynm horheynm marked this pull request as ready for review December 23, 2024 14:12
@horheynm horheynm marked this pull request as draft December 23, 2024 15:03
@horheynm horheynm marked this pull request as ready for review January 10, 2025 13:43
kylesayrs
kylesayrs previously approved these changes Jan 10, 2025
kylesayrs
kylesayrs previously approved these changes Jan 10, 2025
Copy link
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

As per offline conversation, lets decompress before running finetune
and verify if we still need the skipif

dsikka
dsikka previously approved these changes Jan 14, 2025
Copy link
Collaborator

@dsikka dsikka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. please fix quality

rahul-tuli
rahul-tuli previously approved these changes Jan 20, 2025
Copy link
Collaborator

@rahul-tuli rahul-tuli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator

@kylesayrs kylesayrs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need some clarification on params_quantized, otherwise LGTM!

else 0
num_params = 0
for name, layer in get_quantized_layers(self.module):
num_param_weight = torch.numel(
Copy link
Collaborator

@kylesayrs kylesayrs Jan 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if getattr(layer, "weight", None) is not None:
    num_params += torch.numel(layer.weight)
if getattr(layer, "bias", None) is not None:
    num_params += torch.numel(layer.bias)

@dsikka dsikka added the ready When a PR is ready for review label Jan 22, 2025
@dsikka dsikka merged commit b105c55 into main Jan 23, 2025
5 of 7 checks passed
@dsikka dsikka deleted the quant-then-finetune branch January 23, 2025 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready When a PR is ready for review
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants