Concat Threshold Update #1016
Closed
Concat Threshold Update #1016
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| @@ -0,0 +1,59 @@ | |||
| # Copyright 2022 Sony Semiconductor Israel, Inc. All rights reserved. | |||
| # | |||
There was a problem hiding this comment.
Please fix the license year.
|
|
||
| MATCHER = NodeOperationMatcher(Concatenate) | ||
|
|
||
| class threshold_updater(common.BaseSubstitution): |
There was a problem hiding this comment.
Please rename the class to include 'concat' or something like that (for example: ConcatThresholdUpdate)
| from model_compression_toolkit.core.common.graph.graph_matchers import NodeOperationMatcher | ||
| from model_compression_toolkit.constants import THRESHOLD | ||
|
|
||
| MATCHER = NodeOperationMatcher(Concatenate) |
There was a problem hiding this comment.
What about tf.concat?
| for prev_node in prev_nodes: | ||
| if len(graph.get_next_nodes(prev_node))==1: | ||
| prev_node.candidates_quantization_cfg[0].activation_quantization_cfg.activation_quantization_params[THRESHOLD] = concat_threshold | ||
|
|
There was a problem hiding this comment.
We also need to deal with mixed-precision case since the threshold can be different for each cfg in candidates_quantization_cfg.
There was a problem hiding this comment.
several important things:
- we need to check that both the concat node and the previous node have a 'threshold' - this will ensure only symmetric or power of 2 quantization (and not uniform)
- we need to check that we have only a single candidate cfg - since we want to avoid the substitution in case of mixed precision.
- maybe we can add a "todo" comment that it can be expanded to mixed precision and to uniform quantizer in the future.
There was a problem hiding this comment.
changed code to leave the case of MP. will add a fix if results for the method are good for ptq.
...ompression_toolkit/core/pytorch/graph_substitutions/substitutions/concat_threshold_update.py
Show resolved
Hide resolved
model_compression_toolkit/core/common/quantization/quantization_config.py
Show resolved
Hide resolved
tests/pytorch_tests/model_tests/feature_models/concat_threshold_test.py
Outdated
Show resolved
Hide resolved
tests/keras_tests/feature_networks_tests/feature_networks/concatination_threshold_update.py
Outdated
Show resolved
Hide resolved
|
made the relevant changes. let me know if i have filled in all the correct documentation. |
Idan-BenAmi
reviewed
Apr 3, 2024
| for prev_node in prev_nodes: | ||
| if len(graph.get_next_nodes(prev_node))==1: | ||
| prev_node.candidates_quantization_cfg[0].activation_quantization_cfg.activation_quantization_params[THRESHOLD] = concat_threshold | ||
|
|
There was a problem hiding this comment.
several important things:
- we need to check that both the concat node and the previous node have a 'threshold' - this will ensure only symmetric or power of 2 quantization (and not uniform)
- we need to check that we have only a single candidate cfg - since we want to avoid the substitution in case of mixed precision.
- maybe we can add a "todo" comment that it can be expanded to mixed precision and to uniform quantizer in the future.
..._compression_toolkit/core/keras/graph_substitutions/substitutions/concat_threshold_update.py
Outdated
Show resolved
Hide resolved
..._compression_toolkit/core/keras/graph_substitutions/substitutions/concat_threshold_update.py
Outdated
Show resolved
Hide resolved
...ompression_toolkit/core/pytorch/graph_substitutions/substitutions/concat_threshold_update.py
Outdated
Show resolved
Hide resolved
...ompression_toolkit/core/pytorch/graph_substitutions/substitutions/concat_threshold_update.py
Outdated
Show resolved
Hide resolved
...ompression_toolkit/core/pytorch/graph_substitutions/substitutions/concat_threshold_update.py
Outdated
Show resolved
Hide resolved
addition of zscore mct tutorial including updates to tutorial utils
…#1022) Co-authored-by: Ofir Gordon <[email protected]>
* Update docs and version number
--------- Co-authored-by: Ofir Gordon <[email protected]>
Use TF 2.15 for test_suit_pip_keras.yml
* Yolov8n tutorial - add gptq option for optimized for imx500
…ny#1000) * Update nanodet-keras-model in mct_model_garden to support configurable number of classes
…izers fro all modules in model (instaed of childrens) (sony#1028)
* Increase coverage threshold for test success to 96% * Include only model_compression_toolkit files in the report. * Archive an HTML coverage report. * Remove unused files like model_compression_toolkit/core/exporter.py and model_compression_toolkit/core/keras/graph_substitutions/substitutions/remove_relu_upper_bound.py. --------- Co-authored-by: reuvenp <[email protected]>
…vment to bounding box decoding (sony#1032)
* Yolov8 Object Detection PyTorch tutorial * Add Yolov8 Object Detection to the PyTorch MCT Model Garden * The PyTorch implementation of Yolov8n object detection model, following https://github.com/ultralytics/ultralytics. This implementation includes a slightly modified version of the yolov8 detection-head optimized for model quantization.
This commit fixes these issues: * The float graph used for comparison in the similarity analyzer underwent stats correction. * The number of samples to display did not work as expected, and a single sample was always used. * An error was thrown when no comparison points were found. The fixes are done by: * Changing the analyzer_model_quantization API to receive both float and quantized graphs. * Add a method to the visualizer to query the existence of compare points and skip the analyzer with a warning in case no points were found. * Change the computation of the number of visualized samples. --------- Co-authored-by: reuvenp <[email protected]>
|
closing pr due to commit issues re opoened as pr 1036 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description:
feature to force the alignment of thresholds pre and post concatenation layer. implemented for both torch and keras. Including testing for each.
update does not occur when previous layer has multiple outputs.
Checklist before requesting a review: