Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump transformers version #3272

Merged
merged 10 commits into from
Feb 21, 2025
Merged

Bump transformers version #3272

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
5fe5106
Bump transformers version
nikita-malininn Feb 12, 2025
0f5f4ab
Bump optimum-intel version as dependency
nikita-malininn Feb 12, 2025
bb888e7
Bump optimum as dependency
nikita-malininn Feb 12, 2025
6330134
Update statistics caching test
nikita-malininn Feb 12, 2025
8f346f2
Bump accelerate
nikita-malininn Feb 12, 2025
29ef0be
More test fixes
nikita-malininn Feb 12, 2025
bedf993
Update version
nikita-malininn Feb 18, 2025
261182d
Update references
nikita-malininn Feb 18, 2025
75fa0e0
WA for failed tests
ljaljushkin Feb 19, 2025
dc54587
Merge pull request #7 from ljaljushkin/nm/bump_transformers
nikita-malininn Feb 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions examples/llm_compression/openvino/smollm2_360m_fp8/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
datasets
openvino==2024.6
optimum-intel[openvino]
transformers
optimum-intel[openvino]>=1.22.0
transformers>=4.48.0
onnx==1.17.0
4 changes: 2 additions & 2 deletions examples/llm_compression/openvino/tiny_llama/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
transformers
transformers>=4.48.0
datasets==2.14.7
openvino==2025.0
optimum-intel[openvino]
optimum-intel[openvino]>=1.22.0
onnx==1.17.0
4 changes: 2 additions & 2 deletions examples/llm_compression/openvino/tiny_llama_find_hyperparams/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ datasets
whowhatbench @ git+https://github.com/openvinotoolkit/openvino.genai#subdirectory=tools/who_what_benchmark
numpy>=1.23.5,<2
openvino==2025.0
optimum-intel>=1.13.0
transformers>=4.35.2
optimum-intel>=1.22.0
transformers>=4.48.0
onnx==1.17.0
4 changes: 2 additions & 2 deletions examples/llm_compression/openvino/tiny_llama_synthetic_data/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,6 @@ torch==2.5.1
datasets==3.0.1
numpy>=1.23.5,<2
openvino==2025.0
optimum-intel>=1.13.0
transformers>=4.35.2
optimum-intel>=1.22.0
transformers>=4.48.0
onnx==1.17.0
2 changes: 1 addition & 1 deletion nncf/torch/exporter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ class PTExporter(Exporter):
This class provides export of the compressed model to the ONNX format.
"""

_ONNX_DEFAULT_OPSET = 13
_ONNX_DEFAULT_OPSET = 14
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AlexanderDokuchaev do we have some tests for torch export outside pre-commit scope?

Copy link
Contributor

@ljaljushkin ljaljushkin Feb 20, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nikita-malininn I found PTQ conformance tests with export:
ImageClassificationTimm and ImageClassificationTorchvision test cases in https://github.com/openvinotoolkit/nncf/blob/develop/tests/post_training/model_scope.py
IMO, It's worth launching it


@staticmethod
def parse_format(save_format: str) -> Tuple[str, dict]:
Expand Down
6 changes: 6 additions & 0 deletions tests/openvino/native/quantization/test_weights_compression_statistics_caching.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
from typing import Tuple

import datasets
import numpy as np
import openvino as ov
from optimum.intel.openvino import OVModelForCausalLM
from transformers import AutoTokenizer
Expand All @@ -37,6 +38,11 @@ def transform_fn(data, model=model, tokenizer=tokenizer):
input_ids = tokenized_text["input_ids"]
inputs = {"input_ids": input_ids, "attention_mask": tokenized_text["attention_mask"]}

if "position_ids" in model.input_names:
position_ids = np.cumsum(inputs["attention_mask"], axis=1) - 1
position_ids[inputs["attention_mask"] == 0] = 1
inputs["position_ids"] = position_ids

batch_size = input_ids.shape[0]
if hasattr(model, "key_value_input_names"):
for input_name in model.key_value_input_names:
Expand Down
6 changes: 3 additions & 3 deletions tests/openvino/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@ addict>=2.4.0
timm==0.9.2
efficientnet_pytorch==0.7.1
datasets==3.0.1
transformers==4.45.2
optimum-intel==1.20.0
optimum==1.23.1
transformers==4.48.3
optimum-intel==1.22.0
optimum==1.24.0
6 changes: 3 additions & 3 deletions tests/post_training/requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,13 @@ pytest-split

librosa==0.10.0
memory-profiler==0.61.0
optimum-intel==1.21.0
optimum==1.23.3
optimum-intel==1.22.0
optimum==1.24.0
scikit-learn>=1.2.2,<=1.5.0
soundfile==0.12.1
tensorboard==2.13.0
tensorflow-io==0.32.0
timm==0.9.2
transformers==4.46.3
transformers==4.48.3
whowhatbench @ git+https://github.com/openvinotoolkit/[email protected]#subdirectory=tools/who_what_benchmark
datasets==3.1.0
968 changes: 491 additions & 477 deletions ...ata/experimental/sparsify_activations/dummy_llama_int8_sym_weights_sparse_activations.dot
View file
Open in desktop

Large diffs are not rendered by default.

848 changes: 431 additions & 417 deletions tests/torch/data/experimental/sparsify_activations/dummy_llama_sparse_activations.dot
View file
Open in desktop

Large diffs are not rendered by default.

69 changes: 27 additions & 42 deletions tests/torch/data/reference_graphs/pruning_groups/1_layer_BERT.dot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,61 +2,46 @@ strict digraph {
0 [color=grey, label="Block: S:1__O:0
Producers: 10
Consumers: ", style=filled];
1 [color=grey, label="Block: S:64__O:0
1 [color=red, label="Block: S:64__O:0
Producers: 10
Consumers: ", style=filled];
2 [color=grey, label="Block: S:64__O:0
Producers: 10,11
Consumers: ", style=filled];
3 [color=green, label="Block: S:64__O:0
Producers: 10,11,14
Consumers: 29", style=filled];
4 [color=grey, label="Block: S:1__O:64
2 [color=red, label="Block: S:1__O:64
Producers: 10
Consumers: ", style=filled];
5 [color=green, label="Block: S:1__O:64
Producers: 10,11
Consumers: 20", style=filled];
6 [color=grey, label="Block: S:1__O:0
Producers: 11
3 [color=grey, label="Block: S:1__O:0
Producers: 13
Consumers: ", style=filled];
4 [color=red, label="Block: S:64__O:0
Producers: 13
Consumers: ", style=filled];
7 [color=grey, label="Block: S:64__O:0
Producers: 11
5 [color=red, label="Block: S:1__O:64
Producers: 13
Consumers: ", style=filled];
8 [color=grey, label="Block: S:1__O:64
Producers: 11
6 [color=grey, label="Block: S:1__O:0
Producers: 16
Consumers: ", style=filled];
9 [color=grey, label="Block: S:1__O:0
Producers: 14
7 [color=red, label="Block: S:64__O:0
Producers: 16
Consumers: ", style=filled];
10 [color=grey, label="Block: S:64__O:0
Producers: 14
8 [color=red, label="Block: S:1__O:64
Producers: 16
Consumers: ", style=filled];
11 [color=green, label="Block: S:1__O:64
Producers: 14
Consumers: 29", style=filled];
12 [color=red, label="Block: S:1__O:0
Producers: 29
9 [color=red, label="Block: S:1__O:0
Producers: 22
Consumers: ", style=filled];
13 [color=green, label="Block: S:1__O:0
Producers: 33
Consumers: 35", style=filled];
14 [color=red, label="Block: S:1__O:0
Producers: 35
10 [color=green, label="Block: S:1__O:0
Producers: 26
Consumers: 28", style=filled];
11 [color=red, label="Block: S:1__O:0
Producers: 28
Consumers: ", style=filled];
15 [color=red, label="Block: S:1__O:0
Producers: 39
12 [color=red, label="Block: S:1__O:0
Producers: 32
Consumers: ", style=filled];
0 -> 1;
0 -> 4;
1 -> 2;
2 -> 3;
4 -> 5;
0 -> 2;
3 -> 4;
3 -> 5;
6 -> 7;
6 -> 8;
7 -> 2;
8 -> 5;
9 -> 10;
9 -> 11;
10 -> 3;
}
65 changes: 25 additions & 40 deletions tests/torch/data/reference_graphs/pruning_groups/CLIP.dot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,62 +2,47 @@ strict digraph {
0 [color=grey, label="Block: S:1__O:0
Producers: 10
Consumers: ", style=filled];
1 [color=grey, label="Block: S:2__O:0
1 [color=red, label="Block: S:2__O:0
Producers: 10
Consumers: ", style=filled];
2 [color=grey, label="Block: S:2__O:0
Producers: 10,12
Consumers: ", style=filled];
3 [color=green, label="Block: S:2__O:0
Producers: 10,12,16
Consumers: 34", style=filled];
4 [color=grey, label="Block: S:1__O:2
2 [color=red, label="Block: S:1__O:2
Producers: 10
Consumers: ", style=filled];
5 [color=green, label="Block: S:1__O:2
Producers: 10,12
Consumers: 27", style=filled];
3 [color=grey, label="Block: S:1__O:0
Producers: 11
Consumers: ", style=filled];
4 [color=red, label="Block: S:2__O:0
Producers: 11
Consumers: ", style=filled];
5 [color=red, label="Block: S:1__O:2
Producers: 11
Consumers: ", style=filled];
6 [color=grey, label="Block: S:1__O:0
Producers: 12
Consumers: ", style=filled];
7 [color=grey, label="Block: S:2__O:0
7 [color=red, label="Block: S:2__O:0
Producers: 12
Consumers: ", style=filled];
8 [color=grey, label="Block: S:1__O:2
8 [color=red, label="Block: S:1__O:2
Producers: 12
Consumers: ", style=filled];
9 [color=grey, label="Block: S:1__O:0
Producers: 16
9 [color=red, label="Block: S:1__O:0
Producers: 22
Consumers: ", style=filled];
10 [color=grey, label="Block: S:2__O:0
Producers: 16
10 [color=grey, label="Block: S:1__O:0
Producers: 25
Consumers: ", style=filled];
11 [color=green, label="Block: S:1__O:2
Producers: 16
Consumers: 34", style=filled];
11 [color=green, label="Block: S:1__O:0
Producers: 25
Consumers: 29", style=filled];
12 [color=red, label="Block: S:1__O:0
Producers: 34
Consumers: ", style=filled];
13 [color=grey, label="Block: S:1__O:0
Producers: 37
Consumers: ", style=filled];
14 [color=green, label="Block: S:1__O:0
Producers: 37
Consumers: 41", style=filled];
15 [color=red, label="Block: S:1__O:0
Producers: 41
Producers: 29
Consumers: ", style=filled];
0 -> 1;
0 -> 4;
1 -> 2;
2 -> 3;
4 -> 5;
0 -> 2;
3 -> 4;
3 -> 5;
6 -> 7;
6 -> 8;
7 -> 2;
8 -> 5;
9 -> 10;
9 -> 11;
10 -> 3;
13 -> 14;
10 -> 11;
}
73 changes: 29 additions & 44 deletions tests/torch/data/reference_graphs/pruning_groups/DistilBERT.dot
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,62 +1,47 @@
strict digraph {
0 [color=grey, label="Block: S:1__O:0
Producers: 7
Producers: 8
Consumers: ", style=filled];
1 [color=grey, label="Block: S:2__O:0
Producers: 7
1 [color=red, label="Block: S:2__O:0
Producers: 8
Consumers: ", style=filled];
2 [color=grey, label="Block: S:2__O:0
Producers: 7,10
2 [color=red, label="Block: S:1__O:2
Producers: 8
Consumers: ", style=filled];
3 [color=green, label="Block: S:2__O:0
Producers: 7,10,13
Consumers: 29", style=filled];
4 [color=grey, label="Block: S:1__O:2
Producers: 7
3 [color=grey, label="Block: S:1__O:0
Producers: 11
Consumers: ", style=filled];
5 [color=green, label="Block: S:1__O:2
Producers: 7,10
Consumers: 18", style=filled];
6 [color=grey, label="Block: S:1__O:0
Producers: 10
4 [color=red, label="Block: S:2__O:0
Producers: 11
Consumers: ", style=filled];
7 [color=grey, label="Block: S:2__O:0
Producers: 10
5 [color=red, label="Block: S:1__O:2
Producers: 11
Consumers: ", style=filled];
8 [color=grey, label="Block: S:1__O:2
Producers: 10
6 [color=grey, label="Block: S:1__O:0
Producers: 14
Consumers: ", style=filled];
9 [color=grey, label="Block: S:1__O:0
Producers: 13
7 [color=red, label="Block: S:2__O:0
Producers: 14
Consumers: ", style=filled];
10 [color=grey, label="Block: S:2__O:0
Producers: 13
8 [color=red, label="Block: S:1__O:2
Producers: 14
Consumers: ", style=filled];
11 [color=green, label="Block: S:1__O:2
Producers: 13
Consumers: 29", style=filled];
12 [color=red, label="Block: S:1__O:0
Producers: 29
9 [color=red, label="Block: S:1__O:0
Producers: 21
Consumers: ", style=filled];
13 [color=green, label="Block: S:1__O:0
Producers: 32
Consumers: 34", style=filled];
14 [color=red, label="Block: S:1__O:0
Producers: 34
10 [color=green, label="Block: S:1__O:0
Producers: 24
Consumers: 26", style=filled];
11 [color=red, label="Block: S:1__O:0
Producers: 26
Consumers: ", style=filled];
15 [color=red, label="Block: S:1__O:0
Producers: 39
12 [color=red, label="Block: S:1__O:0
Producers: 31
Consumers: ", style=filled];
0 -> 1;
0 -> 4;
1 -> 2;
2 -> 3;
4 -> 5;
0 -> 2;
3 -> 4;
3 -> 5;
6 -> 7;
6 -> 8;
7 -> 2;
8 -> 5;
9 -> 10;
9 -> 11;
10 -> 3;
}
Loading