Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the pad_token_id error #1394

Merged
merged 4 commits into from
Mar 5, 2024
Merged

Fix the pad_token_id error #1394

merged 4 commits into from
Mar 5, 2024

Conversation

yuanwu2017
Copy link
Contributor

  1. Fix the following error:
File "/mnt/disk4/yuanwu/workspace/trl/examples/research_projects/stack_llama/scripts/rl_training.py", line 263, in <module>
  pipe_outputs = sentiment_pipe(texts, **sent_kwargs)
File "/mnt/disk4/yuanwu/workspace/transformers/src/transformers/pipelines/text_classification.py", line 156, in __call__
  result = super().__call__(*args, **kwargs)
File "/mnt/disk4/yuanwu/workspace/transformers/src/transformers/pipelines/base.py", line 1143, in __call__
  outputs = list(final_iterator)
File "/mnt/disk4/yuanwu/workspace/transformers/src/transformers/pipelines/pt_utils.py", line 124, in __next__
  item = next(self.iterator)
File "/mnt/disk4/yuanwu/workspace/transformers/src/transformers/pipelines/pt_utils.py", line 125, in __next__
  processed = self.infer(item, **self.params)
File "/mnt/disk4/yuanwu/workspace/transformers/src/transformers/pipelines/base.py", line 1068, in forward
  model_outputs = self._forward(model_inputs, **forward_params)
File "/mnt/disk4/yuanwu/workspace/transformers/src/transformers/pipelines/text_classification.py", line 187, in _forward
  return self.model(**model_inputs)
File "/mnt/disk4/yuanwu/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
  return self._call_impl(*args, **kwargs)
File "/mnt/disk4/yuanwu/env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
  return forward_call(*args, **kwargs)
File "/mnt/disk4/yuanwu/workspace/accelerate/src/accelerate/hooks.py", line 166, in new_forward
  output = module._old_forward(*args, **kwargs)
File "/mnt/disk4/yuanwu/workspace/transformers/src/transformers/models/llama/modeling_llama.py", line 1369, in forward
  raise ValueError("Cannot handle batch sizes > 1 if no padding token is defined.")
ValueError: Cannot handle batch sizes > 1 if no padding token is defined.
  1. Add the load_in_8bit arguments for rl_training script.

@younesbelkada
Copy link
Contributor

Thank you for the fix @yuanwu2017 ! can you run the styling checks? make precommit

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@yuanwu2017
Copy link
Contributor Author

Done. Thanks

Copy link
Contributor

@younesbelkada younesbelkada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your work on this!

@younesbelkada younesbelkada merged commit 4219cbf into huggingface:main Mar 5, 2024
3 of 9 checks passed
kashif pushed a commit to fe1ixxu/trl that referenced this pull request Mar 15, 2024
* Fix the pad_token_id error

Signed-off-by: yuanwu <[email protected]>

* Add the load_in_8bit argument in rl_training.py

Signed-off-by: yuanwu <[email protected]>

* Reformate the patch

Signed-off-by: yuanwu <[email protected]>

* Fix the check failed

Signed-off-by: yuanwu <[email protected]>

---------

Signed-off-by: yuanwu <[email protected]>
lapp0 pushed a commit to lapp0/trl that referenced this pull request May 10, 2024
* Fix the pad_token_id error

Signed-off-by: yuanwu <[email protected]>

* Add the load_in_8bit argument in rl_training.py

Signed-off-by: yuanwu <[email protected]>

* Reformate the patch

Signed-off-by: yuanwu <[email protected]>

* Fix the check failed

Signed-off-by: yuanwu <[email protected]>

---------

Signed-off-by: yuanwu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants