Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

refactor: In PromptNode reuse tokenizer instead of loading new one for stop words #4016

Merged
merged 2 commits into from
Feb 1, 2023
Merged

refactor: In PromptNode reuse tokenizer instead of loading new one for stop words #4016

merged 2 commits into from
Feb 1, 2023

Conversation

sjrl
Copy link
Contributor

@sjrl sjrl commented Jan 31, 2023 •
edited
Loading

Related Issues

Proposed Changes:

In PromptNode reuse tokenizer instead of loading a new one for stop words when using HF models.

How did you test it?

Existing tests

Notes for the reviewer

Checklist

  • I have read the contributors guidelines and the code of conduct
  • I have updated the related issue with new insights and changes
  • I added tests that demonstrate the correct behavior of the change
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.
  • I documented my code
  • I ran pre-commit hooks and fixed any issue

@sjrl sjrl requested a review from a team as a code owner January 31, 2023 14:16
@sjrl sjrl requested review from julian-risch and removed request for a team January 31, 2023 14:16
@sjrl sjrl added the type:refactor Not necessarily visible to the users label Jan 31, 2023
@sjrl sjrl mentioned this pull request Jan 31, 2023
Closed
julian-risch
julian-risch approved these changes Feb 1, 2023
View reviewed changes
Copy link
Member

@julian-risch julian-risch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good to me! 👍 Really nice to have this improvement. Should speed things up a little bit and also reduce memory footprint. Great job!

@sjrl sjrl merged commit 7b3d7ee into main Feb 1, 2023
@sjrl sjrl deleted the prompt_node/stop_word_update branch February 1, 2023 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@julian-risch julian-risch julian-risch approved these changes

Assignees
No one assigned
Labels
topic:LLM type:refactor Not necessarily visible to the users
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

refactor: In PromptNode reuse tokenizer instead of loading new one for stop words
3 participants
@sjrl @julian-risch @vblagoje