Skip to content

Commit

Permalink
Fix assertion failure during chat session.
Browse files Browse the repository at this point in the history
This fixes the following assert that is easy to repro in any chat session:

```
Traceback (most recent call last):
    File "/home/ubuntu/cali/torchchat/torchchat.py", line 69, in <module>
        generate_main(args)
    File "/home/ubuntu/cali/torchchat/generate.py", line 896, in main
        for _ in gen.chat(generator_args):
    File "/home/ubuntu/cali/torchchat/generate.py", line 748, in chat
        self.chat_formatter.encode_header(
    File "/home/ubuntu/cali/torchchat/generate.py", line 53, in encode_header
        tokens.extend(self.tokenizer.encode(role, bos=False, eos=False))
    File "/home/ubuntu/cali/torchchat/tokenizer/tiktoken.py", line 133, in encode
        assert type(s) is str
```

I believe this regressed with #1035.
  • Loading branch information
prideout authored Aug 25, 2024
1 parent 7465842 commit 1d52f1b
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions generate.py
Original file line number Diff line number Diff line change
Expand Up @@ -745,9 +745,7 @@ def chat(
{"role": "user", "content": prompt}
)
encoded.extend(
self.chat_formatter.encode_header(
{"role": "assistant", "content": ""}
)
self.chat_formatter.encode_header("assistant")
)
encoded = torch.tensor(
encoded, dtype=torch.int, device=self.builder_args.device
Expand Down

0 comments on commit 1d52f1b

Please sign in to comment.