Skip to content

Commit

Permalink
Fix assertion failure during chat session. (#1061)
Browse files Browse the repository at this point in the history
This fixes the following assert that is easy to repro in any chat session:

```
Traceback (most recent call last):
    File "/home/ubuntu/cali/torchchat/torchchat.py", line 69, in <module>
        generate_main(args)
    File "/home/ubuntu/cali/torchchat/generate.py", line 896, in main
        for _ in gen.chat(generator_args):
    File "/home/ubuntu/cali/torchchat/generate.py", line 748, in chat
        self.chat_formatter.encode_header(
    File "/home/ubuntu/cali/torchchat/generate.py", line 53, in encode_header
        tokens.extend(self.tokenizer.encode(role, bos=False, eos=False))
    File "/home/ubuntu/cali/torchchat/tokenizer/tiktoken.py", line 133, in encode
        assert type(s) is str
```

I believe this regressed with #1035.
  • Loading branch information
prideout authored Aug 25, 2024
1 parent 7465842 commit 2f4ba2d
Showing 1 changed file with 1 addition and 3 deletions.
4 changes: 1 addition & 3 deletions generate.py
Original file line number Diff line number Diff line change
Expand Up @@ -745,9 +745,7 @@ def chat(
{"role": "user", "content": prompt}
)
encoded.extend(
self.chat_formatter.encode_header(
{"role": "assistant", "content": ""}
)
self.chat_formatter.encode_header("assistant")
)
encoded = torch.tensor(
encoded, dtype=torch.int, device=self.builder_args.device
Expand Down

0 comments on commit 2f4ba2d

Please sign in to comment.