Fixing flaky conversational test + flag it as a pipeline test. #9837
Conversation
| @@ -52,9 +53,11 @@ def get_pipeline(self): | |||
| # Force model output to be L | |||
There was a problem hiding this comment.
what does "Force model output to be L" mean?
There was a problem hiding this comment.
The model is a random one that I generate on the fly.
Doing that enables me to actually make the output of that model consistent (because the logits are always 0, 0, 0, 1, 0, 0, ....)
I could always do what I did with the tokenizer which is move it to a dummy model on the hub and use AutoModel instead.
It does respect more previous testing
I used this here because makes that logic explicit in the test, so the tests gets easier to understand from a reader's perspective (is the "L" correct ? Can I change it etc..). I've seen quite a few tests in pipelines that were not detecting bugs because of hidden test logic.
I'll happily switch to AutoModel if you want to.
@LysandreJik
There was a problem hiding this comment.
I don't understand the L comment either
There was a problem hiding this comment.
Ah, it comes from the expected result:
Conversation(
None, past_user_inputs=["What's the last book you have read?"], generated_responses=["L"]
),
| @@ -52,9 +53,11 @@ def get_pipeline(self): | |||
| # Force model output to be L | |||
There was a problem hiding this comment.
I don't understand the L comment either
| @@ -52,9 +53,11 @@ def get_pipeline(self): | |||
| # Force model output to be L | |||
There was a problem hiding this comment.
Ah, it comes from the expected result:
Conversation(
None, past_user_inputs=["What's the last book you have read?"], generated_responses=["L"]
),
What does this PR do?
Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
@patrickvonplaten
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.