140/best n sampling

[metric-space]

Attempts to deliver work for #140

Fixes #140

Gist

Showcase usage of best-of-n sampling

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[younesbelkada]

Maybe add a few sentences to explain the motivation of the notebook and what does it showcases

Reply via ReviewNB

[younesbelkada]

Thanks a lot for this!
Can you also add few notes in the notebook and the README.md file of examples explaining the demonstration?
From my understanding this is a new rollout strategy that a user can use in replacement of the existing one. So this should be highlighted for better understanding and broader adoption

[metric-space]

@younesbelkada thank you for the review. I took the liberty of shifting the notebook to a directory within examples and have included a README.md within said directory. Whenver you can, please let me know what you think of it

[younesbelkada]

Awesome work! Thanks a lot for your contribution

[natolambert]

Sorry was a little swamped with travel and stuff last week, but I'm stoked about this @metric-space !

[metric-space]

Hey folks, I just realized that the description was edited to say this fixes #140 (implies finality to me) but I think the notebook is a precursor to actual framework code for best-of-n sampling as put here
#140 (comment)

One idea would be to wrap that logic in a helper class like BestOfNSampler, but starting with a notebook sounds like a good way to go!

Happy to put in the work to get said additions in. Just waiting on more PR reviews to see whether I'm in the clear

[metric-space]

Hey folks @natolambert and @younesbelkada, hate to poke/ping people but also don't want this development to remain dormant and go cold. Any word on this?

[lvwerra]

Looks good to me. Let's merge the notebook and think about BestOfNSampler in a separate PR?

[metric-space]

Thank you folks for this and I appreciate the response. I'll start up a draft PR and conversations on the thread (if needed) around the BestOfNSampler shortly