🎯 [SFT] add token accuracy metric

[kashif]

What does this PR do?

Adds a token_accuracy metric for training and evaluation in the SFTTrainer

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a GitHub issue? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[qgallouedec]

Dors it work with multi gpu?

[kashif]

i moved it to gather_for_metrics

[kashif]

@lewtun it is now logging to the terminal nicely too:

{'loss': 14.3862, 'grad_norm': 40.22147750854492, 'learning_rate': 1.995675675675676e-05, 'mean_token_accuracy': 0.5978433528836755, 'epoch': 0.0}
{'eval_loss': 1.7303236722946167, 'eval_runtime': 93.3417, 'eval_samples_per_second': 2.046, 'eval_steps_per_second': 0.257, 'eval_mean_token_accuracy': 0.6009875541125541, 'epoch': 0.0}            
{'loss': 13.1112, 'grad_norm': 28.22303581237793, 'learning_rate': 1.9913513513513515e-05, 'mean_token_accuracy': 0.6168438416422287, 'epoch': 0.0}                                                

[qgallouedec]

I thought this syntax was introduced in python3.10

[kashif]

checking

[qgallouedec]

it shines ✨