Fix Trainer and Args to mention AdamW, not Adam.

[gchhablani]

This PR fixed the issue with Docs and labels in Trainer and TrainingArguments Class for AdamW, current version mentions adam in several places.

Fixes #9628

The Trainer class in trainer.py uses AdamW as the default optimizer. The TrainingArguments class mentions it as Adam in the documentation, which was confusing.

I have also changed variable names to adamw_beta1, adamw_beta2, adamw_epsilon in trainer.py.

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

@LysandreJik

[LysandreJik]

Thanks for opening the PR. As it stands this would break every existing script leveraging the parameters defined, so renaming the parameters is probably not the way to go.

@sgugger, your insight on this would be very welcome.

[sgugger]

There is no reason to change all the names of the parameters indeed, and it would be a too-heavy breaking change. AdamW is not a different optimizer from Adam, it's just Adam with a different way (some might say the right way) of doing weight decay. I don't think we need to do more than a mention at the beginning of the docstring saying that all mentions of Adam are actually about AdamW, with a link to the paper.

[gchhablani]

Hi @LysandreJik @sgugger. Thanks for your comments, I'll be changing the variables back.

I apologize if this is too silly a question, but how can I run and see how the docs look on a browser after the changes?

[sgugger]

You should check this page for all the information on generating/writing the documentation :-)

[gchhablani]

I have updated it and also added that by default, the weight decay is applied to all layers except bias and LayerNorm weights while training.

[sgugger]

This looks good to me this way, thanks!

[gchhablani]

@sgugger My code passed only 3 out of 12 checks, I was unable to run CirlceCI properly. Can you point out the reasons why this happened?

[sgugger]

We are trying to get support from them to understand why, but the checks on your PR were all cancelled. I couldn't retrigger them from our interface either.

[gchhablani]

Hi @sgugger

I believe a possible reason could be that I followed transformers on CircleCI. Maybe it performs checks on my fork of transformers and expects to find some "resources" which aren't there.

I'm not sure how CircleCI works, so this is just a wild guess.