-
Notifications
You must be signed in to change notification settings - Fork 614
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rephrase documentation of adabelief #2704
Conversation
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). For more information, open the CLA check for this pull request. |
You are owner of some files modified in this pull request. |
@@ -113,7 +113,7 @@ def __init__( | |||
rectify: boolean. Whether to apply learning rate rectification as | |||
from RAdam. | |||
total_steps: An integer. Total number of training steps. Enable | |||
warmup by setting a positive value. | |||
warmup by setting a value greater than zero. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think that it was correct:
http://oeis.org/A000027
http://oeis.org/A001477
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice. However, "greater than zero" I think requires less expertise to be understood. I thought zero was a positive number for example, and I have a PhD.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There could be some confusion with signed zero but on integer I think it is hard to be confused. I will merge this as it is just an alternative.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks
Looks great to me. The same issue is with
|
This PR fixes the documentation of
total_steps
, which enables the warmup only if the value is bigger than zero. However, the documentation says "positive value". Zero (the default value) is positive though