Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

don't apply dropout to dense input embeddings #5696

Merged
merged 2 commits into from
Apr 22, 2020
Merged

don't apply dropout to dense input embeddings #5696

merged 2 commits into from
Apr 22, 2020

Conversation

Ghostvv
Copy link
Contributor

@Ghostvv Ghostvv commented Apr 22, 2020 •
edited
Loading

Proposed changes:

  • experiments show, that dropping dense embedding input is not a good idea, however dropping dense representation of sparse input is fine, since there is a trainable layer in between
  • fix for Input dropout #5636

Status (please check what you already did):

  • added some tests for the functionality
  • updated the documentation
  • updated the changelog (please check changelog for instructions)
  • reformat files using black (please check Readme for instructions)

@Ghostvv Ghostvv requested review from tabergma and dakshvar22 April 22, 2020 11:05
@Ghostvv
Copy link
Contributor Author

Ghostvv commented Apr 22, 2020

I don't think there is a need for changelog, since original changelog for input-dropout PR covers it correctly

tabergma
tabergma approved these changes Apr 22, 2020
View reviewed changes
Copy link
Contributor

@tabergma tabergma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@Ghostvv Ghostvv merged commit c3bbc31 into master Apr 22, 2020
@Ghostvv Ghostvv deleted the fix-input-dropout branch April 22, 2020 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tabergma tabergma tabergma approved these changes

@dakshvar22 dakshvar22 Awaiting requested review from dakshvar22

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@Ghostvv @tabergma