Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an XLMRobertaMaskedLM task model #720

Closed
4 tasks
mattdangerw opened this issue Feb 3, 2023 · 8 comments · Fixed by #950
Closed
4 tasks

Add an XLMRobertaMaskedLM task model #720

mattdangerw opened this issue Feb 3, 2023 · 8 comments · Fixed by #950
Assignees
Labels
stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues type:feature New feature or request

Comments

@mattdangerw
Copy link
Member

mattdangerw commented Feb 3, 2023

In #653 we added a masked language modeling task for RoBERTa. We can make a similar change for the XLMRoberta model.

  • Update XLMRobertaTokenizer to expect a mask token.
  • Add a XLMRobertaMaskedLMPreprocessor preprocessor layer and tests.
  • Add a XLMRobertaMaskedLM task model and tests.
  • Update keras_nlp/models/__init__.py to export XLMRobertaMaskedLM and XLMRobertaMaskedLMPreprocessor.
@mattdangerw mattdangerw added type:Bug Something isn't working type:feature New feature or request stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues and removed type:Bug Something isn't working labels Feb 3, 2023
@soma2000-lang
Copy link
Contributor

I would love to work on this issue!

@soma2000-lang
Copy link
Contributor

/assign

@shivance
Copy link
Collaborator

Hi @soma2000-lang are you still working on it? if not then I would take this one up !

@soma2000-lang
Copy link
Contributor

@shivance yes ,I had also opened a pr for it,but had to close it for some reason.Will reopen another one.

@jayam30
Copy link

jayam30 commented Feb 23, 2023

Hey!
I would like to take this issue up, if others are not @soma2000-lang

@soma2000-lang
Copy link
Contributor

@jayam30 already opened a pr

@shivance
Copy link
Collaborator

Hi @mattdangerw , it's already been 1month+ since this issue has gone stale. Please assign me this issue, I'm already having a crack at it to see how challenging it is. So far the tests pass in local.

If you allow, I could open a PR. Thanks !

@soma2000-lang
Copy link
Contributor

@shivance I am working on it but the tests are failing currently ,working on that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stat:contributions welcome Add this label to feature request issues so they are separated out from bug reporting issues type:feature New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants