Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Backbone base class #621

Merged
merged 11 commits into from
Jan 5, 2023
Merged

Create Backbone base class #621

merged 11 commits into from
Jan 5, 2023

Conversation

jbischof
Copy link
Contributor

@jbischof jbischof commented Dec 28, 2022

Pilot for how base classes could improve the maintainability of our library. The Backbone class can be easily extended to other model types and Task, Preprocessor, and Tokenizer base classes can also be added.

I've also simplified the from_preset docstring so we don't have to write a lot of custom copy for each model class. We already have all the information we need in the class docstring.

Part of #530

@jbischof jbischof requested a review from mattdangerw December 28, 2022 23:53
Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! This will be a nice change. Left some comments.

keras_nlp/models/base.py Outdated Show resolved Hide resolved
keras_nlp/models/base.py Outdated Show resolved Hide resolved
keras_nlp/models/base.py Outdated Show resolved Hide resolved
keras_nlp/models/bert/bert_backbone.py Outdated Show resolved Hide resolved
keras_nlp/models/bert/bert_backbone.py Show resolved Hide resolved
Copy link
Member

@mattdangerw mattdangerw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

@mattdangerw mattdangerw mentioned this pull request Jan 5, 2023
@jbischof
Copy link
Contributor Author

jbischof commented Jan 5, 2023

New from_preset docstring example below. Now the list of presets has quotes 😄

"""Instantiate BertBackbone model from preset architecture and weights.

        Args:
            preset: string. Must be one of "bert_tiny_en_uncased", "bert_small_en_uncased", "bert_medium_en_uncased", "bert_base_en_uncased", "bert_base_en", "bert_base_zh", "bert_base_multi", "bert_large_en_uncased", "bert_large_en".
            load_weights: Whether to load pre-trained weights into model.
                Defaults to `True`.

        Examples:
        ```python
        # Load architecture and weights from preset
        model = BertBackbone.from_preset("bert_base_en_uncased")

        # Load randomly initialized model from preset architecture
        model = BertBackbone.from_preset(
            "bert_base_en_uncased",
            load_weights=False
        )
        ```
"""

@jbischof jbischof marked this pull request as ready for review January 5, 2023 00:47
@jbischof jbischof merged commit eb76480 into keras-team:master Jan 5, 2023
@jbischof jbischof deleted the base_class branch January 5, 2023 01:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants