add create_reference_model #61
Conversation
| """ | ||
|
|
||
| parameter_names = [n for n, _ in model.named_parameters()] | ||
| ref_model = deepcopy(model) |
There was a problem hiding this comment.
Do you think this can blow up the memory if manipulating very large models?
I was thinking of leveraging accelerate.init_empty_weights context manager and initialize an empty model and populating it step by step. This can be boilerplate a bit so maybe let's leave it in a follow up PR
There was a problem hiding this comment.
Sounds good - let's see how it works and maybe adapt it to init_empty_weights in a follow up :)
Co-authored-by: Younes Belkada <[email protected]>
trl/models/modeling_base.py
Outdated
| ref_model = deepcopy(model) | ||
|
|
||
| # if no layers are shared, return copy of model | ||
| if share_layers is None: |
There was a problem hiding this comment.
do we need to extend this logic to handle the case when share_layers=0 or does the indexing start with 0?
There was a problem hiding this comment.
indeed indexing starts at 0 so this should be fine
|
The documentation is not available anymore as the PR was closed or merged. |
This PR adds the
create_reference_modelfunction. It can be used to create a static reference model from an existing model:The reference model can also share layers with the original model:
In that case the first three layers are frozen for both models and the remaining layers can be updated in the active model.
The layers are identified via string matching of their names. This works for GPT2/BLOOM/OPT/GPT-neo. If a custom pattern is necessary one could use the
patternkeyword.