-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add initialization hook to customize botocore sessions #2682
Conversation
Codecov Report
@@ Coverage Diff @@
## develop #2682 +/- ##
========================================
Coverage 95.29% 95.30%
========================================
Files 60 60
Lines 12252 12261 +9
========================================
+ Hits 11676 11685 +9
Misses 576 576
Continue to review full report at Codecov.
|
This adds an initialization hook to customize any botocore sessions. Botocore sessions provide context for creating one or more clients. It allows you to provide configuration, components, hooks, etc. that can be shared across all clients created within that session. There is no similar concept for sessions. Specifically, there is no way to provide a common set of configuration, components, hooks, etc. that can shared across all sessions created within botocore. This change adds this initialization hook. You can now register an initialization callback that's invoked whenever a session is created: ```python import botocore def my_initializer(session: botocore.session.Session) -> None: session.register_component('data_loader', MyNewCrazyLoader()) botocore.register_initializer(my_initializer) ``` Now every session will automatically use this new loader: ```python some_session = botocore.session.get_session() assert isinstance( some_session.get_component('data_loader'), MyNewCrazyLoader ) ```
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me! Only thought is maybe it would be useful to have a method that invokes specific initializers rather than all of them carte blanche. Would require a refactor since invoke_initializers
is called at initialization, but just an idea! 🚢
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
* release-1.27.12: Bumping version to 1.27.12 Update to latest endpoints Update to latest models Add initialization hook for session creation (#2682)
This adds an initialization hook to customize any botocore sessions.
Botocore sessions provide context for creating one or more clients.
It allows you to provide configuration, components, hooks, etc. that
can be shared across all clients created within that session.
There is no similar concept for sessions. Specifically, there is no way
to provide a common set of configuration, components, hooks, etc. that
can shared across all sessions created within botocore.
This PR adds this initialization hook. You can now register an initialization
callback that's invoked whenever a session is created:
Now every session will automatically use this new loader:
Rationale
Botocore's extensibility points allows customers to customize the behavior of
botocore. However, in order to leverage these extensibility points, customers
must write explicit code to register these hooks that requires a reference to a
specific session (e.g
session.register
,session.register_component
,session.set_default_client_config
). There is no way to provide "plugin" likebehavior where customers can declaratively specify functionality without having
to modify all their existing code.
Here's one concrete use case. Botocore has a data loader component that's
responsible for loading the service model based on various parameters
(
service_name
,type_name
,api_version
). The current implementation usesJSON, which is not efficient in terms of disk space nor load time. There have
been initiatives to improve this (#2628), but in order to make this default
behavior, the existing
JSONFileLoader
class had to be modified.There are also crazier use case specific ideas to further optimize both the
load time and disk size of our service models. However, in order to use
a new loader, users must add this line of code to every session they create:
This is fine if you are able to obtain a reference to every single
session you create, but this isn't always the case, especially if
you're not directly using a session (e.g. some high level library
that wraps sessions/clients for you).
I maintain a framework, Chalice, where I would like to allow users
to opt-in to more optimized data loaders through a framework-specific
config file, without having to change any of their existing code.