-
Notifications
You must be signed in to change notification settings - Fork 19
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
support for CPU/GPU choice and initialization before starting the app #2
Conversation
model: Optional[str], | ||
input: Union[str, List[str]] | ||
): | ||
def initialize_embeddings(model: Optional[str] = None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for send me the PR.
It make sense to load the text embeddings model before the first request. I leave the code of loading the embedding model on-demand back then due to I haven't make up my mind should the API serve multiple models or single model. I think it is clear now that the API serve single text embeddings model is the way to go, the code is simpler. If another text embeddings model is needed, the user can simply deploy another instance of the API.
Hence, the model
parameter no longer needed here:
def initialize_embeddings(model: Optional[str] = None): | |
def initialize_embeddings(): |
|
||
if __name__ == "__main__": | ||
initialize_embeddings() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will move the initialize_embeddings()
to create_app()
as the current approach will break the code of aws.py.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will merge the PR and make some minor changes as per my comments.
Just release the package at https://pypi.org/project/open-text-embeddings/. Kindly let's me know if you think there are ways to improve it. This is the first python package I publish to pypi. |
initialize_embeddings
function which initialize the embeddings before starting the app