docs: Update docs to make isvc focal #190
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Paul Van Eck <[email protected]>
pvaneck
force-pushed
the
update-isvc-docs
branch
from
35cd183 to
e623d96
Compare
| @@ -11,9 +11,9 @@ The model data itself is pulled from one or more external [storage instances](pr | |||
| ModelMesh Serving makes use of two core Kubernetes Custom Resource types: | |||
|
|
|||
| - `ServingRuntime` - Templates for Pods that can serve one or more particular model formats. There are three "built in" runtimes that cover the out-of-the-box model types (Triton, MLServer and OpenVINO Model Server OVMS), [custom runtimes](runtimes/) can be defined by creating additional ones. | |||
| - [`Predictor`](predictors/) - This represents a logical endpoint for serving predictions using a particular model. The Predictor spec specifies the model type, the storage in which it resides and the path to the model within that storage. The corresponding endpoint is "stable" and will seamlessly transition between different model versions or types when the spec is updated. | |||
| - [`InferenceService`](predictors/) - This is the main interface KServe uses for managing models on Kubernetes. ModelMesh Serving can be used for deploying `InferenceService` predictors which represent a logical endpoint for serving predictions using a particular model. The `InferenceService` predictor spec specifies the model format, the storage location in which the model resides, and other optional configuration. The corresponding endpoint is "stable" and will seamlessly transition between different model versions or types when the spec is updated. Note that many features like transformers, explainers, and canary rollouts do not currently apply or fully work using InferenceServices with `deploymentMode` set to `ModelMesh`. And `PodSpec` fields that are set in the `InferenceService` predictor spec will be ignored. | |||
There was a problem hiding this comment.
I am not sure the "InferenceService link" should point to the ModelMesh predictors docs. I understand ModelMesh implements only the predictor component in the KServe InferenceService spec. But it feels a little strange to follow the link and see nothing about InferenceService. I would suggest either remove the link or have it point to KServe InferenceService.
| - `modelId` - The internal id of the model in question. This includes a hash of the InferenceService's predictor spec. | ||
| - `time` - The time at which the failure occurred, if applicable. | ||
|
|
||
| Upon creation, InferenceService the model status will always transition to `Loaded` state (unless the loading fails), but later if unused it is possible that they end up in a `Standby` state which means they are still available to serve requests but the first request could incur a loading delay. Whether this happens is a function of the available capacity and usage pattern of other models. It's possible that models will transition from `Standby` back to `Loaded` "by themselves" if more capacity becomes available. |
There was a problem hiding this comment.
Maybe just me, not quite sure
- what "InferenceService the model status" means. I suppose it means the model status of the InferenceService?
- what "they" refer to in "...they end up in a
Standbystate which means they are still available to serve requests..." I suppose they means InferenceServices?
There was a problem hiding this comment.
Yea, it's not quite clear here. I will reword.
There was a problem hiding this comment.
Thanks, @pvaneck ! Looks very good, just a few minor comments.
Signed-off-by: Paul Van Eck <[email protected]>
|
Thanks, @chinhuang007 for the thorough review! |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: chinhuang007, pvaneck The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
pvaneck
added a commit
to pvaneck/modelmesh-serving
that referenced
this pull request
Jul 21, 2022
#### Motivation The InferenceService CRD has evolved enough to become the primary interface for interacting with ModelMesh. The documentation should reflect that. #### Modifications Documentation was adjusted to make the InferenceService CRD focal as opposed to the previous Predictor CRD. Examples and snippets were added for InferenceServices. #### Result Users will learn and become more familiar with deploying on ModelMesh using the KServe InferenceService. Signed-off-by: Paul Van Eck <[email protected]>
pvaneck
added a commit
that referenced
this pull request
Jul 21, 2022
#### Motivation The InferenceService CRD has evolved enough to become the primary interface for interacting with ModelMesh. The documentation should reflect that. #### Modifications Documentation was adjusted to make the InferenceService CRD focal as opposed to the previous Predictor CRD. Examples and snippets were added for InferenceServices. #### Result Users will learn and become more familiar with deploying on ModelMesh using the KServe InferenceService. Signed-off-by: Paul Van Eck <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
The InferenceService CRD has evolved enough to become the primary interface for interacting with ModelMesh.
The documentation should reflect that.
Modifications
Documentation was adjusted to make the InferenceService CRD focal as opposed to the previous Predictor CRD.
Examples and snippets were added for InferenceServices.
Result
Users will learn and become more familiar with deploying on ModelMesh using the KServe InferenceService.