Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Update install to support ClusterServingRuntime #252

Merged
merged 9 commits into from
Oct 31, 2022
Merged

feat: Update install to support ClusterServingRuntime #252

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
5843682
Update install to support ClusterServingRuntime
chinhuang007 Sep 29, 2022
2c77451
switch to cluster scope for FVT
chinhuang007 Oct 4, 2022
9ff1097
make CSR as default runtimes
chinhuang007 Oct 6, 2022
8f3391f
fix unit tests to use ServingRuntimes
chinhuang007 Oct 7, 2022
e3ec525
make e2e test to run in cluster scope mode
chinhuang007 Oct 7, 2022
af0baee
update fvt and install to address comments
chinhuang007 Oct 10, 2022
8e4a032
code update, restructure and cleanup based on review comments
chinhuang007 Oct 27, 2022
2319240
A couple more minor changes from review
njhill Oct 29, 2022
164e26a
Linting fix :facepalm:
njhill Oct 29, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/run-fvt.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,4 +81,5 @@ jobs:
go install github.com/onsi/ginkgo/v2/ginkgo
export PATH=/root/go/bin/:$PATH
export NAMESPACE=modelmesh-serving
export NAMESPACESCOPEMODE=false
make fvt
32 changes: 31 additions & 1 deletion config/crd/bases/serving.kserve.io_clusterservingruntimes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copied from https://github.com/kserve/kserve/blob/39ec2d19de1d856a9527b800fda8fb380bc9441f/config/crd/serving.kserve.io_clusterservingruntimes.yaml
# Copied from https://github.com/kserve/kserve/blob/ff7014b0c1a79672978d5b0a23af6c5ae1158b3b/config/crd/serving.kserve.io_clusterservingruntimes.yaml
#
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
Expand Down Expand Up @@ -677,6 +677,16 @@ spec:
failureThreshold:
format: int32
type: integer
grpc:
properties:
port:
format: int32
type: integer
service:
type: string
required:
- port
type: object
httpGet:
properties:
host:
Expand Down Expand Up @@ -771,6 +781,16 @@ spec:
failureThreshold:
format: int32
type: integer
grpc:
properties:
port:
format: int32
type: integer
service:
type: string
required:
- port
type: object
httpGet:
properties:
host:
Expand Down Expand Up @@ -919,6 +939,16 @@ spec:
failureThreshold:
format: int32
type: integer
grpc:
properties:
port:
format: int32
type: integer
service:
type: string
required:
- port
type: object
httpGet:
properties:
host:
Expand Down
32 changes: 31 additions & 1 deletion config/crd/bases/serving.kserve.io_servingruntimes.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copied from https://github.com/kserve/kserve/blob/39ec2d19de1d856a9527b800fda8fb380bc9441f/config/crd/serving.kserve.io_servingruntimes.yaml
# Copied from https://github.com/kserve/kserve/blob/ff7014b0c1a79672978d5b0a23af6c5ae1158b3b/config/crd/serving.kserve.io_servingruntimes.yaml
#
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
Expand Down Expand Up @@ -677,6 +677,16 @@ spec:
failureThreshold:
format: int32
type: integer
grpc:
properties:
port:
format: int32
type: integer
service:
type: string
required:
- port
type: object
httpGet:
properties:
host:
Expand Down Expand Up @@ -771,6 +781,16 @@ spec:
failureThreshold:
format: int32
type: integer
grpc:
properties:
port:
format: int32
type: integer
service:
type: string
required:
- port
type: object
httpGet:
properties:
host:
Expand Down Expand Up @@ -919,6 +939,16 @@ spec:
failureThreshold:
format: int32
type: integer
grpc:
properties:
port:
format: int32
type: integer
service:
type: string
required:
- port
type: object
httpGet:
properties:
host:
Expand Down
3 changes: 1 addition & 2 deletions config/crd/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ resources:
# Including creation of Service here for now, will later be done automatically
- bases/serving.kserve.io_predictors.yaml
- bases/serving.kserve.io_inferenceservices.yaml
# ClusterServingRuntime not yet supported by modelmesh-serving
# - bases/serving.kserve.io_clusterservingruntimes.yaml
- bases/serving.kserve.io_clusterservingruntimes.yaml
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering whether we should include this in namespace scope case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. Since we won't do anything with CSRs in namespace scope mode, probably can exclude it during install.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chinhuang007 WDYT about excluding this for namespace mode? We could just have a separate cluster_crd kustomization which includes this one + the CSR CRD

- bases/serving.kserve.io_servingruntimes.yaml
# +kubebuilder:scaffold:crdkustomizeresource

Expand Down
14 changes: 14 additions & 0 deletions config/namespace-runtimes/kustomization.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
- ../runtimes
patchesJson6902:
- target:
group: serving.kserve.io
version: v1alpha1
kind: ClusterServingRuntime
name: ".*"
patch: |-
- op: replace
path: /kind
value: ServingRuntime
21 changes: 21 additions & 0 deletions config/rbac/cluster-scope/role.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,3 +175,24 @@ rules:
- get
- patch
- update
- apiGroups:
- serving.kserve.io
resources:
- clusterservingruntimes
- clusterservingruntimes/finalizers
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- serving.kserve.io
resources:
- clusterservingruntimes/status
verbs:
- get
- patch
- update
2 changes: 1 addition & 1 deletion config/runtimes/mlserver-0.x.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: serving.kserve.io/v1alpha1
kind: ServingRuntime
kind: ClusterServingRuntime
metadata:
name: mlserver-0.x
labels:
Expand Down
2 changes: 1 addition & 1 deletion config/runtimes/ovms-1.x.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: serving.kserve.io/v1alpha1
kind: ServingRuntime
kind: ClusterServingRuntime
metadata:
name: ovms-1.x
labels:
Expand Down
2 changes: 1 addition & 1 deletion config/runtimes/triton-2.x.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: serving.kserve.io/v1alpha1
kind: ServingRuntime
kind: ClusterServingRuntime
metadata:
name: triton-2.x
labels:
Expand Down
13 changes: 10 additions & 3 deletions controllers/servingruntime_controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ var _ = Describe("Sample Runtime", func() {
m, err = mf.ManifestFrom(mf.Path(filepath.Join("..", sampleFilename)))
m.Client = mfc.NewClient(k8sClient)
Expect(err).ToNot(HaveOccurred())
m, err = m.Transform(mf.InjectNamespace(namespace))
m, err = m.Transform(convertToServingRuntime, mf.InjectNamespace(namespace))
Expect(err).ToNot(HaveOccurred())
err = m.Apply()
Expect(err).ToNot(HaveOccurred())
Expand Down Expand Up @@ -100,7 +100,7 @@ var _ = Describe("Prometheus metrics configuration", func() {
m, err = mf.ManifestFrom(mf.Path("../config/runtimes/mlserver-0.x.yaml"))
m.Client = mfc.NewClient(k8sClient)
Expect(err).ToNot(HaveOccurred())
m, err = m.Transform(mf.InjectNamespace(namespace))
m, err = m.Transform(convertToServingRuntime, mf.InjectNamespace(namespace))
Expect(err).ToNot(HaveOccurred())
err = m.Apply()
Expect(err).ToNot(HaveOccurred())
Expand Down Expand Up @@ -179,7 +179,7 @@ var _ = Describe("REST Proxy configuration", func() {
m, err = mf.ManifestFrom(mf.Path("../config/runtimes/mlserver-0.x.yaml"))
m.Client = mfc.NewClient(k8sClient)
Expect(err).ToNot(HaveOccurred())
m, err = m.Transform(mf.InjectNamespace(namespace))
m, err = m.Transform(convertToServingRuntime, mf.InjectNamespace(namespace))
Expect(err).ToNot(HaveOccurred())
err = m.Apply()
Expect(err).ToNot(HaveOccurred())
Expand All @@ -193,3 +193,10 @@ var _ = Describe("REST Proxy configuration", func() {
Expect(d).To(SnapshotMatcher())
})
})

func convertToServingRuntime(resource *unstructured.Unstructured) error {
if resource.GetKind() == "ClusterServingRuntime" {
resource.SetKind("ServingRuntime")
}
return nil
}
8 changes: 8 additions & 0 deletions fvt/fvtclient.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,11 @@ var (
Version: api.GroupVersion.Version,
Resource: "servingruntimes", // this must be the plural form
}
gvrCRuntime = schema.GroupVersionResource{
Group: api.GroupVersion.Group,
Version: api.GroupVersion.Version,
Resource: "clusterservingruntimes", // this must be the plural form
}
gvrPredictor = schema.GroupVersionResource{
Group: api.GroupVersion.Group,
Version: api.GroupVersion.Version,
Expand Down Expand Up @@ -271,6 +276,9 @@ func (fvt *FVTClient) ListServingRuntimes(options metav1.ListOptions) (*unstruct
return fvt.Resource(gvrRuntime).Namespace(fvt.namespace).List(context.TODO(), options)
}

func (fvt *FVTClient) ListClusterServingRuntimes(options metav1.ListOptions) (*unstructured.UnstructuredList, error) {
return fvt.Resource(gvrCRuntime).List(context.TODO(), options)
}
func (fvt *FVTClient) GetPredictor(name string) *unstructured.Unstructured {
obj, err := fvt.Resource(gvrPredictor).Namespace(fvt.namespace).Get(context.TODO(), name, metav1.GetOptions{})
Expect(err).ToNot(HaveOccurred())
Expand Down
1 change: 1 addition & 0 deletions fvt/globals.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ var Log logr.Logger
var FVTClientInstance *FVTClient

var DefaultTimeout = int64(120)
var NameSpaceScopeMode = false

var DefaultConfig = map[string]interface{}{
"podsPerRuntime": 1,
Expand Down
15 changes: 12 additions & 3 deletions fvt/predictor/predictor_suite_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
. "github.com/kserve/modelmesh-serving/fvt"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
)

func TestPredictorSuite(t *testing.T) {
Expand All @@ -47,7 +48,9 @@ func createFVTClient() {
if controllerNamespace == "" {
controllerNamespace = DefaultControllerNamespace
}
Log.Info("Using environment variables", "NAMESPACE", namespace, "SERVICENAME", serviceName, "CONTROLLERNAMESPACE", controllerNamespace)
NameSpaceScopeMode = os.Getenv("NAMESPACESCOPEMODE") == "true"
Log.Info("Using environment variables", "NAMESPACE", namespace, "SERVICENAME", serviceName,
"CONTROLLERNAMESPACE", controllerNamespace, "NAMESPACESCOPEMODE", NameSpaceScopeMode)

var err error
FVTClientInstance, err = GetFVTClient(Log, namespace, serviceName, controllerNamespace)
Expand All @@ -60,8 +63,14 @@ var _ = SynchronizedBeforeSuite(func() []byte {
// runs *only* on process #1
createFVTClient()

// confirm 3 serving runtimes exist
list, err := FVTClientInstance.ListServingRuntimes(metav1.ListOptions{})
// confirm 3 cluster serving runtimes or serving runtimes exist
var err error
var list *unstructured.UnstructuredList
if NameSpaceScopeMode {
list, err = FVTClientInstance.ListServingRuntimes(metav1.ListOptions{})
} else {
list, err = FVTClientInstance.ListClusterServingRuntimes(metav1.ListOptions{})
}
Expect(err).ToNot(HaveOccurred())
Expect(list.Items).To(HaveLen(3))

Expand Down
15 changes: 12 additions & 3 deletions fvt/scaleToZero/scaleToZero_suite_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ import (
"testing"
"time"

"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"sigs.k8s.io/controller-runtime/pkg/log/zap"

Expand Down Expand Up @@ -51,16 +53,23 @@ var _ = SynchronizedBeforeSuite(func() []byte {
if controllerNamespace == "" {
controllerNamespace = DefaultControllerNamespace
}
Log.Info("Using environment variables", "NAMESPACE", namespace, "SERVICENAME", serviceName, "CONTROLLERNAMESPACE", controllerNamespace)
NameSpaceScopeMode = os.Getenv("NAMESPACESCOPEMODE") == "true"
Log.Info("Using environment variables", "NAMESPACE", namespace, "SERVICENAME", serviceName,
"CONTROLLERNAMESPACE", controllerNamespace, "NAMESPACESCOPEMODE", NameSpaceScopeMode)

var err error
FVTClientInstance, err = GetFVTClient(Log, namespace, serviceName, controllerNamespace)
Expect(err).ToNot(HaveOccurred())
Expect(FVTClientInstance).ToNot(BeNil())
Log.Info("FVTClientInstance created", "client", FVTClientInstance)

// confirm 3 serving runtimes exist
list, err := FVTClientInstance.ListServingRuntimes(metav1.ListOptions{})
// confirm 3 cluster serving runtimes or serving runtimes
var list *unstructured.UnstructuredList
if NameSpaceScopeMode {
list, err = FVTClientInstance.ListServingRuntimes(metav1.ListOptions{})
} else {
list, err = FVTClientInstance.ListClusterServingRuntimes(metav1.ListOptions{})
}
Expect(err).ToNot(HaveOccurred())
Expect(list.Items).To(HaveLen(3))

Expand Down
3 changes: 1 addition & 2 deletions scripts/deploy/iks/test-fvt.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ retry() {
# Run fvt tests and look for PASS
run_fvt() {
local REV=1
local RUN_STATUS="FAILED"

echo " ===== run standard fvt ====="
kubectl get all -n "$SERVING_NS"
Expand All @@ -49,9 +48,9 @@ run_fvt() {
export PATH=/root/go/bin/:$PATH

export NAMESPACE=${SERVING_NS}
export NAMESPACESCOPEMODE=false
ginkgo -v --progress --fail-fast -p fvt/predictor fvt/scaleToZero --timeout 40m > fvt.out
cat fvt.out
RUN_STATUS=$(cat fvt.out | awk '{ print $1}' | grep PASS)

if [[ $(grep "Test Suite Passed" fvt.out) ]]; then
export NAMESPACE="modelmesh-user"
Expand Down
Loading