Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error messages when MPIJob CRD is not installed #665

Closed
alculquicondor opened this issue Mar 29, 2023 · 3 comments · Fixed by #668
Closed

Error messages when MPIJob CRD is not installed #665

alculquicondor opened this issue Mar 29, 2023 · 3 comments · Fixed by #668
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@alculquicondor
Copy link
Contributor

What happened:

Even if a user doesn't plan to use MPIJob, they might observe errors in the logs, which can be confusing.

{"level":"error","ts":"2023-03-29T17:21:00.928174568Z","logger":"controller-runtime.source","caller":"source/source.go:143","msg":"if kind is a CRD, it should be installed before calling Start","kind":"[MPIJob.kubeflow.org](http://mpijob.kubeflow.org/)","error":"no matches for kind \"MPIJob\" in version \"[kubeflow.org/v2beta1\](http://kubeflow.org/v2beta1%5C)"","stacktrace":"[sigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start.func1.1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/source/source.go:143\nk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:235\nk8s.io/apimachinery/pkg/util/wait.WaitForWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:662\nk8s.io/apimachinery/pkg/util/wait.poll\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:596\nk8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext\n\t/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:547\nsigs.k8s.io/controller-runtime/pkg/source.(*Kind).Start.func1\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/source/source.go:136](http://sigs.k8s.io/controller-runtime/pkg/source.%28*Kind%29.Start.func1.1%5Cn%5Ct/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/source/source.go:143%5Cnk8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext%5Cn%5Ct/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:235%5Cnk8s.io/apimachinery/pkg/util/wait.WaitForWithContext%5Cn%5Ct/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:662%5Cnk8s.io/apimachinery/pkg/util/wait.poll%5Cn%5Ct/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:596%5Cnk8s.io/apimachinery/pkg/util/wait.PollImmediateUntilWithContext%5Cn%5Ct/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:547%5Cnsigs.k8s.io/controller-runtime/pkg/source.%28*Kind%29.Start.func1%5Cn%5Ct/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/source/source.go:136)"}

What you expected to happen:

No errors. Perhaps this could be avoided by explicitly enabling MPI support via command line flag.

I wouldn't do this through the configuration API, as we plan to eventually move integrations into their own binaries.

How to reproduce it (as minimally and precisely as possible):

  1. Do not install mpi-operator
  2. Install kueue.
  3. See logs

Anything else we need to know?:

Environment:

  • Kubernetes version (use kubectl version):
  • Kueue version (use git describe --tags --dirty --always):
  • Cloud provider or hardware configuration:
  • OS (e.g: cat /etc/os-release):
  • Kernel (e.g. uname -a):
  • Install tools:
  • Others:
@alculquicondor alculquicondor added the kind/bug Categorizes issue or PR as related to a bug. label Mar 29, 2023
@alculquicondor
Copy link
Contributor Author

/assign @mimowo

@mimowo
Copy link
Contributor

mimowo commented Mar 30, 2023

I wouldn't do this through the configuration API, as we plan to eventually move integrations into their own binaries.

When we split the mpi integration into a separate binary the problem will go away naturally.

Do you think we should be solving this short-term? If we are going to a short term solution we could probably supply a custom logger which filters out the message.

cc @mwielgus

@alculquicondor
Copy link
Contributor Author

/unassign @mimowo
/assign @mwielgus

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants