When trying to get the MPI Plugin running, I got ...
# ask-the-community
v
When trying to get the MPI Plugin running, I got this error:
Copy code
│ flyte 2023/06/23 17:20:25 /go/pkg/mod/gorm.io/gorm@v1.24.1-0.20221019064659-5dd2bb482755/finisher_api.go:509                                                                                                                            │
│ flyte [1.083ms] [rows:1] SELECT count(*) FROM pg_indexes WHERE tablename = 'artifacts' AND indexname = 'artifacts_dataset_uuid_idx' AND schemaname = CURRENT_SCHEMA()                                                                   │
│ flyte {"json":{"src":"start.go:185"},"level":"panic","msg":"Failed to start Propeller, err: Error running FlytePropeller.: failed to load plugin - mpi: [PluginInitializationFailed] Error getting informer for %!s(\u003cnil\u003e), c │
│ flyte panic: (*logrus.Entry)
The MPI Kubernetes Operator works. I followed the documentation from here: https://docs.flyte.org/en/latest/deployment/plugins/k8s/index.html#deployment-plugin-setup-k8s and we deployed the most reason 1.7 flyte release.
k
how do you install mpi-plugin? from mpi-operator repo?
v
The MPI operator is from this repo https://github.com/kubeflow/mpi-operator
k
could you try to use training operator instead? https://github.com/kubeflow/training-operator
we just tested 1.5, and it works for us.
Copy code
kubectl apply -k "<http://github.com/kubeflow/training-operator/manifests/overlays/standalone?ref=v1.5.0|github.com/kubeflow/training-operator/manifests/overlays/standalone?ref=v1.5.0>"
v
Sorry for the late response! The training operator did the job. Thanks!
104 Views