https://flyte.org logo
#ask-the-community
Title
# ask-the-community
v

Vitali

06/26/2023, 3:42 PM
When trying to get the MPI Plugin running, I got this error:
Copy code
│ flyte 2023/06/23 17:20:25 /go/pkg/mod/gorm.io/gorm@v1.24.1-0.20221019064659-5dd2bb482755/finisher_api.go:509                                                                                                                            │
│ flyte [1.083ms] [rows:1] SELECT count(*) FROM pg_indexes WHERE tablename = 'artifacts' AND indexname = 'artifacts_dataset_uuid_idx' AND schemaname = CURRENT_SCHEMA()                                                                   │
│ flyte {"json":{"src":"start.go:185"},"level":"panic","msg":"Failed to start Propeller, err: Error running FlytePropeller.: failed to load plugin - mpi: [PluginInitializationFailed] Error getting informer for %!s(\u003cnil\u003e), c │
│ flyte panic: (*logrus.Entry)
The MPI Kubernetes Operator works. I followed the documentation from here: https://docs.flyte.org/en/latest/deployment/plugins/k8s/index.html#deployment-plugin-setup-k8s and we deployed the most reason 1.7 flyte release.
k

Kevin Su

06/26/2023, 4:02 PM
how do you install mpi-plugin? from mpi-operator repo?
v

Vitali

06/27/2023, 7:05 AM
The MPI operator is from this repo https://github.com/kubeflow/mpi-operator
k

Kevin Su

06/27/2023, 7:09 AM
could you try to use training operator instead? https://github.com/kubeflow/training-operator
we just tested 1.5, and it works for us.
Copy code
kubectl apply -k "<http://github.com/kubeflow/training-operator/manifests/overlays/standalone?ref=v1.5.0|github.com/kubeflow/training-operator/manifests/overlays/standalone?ref=v1.5.0>"
v

Vitali

07/17/2023, 7:43 PM
Sorry for the late response! The training operator did the job. Thanks!
3 Views