When trying to get the MPI Plugin running, I got this error:

│ flyte 2023/06/23 17:20:25 /go/pkg/mod/gorm.io/gorm@v1.24.1-0.20221019064659-5dd2bb482755/finisher_api.go:509                                                                                                                            │
│ flyte [1.083ms] [rows:1] SELECT count(*) FROM pg_indexes WHERE tablename = 'artifacts' AND indexname = 'artifacts_dataset_uuid_idx' AND schemaname = CURRENT_SCHEMA()                                                                   │
│ flyte {"json":{"src":"start.go:185"},"level":"panic","msg":"Failed to start Propeller, err: Error running FlytePropeller.: failed to load plugin - mpi: [PluginInitializationFailed] Error getting informer for %!s(\u003cnil\u003e), c │
│ flyte panic: (*logrus.Entry)

The MPI Kubernetes Operator works. I followed the documentation from here: https://docs.flyte.org/en/latest/deployment/plugins/k8s/index.html#deployment-plugin-setup-k8s and we deployed the most reason 1.7 flyte release.

how do you install mpi-plugin? from mpi-operator repo?

The MPI operator is from this repo https://github.com/kubeflow/mpi-operator

could you try to use training operator instead? https://github.com/kubeflow/training-operator

Sorry for the late response! The training operator did the job. Thanks!