Hi, a little bit late introduction, since I am lurking here for a few months already. I am a software engineer at Atos (with background in DevOps). With a small team, we are adding machine learning capabilities (timeseries forecasting) to an existing (internal) product. Aside of general development tasks I have assumed an MLOps role, and use Flyte to orchestrate our pipelines: data transformations, model training and inference. We are 100% on prem (due to regulations) and we had to fit Flyte to our existing automation which was a bit tricky. I am still learning a lot while preparing for a pilot deployment of our application. Hope at some point I will be able to contribute in some meaningful way. Thanks for your great work!

Thank you for sharing

Welcome Maciej and thanks for sharing. I'm interested to learn about your on-prem deployment

Sure, I'm happy to provide more info (as much as I am legally allowed anyway). We host our infra on vSphere and chose microk8s as Kubernetes distribution. Infrastructure and platform (up to the application level) is managed with Ansible, triggered with declarative CI/CD pipelines we wrote in GitLab (also hosted on prem). Our applications on the other hand are deployed using GitOps pattern, so manifests are merged to repos and applied by GitLab Agent for Kubernetes. We have faced a bit of challenge here, because we did not really use Helm for anything major (because there is no support for it in GitLab Agent), so we had no pattern for managing Helm "values" files. We ended up with deploying charts using Ansible, while Flyte configuration is delivered with

externalConfigMap

and

externalSecretRef

. Not great, not terrible. We had to deploy MinIO for object storage, and we also use Ray to accelerate computation. Therefore, currently the biggest headache is with integrating Ray and Flyte. It works, but needs very specific versions and is limited when it comes to using RayJobs (I saw some merged PRs which I think are going to fix it, great!). I hope we will reach an MVP phase using Flyte 🙂.

interesting, I used to be very deeply engaged with vSphere... So I've been going through the hurdles of deploying Flyte on a multi-node K8s environment using minio too but on top of NFS. I'll be sharing the results in the form of a guide for anyone interested Let us know any other way we can support you.

We are using https://github.com/kubernetes-sigs/vsphere-csi-driver, but I am very interested in checking the guide you are working on to compare our approach. Thank you for your work!