👋 Some sanity checks on Flyte deployments 1. The current flyte-binary helm chart only supports using

s3

and

gcs

storage backends, correct? I saw this discussion post that showed an alternate way to configure

stow

with Azure, but it is unclear which chart is being used (I assume the flyte one). Reading the

flyte-binary

chart source, I can't see how any other backend is possible. 2. The

stow

sdk only supports using those unfortunate azure shared keys (stow code referenced here). Is this right, or should I be looking somewhere else to enable AD auth. Ideally we'd want Flyte's storage auth do the simple

azidentity.NewDefaultAzureCredential(nil)

thing so Azure Workload Identity can be used.

Hey @Terence Kent, 1. You are right, this is a miss. Would you be open to pushing a PR to allow more customizations for flyte-binary helm chart? 2. Also correct, this is forked off stow/stow project that's no longer maintained. There are a few Azure experts on this channel who have been running Flyte in prod on Azure for months+ years+ (@Mathias Andersen, @Gopal Vashishtha @Anant Mital may be able to help?) There is #azure-wg with excellent talent that can help you navigate this... I would love to lean on the community to bring Azure support to the 21st century 🙂 changes like what you suggested for Auth is so welcome and we will be happy to merge those

@Haytham Abuelfutuh - thanks for the sanity checks! In our AKS deployments, we'll need to use Azure blobs and we can't have shared keys enabled on any of our storage accounts. Since I'm hoping to get a production deployment up and going soon, I'll look into sending along PRs. Mainly depends on how reasonable testing the changes are. The good news is, it seems our EKS will be a breeze.

@Terence Kent are you on AKS or EKS?

(both)

wow!

We don't operate a large number of nodes ATM, so don't take the cloud spanning to indicate size 🙂. It's just that our company's services are split between AWS and Azure. Our team is trying to use AWS for everything, but we operate a couple of k8s clusters in each. For expediency, it's often easier to run workloads in our AKS clusters where the nodes already have network access to Azure services without any public network access.

fair