hi all we currently run both our Flyte clusters in AWS EKS d

Question

hi all we currently run both our Flyte clusters in AWS EKS dev prod one of our clients wants us to run their ML workloads in Huawei cloud looking at the helm charts it looks like only `s3` and `gcs` are the only accepted `provider` for object storage any advice on how we can deploy Flyte in Huawei cloud has anyone done it before my understanding is flyte binary requires postgres db and 2 object storage buckets almost everything else is kubernetes native

Chris Grass · Answer

Hi Brian i created a recently that formalizes azure storage in `flyte binary` i think you can also use the `custom` type if config is just a passthrough to storage via but that lib might need an implementation for the huawei client

Chris Grass · Answer

your issue brings up a good point for the community to discuss though does the configuration in flyte binary need to be implementation specific or should it be a generic passthrough for stow i started a conversation <https flyte org slack com archives CP2HDHKE1 p1698250094391179|here> about whether or not the product specific storage clients should be used directly in flyte or if that work should be offloaded to stow flyte might feel a little bloated if there was an implementation for every cloud storage provider which would also likely be duplicated in stow

Ketan (kumare3) · Answer

< Brian Tang> that is right but no one has tried Flyte in huawei

Brian Tang · Answer

thanks < Chris Grass> so essentially a huawei implementation in stow + having it merged into the forked stow repo + support in the flyte binary configmap either having a kind `huawei` or using `custom` what about deploying minio as a service in the cluster and exposing those buckets to flyte this isn t ideal but i suppose that s an alternative to solving the object storage problem

Ketan (kumare3) · Answer

< Brian Tang> that should work out of the box Does it now Minio is what is used in sandbox

Chris Grass · Answer

yeah i think those are the two options the stow huawei implementation is non trivial but should be possible minio is certainly the easier option if it s available to you

Ketan (kumare3) · Answer

You don t need a full stow client implementation ideally if you look at flytestdlib storage only get put head and signedurl are used