https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
x

Xinzhou Liu

11/20/2023, 7:20 AM
Does Flyte support a multi-cluster deployment with 
multiple cloud providers
? For example, can the data planes be distributed across a mixture of GCP clusters and AWS clusters? This doc is not very clear whether it’s possible or not.
y

Yee

11/20/2023, 9:26 AM
even if it is possible, this is thoroughly untested at least in oss and we do not recommend this.
you’d also need to pay a substantial amount of care around performance and cross-provider network/data traffic costs.
that said this is one of the features supported by union cloud
the multi-cluster deployment guide is generally for multiple data clusters, or for users who want to split control plane and data plane.
the exception to the above is that people have successfully used both aws and on-prem. however i don’t know of any oss users who have used both aws and gcp
could be wrong
k

Ketan (kumare3)

11/20/2023, 3:13 PM
@Xinzhou Liu it’s possible but not tested and there are a few rough edges or some missing parts as this is not common. Core Flyte is portable by design Union.ai s hosted implementation is out of the box supports multi cluster like kubelets and you can indeed bring up clusters anywhere - happy to talk about u ion as well
x

Xinzhou Liu

11/20/2023, 5:10 PM
Thank you both for the clarification! The reason I asked is because I think it might be cost efficient to utilize spot instances from both cloud providers to run more workflows. Sounds like union.ai only supports cluster deployment within single cloud provider?
k

Ketan (kumare3)

11/20/2023, 5:13 PM
No infact the reverse, but we would love to talk more
x

Xinzhou Liu

11/20/2023, 5:14 PM
oh, i misunderstood, great! I’d love to know more
Let’s schedule a chat sometime? Meanwhile, is there any doc on how to do that with union.ai?
k

Ketan (kumare3)

11/20/2023, 6:05 PM
cc @Sean Chiarito can you help to connect
s

Sean Chiarito

11/20/2023, 10:25 PM
@Xinzhou Liu great to e-meet. happy to find time for us to chat
x

Xinzhou Liu

11/20/2023, 10:40 PM
Thanks for reaching out, Sean! I haven’t extensively discussed this idea with my team, but I’d like to learn about the solutions union.ai can offer. Let’s meet virtually sometime this week or next.
Powered by