Any particular reason to have the replica count in flyte-binary helm chat to be hard-coded to

1

? If not, I can make a quick PR make that as variable.

The flyte-binary helm chart is for the single binary deployment strategy correct? If so, there are scaling considerations (ex. only a single propeller instance should be running at a time). cc @average-finland-92144

That was my suspicion @hallowed-mouse-14616 , with single binary and multiple replicas, is there still a leader election mechanism in place?

@hallowed-mouse-14616 / @average-finland-92144 in my POC env, i tried to scale the

flyte-binary

deployment to 3 pods. From the logs I do not find a leader election mechanism. it appears to be working as normal.

From the logs I do not find a leader election mechanism.

This could be problematic, for consistency there can be only a propeller instance running. This is gracefully handled in`flyte-core`

@cuddly-energy-9218 what is your requirement for scaling the

flyte-binary

setup? If it is resilience, my understanding is that

flyte-binary

is meant as a quick and easy deploy strategy but more complex production-level deployments are better using a different model. @average-finland-92144 do I understand this correctly?

flyte-binary

is backed by a Deployment so it has some level of resiliency with

flyte-core

more geared towards multicluster setups. Being able to independently scale Flyte components (like propeller) is another reason to consider flyte-core

flyte-binary

is backed by a Deployment so it has some level of resiliency

Deployment usually have more than one replica right? I do not plan to setup multi cluster setup (each components run in different cluster). However I do want to have some level of resiliency than just a single pod running (typically Deployment does that, by increasing the replica count)

got it. It's just that the tradeoff here between resiliency and consistency makes me think that just going to flyte-core would be a better move

@average-finland-92144 Thank you so much for taking the time to share your knowledge. I really appreciate it. I hope I don't sound argumentative, but I find the Flyte documentation a bit confusing.

single Flyte cluster can handle more than 13,000 nodes.

did they mean this by running just one pod in a deployment? > We recommend the Single Cluster option for a capable though not massively scalable cluster. > This option is appropriate if all your compute can fit on one EKS cluster . As of this writing, a single Flyte cluster can handle more than 13,000 nodes.

While I'm not sure how that number was measured, most of the load in the workflow lifecycle is handled by propeller workers which are goroutines, very lightweight concurrent processing threads which can accommodate such numbers with a not massive configuration of CPU/memory to the Pod. Even with large fan-out workflows Flyte includes some mechanisms to deal with parallelism that doesn't necessarily mean adding another propeller instance. So I don't think load is a parameter to choose one Flyte form factor over the other (binary vs core)