jolly-florist-66728
09/23/2024, 7:08 AMflyte-binary deployment by increasing the number of pods in the deployment? Or if we want scaling out, we need to migrate to using flyte-core and separate deployments for propeller, datacatalog and etc?
# Answer
Scaling out the flyte-binary deployment by increasing the number of pods is not recommended. According to Flyte community discussions, simply adding more binary pods will not effectively scale the system. Instead, you should use flyte-core for better scalability, which allows for separate deployments of FlyteAdmin, FlytePropeller, DataCatalog, etc.
For large-scale deployments, flyte-core is designed to handle more than a thousand concurrent workflows by deploying multiple replicas of each component and enabling independent scaling of FlytePropeller instances (Flyte GitHub issue).
Would you like more details on how to migrate to flyte-core and configure it for scaling?
Sources:
- Flyte community discussions
- Flyte GitHub issue
Slack Link
https://.slack.com/archives/CP2HDHKE1/p1727075325.456639 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.