New RunLLM response generated: >>>*# Question* is it the propeller not being able to handle the 5000 inputs at a time? # Answer FlytePropeller can handle a large number of workflows, but there are considerations for performance and resource limits. According to this discussion, FlytePropeller uses a greedy traversal algorithm, which can be a challenge with workflows that have a large fan-out. To manage this, you can set

max_parallelism

to limit the number of nodes that can run concurrently within an execution. Additionally, FlytePropeller's performance can be optimized by adjusting configurations like memory and cache settings. Would you like more detail on configuring FlytePropeller for handling large inputs? Sources: - FlytePropeller Performance Discussion - FlytePropeller Concurrency Discussion Slack Link https://.slack.com/archives/CP2HDHKE1/p1729797073.462479 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.