#6436 Only enqueue workflows if flytepropeller is leader
Pull request opened by
Sovietaced
Tracking issue
Closes
#6435
Why are the changes needed?
Without these changes followers/replicas in highly available deployments will enqueue workflows until they saturate the work queue. Once the work queue is saturated these workflows may become very stale. Once a follower does become a leader it will need to churn through these stale workflows that might not even exist anymore.
What changes were proposed in this pull request?
Added a thread safe bool for whether the propeller is leader and check it before enqueueing workflows.
How was this patch tested?
Not yet but will test in our development environment.
Check all the applicable boxes
• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.
Summary by Bito
This PR implements leader-only workflow enqueuing in HA flytepropeller deployments using an atomic boolean flag, preventing followers from adding stale workflows to the queue. It enhances system reliability and consistency during leadership transitions, and refines workflow handling. The changes also improve logging of workflow phases for failed executions to facilitate better monitoring and debugging.
Unit tests added: False
Estimated effort to review (1-5, lower is better): 1
flyteorg/flyte