acoustic-carpenter-78188
09/02/2023, 12:36 AMqueuing_budget that would allow a user to specify an amount of time they are willing to wait for their workflow to be executed. FlytePropeller would take this value and decide when "best" to execute the workflow.
Propeller is responsible for ensuring max_queue_time is honored for each execution. Users (via FlyteAdmin) will configure queuing_budget in the workflow CRD similar to how they configure ActiveDeadline today. When starting a new node in the control loop, the propeller does a precondition check (dependencies are satisfied, etc) which will be enhanced to also calculate the current queueing_budget for that node based on what we have consumed so far for that execution. Propeller will keep track of total_queued_time so far for the workflow in the WF status. When a node moves to the running state or terminal state, we update the total_queued_time of the node and the WF (based on the delay in completion caused to the WF by that node).
Describe alternatives you've considered
Flyte component
☐ Overall
☐ Flyte Setup and Installation scripts
☐ Flyte Documentation
☐ Flyte communication (slack/email etc)
☑︎ FlytePropeller
☑︎ FlyteIDL (Flyte specification language)
☑︎ Flytekit (Python SDK)
☐ FlyteAdmin (Control Plane service)
☐ FlytePlugins
☐ DataCatalog
☐ FlyteStdlib (common libraries)
☐ FlyteConsole (UI)
☐ Other
Additional context
Add any other context or screenshots about the feature request here.
Is this a blocker for you to adopt Flyte
Please let us know if this makes it impossible to adopt Flyte
flyteorg/flyteacoustic-carpenter-78188
09/02/2023, 12:36 AM