@Glime How does the client know when a task has completed? Is there an event notification system?

To determine when a task has completed, clients can rely on the event notification system that Flyte provides. Flyte emits various types of events that allow external systems to react to state changes. These include

WorkflowExecutionEventRequest

,

TaskExecutionEventRequest

, and

NodeExecutionEventRequest

events. These events are crucial for tracking the lifecycle of tasks, workflows, and node executions within Flyte. For more granular tracking, especially for tasks within workflows, clients can monitor the progress and completion of tasks by observing the status updates provided through these events. The system's design ensures that even during periods of maintenance or downtime of the core database, the execution of long-running tasks can continue, and their progress can be monitored through these events. This is facilitated by storing all metadata regarding task execution in a highly available blob store, which, despite a potential loss in performance, ensures that task execution and monitoring can proceed uninterrupted. In summary, clients can know when a task has completed by subscribing to and monitoring the appropriate event notifications emitted by Flyte. This system provides a robust mechanism for tracking task execution and completion.