Saravanan Arumugam01/17/2023, 11:55 PM
Byron Hsu01/18/2023, 4:26 AM
Saravanan Arumugam01/18/2023, 4:34 AM
Byron Hsu01/18/2023, 4:38 AM
Dan Rammer (hamersaw)01/18/2023, 5:04 AM
decorator, PodTasks, ContainerTasks. Additionally, there are separate plugins for Spark, Ray, Dask, Tensorflow, PyTorch, etc that use existing k8s operators. I this scheme Flyte creates the requisite k8s custom resource and the operator is tasked with execution, Flyte tracks the job status. And then finally there are web API plugins which Flyte does not create any k8s resources for, rather it interacts with an external service - for example snowflake, databricks, etc.
Byron Hsu01/18/2023, 5:07 AM
Byron Hsu01/18/2023, 5:09 AM
call. While sync will expect a response instantenously. This will block Propeller loops themselves
Byron Hsu01/18/2023, 5:11 AM
Dan Rammer (hamersaw)01/18/2023, 5:13 AM
, if it's complete then write it's results to cache (given it's a cached node), then schedule downstream node
, etc ... With the sync web API plugin checking the status of a node is a synchronous operation, so it requires a request RTT for each node check, these add up if there are many tasks making a single "round" of propeller large. Alternatively, the async web api plugin uses a background process in propeller to periodically check the task status, then in a propeller "round" each task status is stored in a cache and looked up in memory for extremely fast propeller "rounds".
Byron Hsu01/18/2023, 5:31 AM
this adds up if there are many tasksIs it the case that one parent task has many children tasks, so in each round it needs to check many task, and the sync API blocks the check function so it takes much time.
Dan Rammer (hamersaw)01/18/2023, 2:11 PM
Is it the case that one parent task has many children tasksIt could be, more so just instances where propeller is executing many tasks in parallel - obviously if there are 30 tasks running then propeller has to check the status of 30 tasks.
Byron Hsu01/18/2023, 5:39 PM