Saravanan Arumugam
01/17/2023, 11:55 PMByron Hsu
01/18/2023, 4:26 AMSaravanan Arumugam
01/18/2023, 4:34 AMByron Hsu
01/18/2023, 4:38 AMDan Rammer (hamersaw)
01/18/2023, 5:04 AM@task
decorator, PodTasks, ContainerTasks. Additionally, there are separate plugins for Spark, Ray, Dask, Tensorflow, PyTorch, etc that use existing k8s operators. I this scheme Flyte creates the requisite k8s custom resource and the operator is tasked with execution, Flyte tracks the job status. And then finally there are web API plugins which Flyte does not create any k8s resources for, rather it interacts with an external service - for example snowflake, databricks, etc.Ketan (kumare3)
Byron Hsu
01/18/2023, 5:07 AMKetan (kumare3)
Byron Hsu
01/18/2023, 5:09 AMKetan (kumare3)
create
call.
While sync will expect a response instantenously. This will block Propeller loops themselvesByron Hsu
01/18/2023, 5:11 AMDan Rammer (hamersaw)
01/18/2023, 5:13 AMn1
, if it's complete then write it's results to cache (given it's a cached node), then schedule downstream node n2
, etc ... With the sync web API plugin checking the status of a node is a synchronous operation, so it requires a request RTT for each node check, these add up if there are many tasks making a single "round" of propeller large. Alternatively, the async web api plugin uses a background process in propeller to periodically check the task status, then in a propeller "round" each task status is stored in a cache and looked up in memory for extremely fast propeller "rounds".Byron Hsu
01/18/2023, 5:31 AMthis adds up if there are many tasksIs it the case that one parent task has many children tasks, so in each round it needs to check many task, and the sync API blocks the check function so it takes much time.
Dan Rammer (hamersaw)
01/18/2023, 2:11 PMIs it the case that one parent task has many children tasksIt could be, more so just instances where propeller is executing many tasks in parallel - obviously if there are 30 tasks running then propeller has to check the status of 30 tasks.
Byron Hsu
01/18/2023, 5:39 PM