acoustic-carpenter-78188
03/02/2023, 10:17 PMflyteconsole==1.3.4 accessing workflow execution from FlyteConsole has become extremely slow for workflows that spawn a lot of nested dynamic tasks.
I think the newer flyteconsole tries to pull every task, node executions that current parent one has associated with in one request so if the workflow has a lot of dynamic tasks it slows down considerably.
i observed this before when trying to fetch execution using FlyteRemote and using sync operation to get all the node executions that the current execution has.
In flyteadmin i see this error
2022/11/07 17:36:05 /go/src/github.com/flyteorg/flyteadmin/pkg/repositories/gormimpl/task_execution_repo.go:121 SLOW SQL >= 200ms
[204.218ms] [rows:1] SELECT "task_executions"."id","task_executions"."created_at","task_executions"."updated_at","task_executions"."deleted_at","task_executions"."project","task_executions"."domain","task_executions"."name","task_executions"."version","task_executions"."execution_project","task_executions"."execution_domain","task_executions"."execution_name","task_executions"."node_id","task_executions"."retry_attempt","task_executions"."phase","task_executions"."phase_version","task_executions"."input_uri","task_executions"."closure","task_executions"."started_at","task_executions"."task_execution_created_at","task_executions"."task_execution_updated_at","task_executions"."duration" FROM "task_executions" LEFT JOIN tasks ON task_executions.project = tasks.project AND task_executions.domain = tasks.domain AND task_executions.name = tasks.name AND task_executions.version = tasks.version INNER JOIN node_executions ON task_executions.node_id = node_executions.node_id AND task_executions.execution_project = node_executions.execution_project AND task_executions.execution_domain = node_executions.execution_domain AND task_executions.execution_name = node_executions.execution_name INNER JOIN executions ON node_executions.execution_project = executions.execution_project AND node_executions.execution_domain = executions.execution_domain AND node_executions.execution_name = executions.execution_name WHERE executions.execution_project = 'balrog' AND executions.execution_domain = 'production' AND executions.execution_name = 'f54bcr4c4l3v5s' AND node_executions.node_id = 'n0' LIMIT 10000
I reverted the console back to version v1.2.5 and that does not seem to cause any issue. I believe this PR could have introduced the issue 9b10e5f
flyteorg/flyteconsoleacoustic-carpenter-78188
03/02/2023, 10:17 PM