Rahul Mehta
04/14/2023, 6:22 PMDan Rammer (hamersaw)
04/14/2023, 7:27 PM@task
decorator on a python function / ContainerTask
/ PodTask
/ etc. When you're executing a dynamic task as a maptask it isn't executing the same as how it is as just a regular task where it compiles a sub DAG, etc. It is actually executing everything as a single function call, so tasks inside the dynamic are just python function calls.Rahul Mehta
04/14/2023, 7:27 PMDan Rammer (hamersaw)
04/14/2023, 7:28 PMArrayNode
which will change how maptasks are executed internally - heres the RFC and I have very minimal working implementation right now.Rahul Mehta
04/14/2023, 7:32 PMDan Rammer (hamersaw)
04/14/2023, 7:37 PMKetan (kumare3)
@workflow
def sub_wf(i: int):
...
@dynamic
def foo(l: List[int]):
for i in l:
sub_wf(i=i)
@workflow
def wf(l: List[int]):
foo(l=l)
Rahul Mehta
04/16/2023, 12:12 AMKetan (kumare3)
Niels Bantilan
04/17/2023, 6:28 PMis there a sane way to support this use case right now?Just to follow up on this, there’s currently no way to support running
map_task
over anything other than a @task
… Ketan’s suggestion would be the only way to do this.Dan Rammer (hamersaw)
04/17/2023, 6:42 PM