Hi all a flyte newbie here I want to write a workflow that r

Question

Hi all a flyte newbie here I want to write a workflow that runs the same task sequentially in a workflow Any thoughts pointers to examples as to how I can do this in Flyte For example here is a dummy case where I want to run the task add 1 sequentially currently flute would run these simultaneously As there is no dependency between 2 calls

Visak · Answer

``` task Def add 1 n float gt float return n + 1 workflow Def some name I want these two steps to run sequentially instead of parallel c1 = add 1 1 0 c2 = add 1 2 0 ```

Jay Ganbat · Answer

you can use this to create dependency <https docs flyte org projects flytekit en latest generated flytekit core node creation create node html flytekit core node creation create node>

Jay Ganbat · Answer

or you can use the chaining <https docs flyte org projects cookbook en latest auto core control flow chain entities html sphx glr auto core control flow chain entities py>

Jay Ganbat · Answer

chaining tasks might be more appropriate here

Visak · Answer

Thanks ill look into these An additional question that comes to my mind is if chaining would be possible in a loop For example how could I create dependency or chain in this case ``` workflow Def some name I want these two steps to run sequentially instead of parallel for item in list add 1 item ```

Jay Ganbat · Answer

Interesting idea it might not be possible to use list inside workflow does `list` has a predefined values or will it be determined at runtime because workflow is evaluated during registration You might be able to do that in dynamic task < Ketan kumare3 > any insight sweat smile

Visak · Answer

The list can be predefined this case Im more curious about if we can introduce chaining dependency on subsequent calls of the same `task` in a loop

Jay Ganbat · Answer

i wonder if this works laughing ```result = SOME VALUE for item in list result = add 1 item result ``` never though about this before

Visak · Answer

sweat smile can give it a shot my hunch is that since flyte is an acyclic graph this closed loop dependency will be not possible unless there is some way to wait sleep until a task finishes

varsha Parthasarathy · Answer

Does wait for input help here < Jay Ganbat> or < Ketan kumare3 >

varsha Parthasarathy · Answer

I haven t used this construct before but very curious to understand smile

Ketan (kumare3) · Answer

< Visak> < Jay Ganbat> as long as the list is predefined chaining should work just need to create node or use promise chaning capture it in a local Variable I will try to write an example later If you want list to be dynamic then you have to use dynamic workflows

Ketan (kumare3) · Answer

Wait for input does not provide inputs on which you can iterate in a workflow you only get a promise You can materialize the promise in a task and then run it in dynamic

Visak · Answer

Thanks Yes an example would be super helpful here

varsha Parthasarathy · Answer

```x will be different for every run for i in range x splits = split inputs inputs=input tasks s=i r = flytekit map task run maptask fpo concurrency=50 min success ratio=0 9 curation input=splits result append r ``` < Ketan kumare3 > I want flytekit map task to be launched once wait for r append to result and again call flytekit map task for x times But currently all map tasks are launched x times all at once

Ketan (kumare3) · Answer

hmm then the code should look different right

Ketan (kumare3) · Answer

that is implicit dependency

Ketan (kumare3) · Answer

Crazily this works ```from flytekit import task workflow map task dynamic import typing task def square i int gt int return i i task def splits i int gt typing List int return list range i task def join new v typing List int v typing List int gt typing List int v extend new v return v dynamic def do i int x int v typing List int gt typing List int s = splits i=i new v = map task square i=s v = join new v=new v v=v if i == x 1 return v return do i=i+1 x=x v=v workflow def wf x int do i=1 x=x v= ```

Ketan (kumare3) · Answer

cc < varsha Parthasarathy>

Ketan (kumare3) · Answer

cc < Niels Bantilan> < Eduardo Apolinario eapolinario > < Yee> And interesting i got it right in one go in vim smile hmmm functional programming < varsha Parthasarathy> at this point what you need is async workflows Something that we are working specing now

Ketan (kumare3) · Answer

Screenshot 2023 03 21 at 10 09 52 PM png

Ketan (kumare3) · Answer

Screenshot 2023 03 21 at 10 10 11 PM png

Jay Ganbat · Answer

Whoa that is crazy but awesome rolling on the floor laughing

Visak · Answer

This is pretty dope thanks

Ketan (kumare3) · Answer

Haha but not very performant

Ketan (kumare3) · Answer

Also cc < James Sutton> and < Evan Sadler>