https://flyte.org logo
Powered by
Title
e

ewam

02/06/2023, 8:40 AM
@Samhita Alla in the video you linked, they briefly say you could implement "periodic triggers" using 
sleep
(attached screenshot). But there is no example given, unfortunately. Therefore, I'm asking here again: How would you use 
sleep
to trigger a task periodically inside a `@dynamic`workflow and how would the DAG look like in that case?
s

Samhita Alla

02/06/2023, 9:07 AM
Here's an example: 
from datetime import timedelta

from flytekit import dynamic, task
from flytekit.core.gate import sleep


@task
def t1(a: int):
    ...


@dynamic
def wf_sleep(n: int):
    for i in range(n):
        x = sleep(timedelta(minutes=10))
        b = t1(a=5)
        x >> b
t1
task will be triggered 
n
number of times with a fixed interval of 10 minutes between every two triggers.
e

ewam

02/06/2023, 9:26 AM
Thanks!
My misunderstandings come from the fact that I expected 
@dynamic
workflows to be "real" dynamic. But unfortunately, they still require a known DAG. "Real" dynamic DAGs would be a nice addition to flyte imho
s

Samhita Alla

02/06/2023, 9:48 AM
What do you mean by real dynamic?
e

ewam

02/06/2023, 11:14 AM
from datetime import timedelta

from flytekit import dynamic, task
from flytekit.core.gate import sleep


@task
def t1(a: int):
    ...


@dynamic
def wf_sleep():
    while True:
        x = sleep(timedelta(minutes=10))
        b = t1(a=5)
        x >> b
i.e. not just DAG support but also loop support
above 
dynamic
workflow cannot be evaluated pre runtime (while True)
but as I understand it, this is a requirement
what I need is truly event-driven task pipelines I guess
s

Samhita Alla

02/06/2023, 1:02 PM
Yeah. 
while True
won't work. There should exist some sort of determinism.
n

Niels Bantilan

02/06/2023, 2:48 PM
Hi @ewam can you expand on what you mean by “real dynamic”? Might be useful to write psuedo-code in plain Python to show the desired behavior
k

Ketan (kumare3)

02/06/2023, 3:57 PM
@Niels Bantilan his code indicates that he wants to interact with the outputs of a task execution in python and then launch another task. This is what we have been calling eager or async/await
#ask-the-communityJoin Slack
Powered by