I want my_sensor poke every 5s.For example, "senso...
# flyte-supportf
full-evening-87657
10/09/2023, 11:23 AMI want my_sensor poke every 5s.For example, "sensor >> task1". The sensor catch the condition to execute the task1, and the task1 will finish, the sensor is not run again. How to write task to rerun sensor infinity. And the task1 is run only when catch condition, the sensor still run after that. Should i have to write the My_Sensor which recusive callback it's all sensor infiniy. Anyone have the example for this. I am reading airflow sensor to reference this
f
freezing-airport-6809
10/09/2023, 1:52 PM
Infinity is interesting- how many times do you want to run the task
g
glamorous-carpet-83516
10/09/2023, 7:57 PMwhat kind of sensor do you want to run infinity?
glamorous-carpet-83516
10/09/2023, 7:58 PMdoes airflow support infinity?
f
full-evening-87657
10/11/2023, 1:14 AM@freezing-airport-6809 @glamorous-carpet-83516,How about: check the folder, if folder have new file >> do the task.
full-evening-87657
10/11/2023, 1:37 AMAirflow don't support infinity sensor too, it just support more of option of sensor such as number of tries, interval time each poke . Basically, It can be understand sensor is the node task which not know when will have to execution. I'm asking about the weird kind of sensor, because i want to write the dags which contain sensor is dependent( for example, check new file in the folder, and the target only i interest is the folder will have new file or not).
f
freezing-airport-6809
10/11/2023, 3:04 AM
@full-evening-87657 we are working on reactive workflows which model this better
f
full-evening-87657
10/11/2023, 3:20 AMI don't clearly understand the term 'reactive workflows' at this time, i will try to improve my knowledge. thanks a lots for your help
f
freezing-airport-6809
10/11/2023, 3:43 AM
That’s ok - what I mean is a workflow that runs as a reaction to some event like a file appearing
2 Views