I want my_sensor poke every 5s.For example, "sensor >> task1". The sensor catch the condition to execute the task1, and the task1 will finish, the sensor is not run again. How to write task to rerun sensor infinity. And the task1 is run only when catch condition, the sensor still run after that. Should i have to write the My_Sensor which recusive callback it's all sensor infiniy. Anyone have the example for this. I am reading airflow sensor to reference this
10/09/2023, 1:52 PM
Infinity is interesting- how many times do you want to run the task
10/09/2023, 7:57 PM
what kind of sensor do you want to run infinity?
does airflow support infinity?
10/11/2023, 1:14 AM
@Ketan (kumare3)@Kevin Su,How about: check the folder, if folder have new file >> do the task.
Airflow don't support infinity sensor too, it just support more of option of sensor such as number of tries, interval time each poke . Basically, It can be understand sensor is the node task which not know when will have to execution. I'm asking about the weird kind of sensor, because i want to write the dags which contain sensor is dependent( for example, check new file in the folder, and the target only i interest is the folder will have new file or not).
10/11/2023, 3:04 AM
@Ryuu we are working on reactive workflows which model this better
10/11/2023, 3:20 AM
I don't clearly understand the term 'reactive workflows' at this time, i will try to improve my knowledge. thanks a lots for your help
10/11/2023, 3:43 AM
That’s ok - what I mean is a workflow that runs as a reaction to some event like a file appearing