<#5035 Timeout for Flyte file sensors> Issue creat...
# flytekit
c
#5035 Timeout for Flyte file sensors Issue created by shekhar-kotekar ### Motivation: Why do you think this is important? When we start running a workflow and use file sensor, this senor keeps waiting for the file infinitely. This becomes a problem in 2 scenarios. 1. If file does not arrive at all then workflow never expires. We have to manully stop the workflow execution. 2. We schedule the workflow to run at certain interval (say every 2 hours) - with the first execution of the workflow file sensor is waiting for the file to arrive. If file does not arrive within 2 hours then after 2 hours Flyte will schedule another execution of the workflow so both the executions will keep waiting for the same file which will possibly lead to duplicate execution of the workflow. This can lead to unpredictable, unwanted results like - cluster resource being wasted, data duplication, errors, etc. ### Goal: What should the final outcome look like, ideally? Ideally we should be able to specify timeouts for a file sensor like shown belo:
sensor = FileSensor(name="s3_file_sensor", timeout="2h")
We should be able to specify timeout in terms of minutes, hours, days, etc. Example - "5m" for 5 minutes, "3h" for 3 hours, etc. After the timeout workflow can fail so that users can take some action like restarting the workflow or some other action. ### Describe alternatives you've considered As of now we have set workflow execution CRON schedule to run every 12 hours. ### Propose: Link/Inline OR Additional context No response ### Are you sure this issue hasn't been raised already? • Yes ### Have you read the Code of Conduct? • Yes flyteorg/flyte