Hey, I am currently working on using the `FileSens...
# ask-the-community
y
Hey, I am currently working on using the
FileSensor
to verify the existence of multiple files before triggering workflows. While I have successfully used the FileSensor to check for single file, I am facing challenges with extending this functionality to handle a list of file paths. Here’s what I’ve tried so far: 1. Subclassing FileSensor: ◦ Approach: I created a subclass of FileSensor and overrode the poke function to handle a list of paths. ◦ Issue: I encountered an error stating
failed to get sensor task with error: No module named 'src'
, despite not having any imports from module named
src.
◦ Additional Context: Importing the
FileSensor
from
flytekit.sensor.file_sensor
works fine for checking a single file. However, when I tried copying the
FileSensor
class directly into my code, I received the same error:
No module named 'src'
. 2. Looping in Tasks: ◦ Approach: I attempted to loop over the list of paths within a task, invoking the sensor for each path individually. ◦ Issue: This approach resulted in a
PermissionError: Forbidden
when calling the sensor from inside a task. ◦ Additional Context: Adding the
@task
to a worked FileSensor resulted in a
PermissionError: Forbidden
Could you provide some guidance on how to effectively check multiple file paths using FileSensor? Thanks in advance!
s
Approach: I created a subclass of FileSensor and overrode the poke function to handle a list of paths.
instead of subclassing a file sensor, how about you create a new sensor altogether? not sure why
no module
error is popping up though.
when calling the sensor from inside a task.
i don't think a sensor should be triggered from within a task.