Hey Flyte Gang :wave: I am currently trying to ena...
# ask-the-community
j
Hey Flyte Gang 👋 I am currently trying to enable file sensors for our multi cluster setup but i keep running into the following flytepropeller errors while executing this sample wf:
Copy code
from flytekit.sensor.file_sensor import FileSensor
from flytekit import task, workflow

sensor = FileSensor(name="test_sensor")
@task()
def t1():
    print("flyte")

@workflow
def wf():
    sensor(path="<s3://somebucket/sensortest/anotherdir>") >> t1()

if __name__ == "__main__":
    wf()
propeller ouputs a bunch of:
"No plugin found for Handler-type [sensor], defaulting to [container]"
"Runtime error from plugin [container]. Error: [BadTaskSpecification] invalid TaskSpecification, unable to determine Pod configuration"
Configmap for propeller is in this thread Any clue what i am missing?
Copy code
"enabled_plugins.yaml": "tasks:
		  task-plugins:
		    default-for-task-types:
		      container: container
		      container_array: k8s-array
		      sensor: agent-service
		      sidecar: sidecar
		    enabled-plugins:
		    - container
		    - sidecar
		    - k8s-array
		    - agent-service
		",
	"k8s.yaml": "plugins:
		  k8s:
		    default-cpus: 100m
		    default-env-vars: []
		    default-memory: 100Mi
		    agent-service:
		      defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
		      supportedTaskTypes:
		        - sensor
		",
s
@L godlike, looping you in since you've worked with agents. 🙂 Any idea what the issue might be?
please use the latest flytekit, not the released one
Did you restart the deployment after modified the config for sensor?
kubectl rollout restart deployment flyte-sandbox -n flyte
j
I got it working now! Thank you - both of you ❤️ I was in yaml hell, my propeller config was still wrong. This is how it should look like:
Copy code
enabled_plugins.yaml: |
    tasks:
      task-plugins:
        default-for-task-types:
          container: container
          container_array: k8s-array
          sensor: agent-service
          sidecar: sidecar
        enabled-plugins:
        - container
        - sidecar
        - k8s-array
        - agent-service
    plugins:
      agent-service:
        defaultGrpcEndpoint: flyteagent.flyte.svc.cluster.local:8000
        supportedTaskTypes:
        - sensor
  k8s.yaml: |
    plugins:
      k8s:
        default-cpus: 100m
        default-env-vars: []
        default-memory: 100Mi
I think there is no configuration needed in k8s.yaml
If this is the case, the values.yaml of flyte-core is a little misleading no?
l
Can you specify the lines? If it is, I can create a PR to fix it!
Thanks a lot
j
l
Thanks a lot, I will discuss it with Kevin and try to make it less misleading. Will update the PR here.
@Kevin Su Can you take a look here? I can help create a PR