Is it possible to create a sensor plugin that monitors a K8s resource using a K8s client? For instance, the sensor monitors a K8s Job and once the Job status is

Succeeded

, it then triggers the downstream tasks

Yes, it's possible, you can implement it in

flyteplugins

and

flyteplugins

is a part of flytepropeller, which is our Kubernetes operator. https://github.com/flyteorg/flyte/tree/master/flyteplugins/go/tasks/plugins/webapi

Yes, that is possible, but before we go that route, can you talk a bit more about your use case? Usually we refrain from interacting with the k8s api via agents, and instead recommend users to use backend plugins (as @damp-lion-88352) suggested above.

@high-accountant-32689 our use case is that we have a custom flytekit plugin that applies a RayJob manifest to a namespace. We have Kueue enabled on our K8s cluster and so the RayJob is queued and doesn't start running immediately. We want a sensor that could monitor the status of the RayJob and once completed, it unblocks the downstream tasks that rely on the result of the RayJob