🧵 Flyte agent for managing K8s resources. I can see how flyte agent can simplify the dev for a custom task and plugin. I was planning to implement flyte backend plugin, which orchestrates different K8s resources, (such as create StatefulSet, and MPIJob) which requires the new IDL defined with the protobuf, flytebackend plugin implementation and flytekit changes. Then someone pointed me the flyte agent which I can potentially implement the agent, which accepts gRPC call to manage the K8s resources such as StatefulSet and MPIJob in the agent implementation. 1. Has anyone tried on the similar use cases? 2. Are there Gotchas or nasty/confusing behavior it might be involved? 3. since I need to CRUD the K8s resources, python K8s client is not that optimized for API read/write, so wondering is it a bad idea to use agent to do such K8s work? 4. the task definition in agent seems to be simple one, if I need complex data structures/API to define the task, is flyte agent still the good way to go or we should use flyte backend plugin with proper protobuf definition? Thread in Slack Conversation