Hey! We have an internal Flytekit plugin for Ray tasks. The OSS Ray plugin did not satisfy our use cases and did not align with our organization's design. The internal Flytekit plugin uses our SDK to create a Ray Cluster on a separate GKE cluster from the one Flyte is deployed on. It then executes the job on the Ray cluster. And then deletes the Ray cluster once the job is finished. Therefore, all the pod on the Flyte GKE cluster does is manage the Ray cluster lifecycle. We are looking into Flyte Agents and we understand there's a performance gain when compared to the backend plugins, since no pods are created and the Agent can be autoscaled. Does our use case sound like a good candidate to test with Flyte Agents? Is there a reason why the OSS ray plugin is still a backend plugin and not an Agent?

Yes it is a good idea

@damp-lion-88352 do you mean you discussed an agent for the Ray backend plugin?

@dry-egg-91175 this is a great usecase for agents. perfect fit

@dry-egg-91175 Yes

Great! I'm still pretty new to agents so I have a question: We manage the cluster lifecycle through our plugin's

pre_execute

,

execute

and

post_execute

methods. How do these methods work with Agent's

create

,

get

, and

delete

? Should we instead (i) create the Ray cluster + run the job, (ii) get the job status, (iii) delete the Ray cluster in the Agent's (i)

create

, (ii)

get

, and (iii)

delete

methods?

I will discuss with Kevin and reply to you

Yes create should create and maybe even submit the job. You can submit the job in get too

I think databricks agent can be a good example! https://github.com/flyteorg/flytekit/blob/master/plugins/flytekit-spark/flytekitplugins/spark/agent.py

Thank you for the example. I believe it will be very useful if there's documentation on how a task's

pre_execute

,

execute

, and

post_execute

methods interact with an agent's

create

,

get

, and

delete

methods. If there's no pod created for tasks run through an agent, where are the

pre_execute

,

execute

,

post_execute

methods invoked? what's the sequence of invocation? Is there a recommended design pattern on dividing responsibilities between the task's methods and the agent's methods?

Will draw a diagram for you tomorrow

Thank you so much!

There's only

execute

will be invoked if there's no pod created for tasks run through an agent