Hi everybody, I am trying to understand the different integration options available in flyte. What exactly is the difference between a "flyte agent" and an "External Service Backend Plugin"? In the docs there is an agent documentation like the "Memory Machine Cloud" that executes flyte tasks externally. That could have been implemented as external plugin as well. Can somebody explain to me what the optimal use case for each option is and what the limitaions are? Thank you a lot for taking the time 🙌

Hi @Christoph Neumaier and welcome to the Flyte community. Well, I've had a similar question, so let me try to answer from what I know (others can jump and correct/expand). Every interaction that Flyte performs with any outside system (by outside I mean even Kubernetes) uses a plugin. Let's say you have a workflow and one of the tasks uses MMCloud, for example. With a regular plugin, a K8s Pod would be created for each task, so you'd have a Pod whose sole purpose is to execute the request to the MMCloud. Doesn't sound much efficient in terms of resource consumption and overall latency right? Also, the person/team who wrote that plugin has to be not only proficient in Go (not typical in MLOps/Data teams) but to know intimately how to integrate that plugin into Propeller, Flyte's execution engine. In the other hand, if you use, say, the MMCloud Agent, your Task will contact a "Connection pool" on a long-running service (one that could even run multiple different Agents concurrently) so your resource consumption footprint is significantly reduced and the overall performance could be better; so much so you can even have Synchronous Agents. And, finally, Agents are written in Python (potentially it could be other languages too as it uses protobuf for serialization). Does that help?

Hi @David Espejo (he/him) thank you for your response. Your explanation make sense in every way. Sounds like the agent approach is the more efficient one. Thank you for clarification.

something like Human in the Loop? For an async agent you can implement a

get

method but gating the execution on user input would require an additional feature https://docs.flyte.org/en/latest/user_guide/advanced_composition/waiting_for_external_inputs.html is this what you're looking for?

@Christoph Neumaier I want to add couple points quick external service backend plugins and agents are the same thing. We are deprecating external service backend plugins in favor of agents. Check out this blog https://www.union.ai/blog-post/flyte-agents-framework Now to answer second you can wait using wait for input tasks, but you will have to have a three step workflow. Step one check something with the external service, then run a task to wait for approval and then finally use the agent to do something on the external service. It is possible to offer the composition as a simple method for your users. And example is sagemaker deployment https://github.com/flyteorg/flytekit/blob/fa2aa0b215cd2fca3d90d8de92e41c6fadb88ed6/plugins/flytekit-aws-sagemaker/flytekitplugins/awssagemaker_inference/workflow.py#L35 What this does is creates a workflow that can be used in another workflow

@David Espejo (he/him) and @Ketan (kumare3) thank you both for the provided input. I will take a more in depth look at the supplied solutions from both of you and report back after i understood them completely.