Hello, im trying to explore developing custom agen...
# ask-the-community
l
Hello, im trying to explore developing custom agent service and was following the docs , have a couple of questions in 🧵 for better understanding.
• All tasks will run in the default agent unless specified (eg canary deployment doc, so the propeller knows how to map the task to k8 svc). Do Flyte only deploy the default agent, and additional agent would have to be deployed separately? • Specific to the BQ agent. Suppose you have a query that requires lots of resource, if its on-demand, you can specify the k8 resource fitting to the query, have the python bq client, and run use the bqclient directly in task. However having a generic BQ agent, it would have to be configured to high resource while being idle when not in used. Its harder to manage the resource to set on the agent since all bq task will run here and its not scalable. Am I missing anything here? • Comparing agent to flytekit extension. Since the agent is long-running and it shouldn't be running any computational heavy task, but rather calling external service - I don't grasp the benefit compared to flytekit extension besides not having to wait for a pod to spin up. But granted that there are more interface to implement (AgentBase, AgentExecutorMixin) and more deployment steps (new agent image, redeployment of agent and flytepropeller along with the new configmap mappings), it seems like a lot more complicated.
k
I would love to understand and example that needs high resources for BQ - isn’t it just a client with all processing in the BQ service itself
l
That is if the SQL query is doing all the data processing. For example, for batch scoring or data processing/exploration, the data is loaded from BQ, processed within flyte (using python libraries) before persisting back into BQ. Actually im least concern with this as compared to understanding how to leverage agent service for use-cases (especially compared to flytekit extension), trying to find if there are new use-cases that we can solve with this
k
Agent should be use when you want to write integrations with any service like vertex. Ai and then let everyone else simply use it. Checkout this blog https://www.union.ai/blog-post/deploy-segment-anything-model-sam-for-inference-on-amazon-sagemaker