Hi everyone, I have a situation where I have some really lightweight tasks—tasks that only send HTTP requests or process simple strings. Wouldn't deploying a pod on Kubernetes for this be excessive? It takes several seconds to deploy a pod, but the logic inside the pod takes only a few milliseconds. Does Flyte offer a solution for this? Thank you.
Yes, what @damp-lion-88352 said. That is exactly what you need. (Look for the synchronous version). If you’re performance bound and don’t even want to suffer the jump to the agent pod you can try a golang based webapi plugin, but I think that’s too cumbersome. Agent is the best choice.
❤️ 1
f
freezing-airport-6809
09/09/2024, 2:11 PM
We are also thinking h of writing a generic http agent to call http endpoints - cc @high-park-82026
freezing-airport-6809
09/10/2024, 1:50 PM
@loud-barista-15046 if you have requirements around http would love to understand
freezing-airport-6809
09/10/2024, 1:51 PM
And as a last thing on union we have a way to reuse containers, so workloads are just assigned to the same container, greatly reducing the overhead