Hey everyone! Has anyone used flyte for model serving? I'm new to flyte but trying to set up some workflows with ray serve. Has anyone experimented with this before?
Currently we do not have a type for infinite running tasks, so it may be hard
But this is an idea we have on the roadmap
You could proxy that by running with lots of retires and no timeout
b
Ben Rosand
03/28/2023, 5:39 PM
Seems like the banana workflow is more for separating serving from flyte, and just serving models trained with Flyte. I guess more broadly people aren’t using flyte with any kind of serving tools? But I’ll take a look at the proxy idea
If infinite running tasks aren’t specced in I guess serving would be specced in either