I have another question. Let's say I want to creat...
# ask-the-community
I have another question. Let's say I want to create a plugin that is to execute Go tasks (it might be already there, i am still exploring Flyte). I am trying to figure out the changes i will have to make to realize this. any reference would be helpful.
there is no plugin to create go tasks - that is an SDK
it should be easy as the entire system is in go and hence writing go plugins should be easier as most of the machinery exists
flytectl is like a starting point of such a system
The easier way might be bundling the go code into binary and to docker image, and then launch the task with that image?
Yes @Byron Hsu. I had that as the first option. however, i was wondering about providing support like a python task. Thanks a lot @Ketan (kumare3). 🙂 I will look into flytectl.
yes, we would love to get a gosdk
and happy to collaborate
the beauty of gosdk might be super small images and fast runtime
we could also do some other optimizations
So basically @Saravanan Arumugam here is what you need to build a task acceptor - 1. a CLI based invocation system - you control this when you generate the task definition. So in python we generate something like
pyflyte-execute …
2. It will take a path to blob store for inputs and a prefix to put the outputs Read this - https://docs.flyte.org/en/latest/concepts/data_management.html#types-of-data and Read - https://pkg.go.dev/github.com/flyteorg/flyteplugins@v1.0.29/go/tasks/pluginmachinery/core/template#pkg-overview Also better than flytectl - copilot already implements some of this - https://pkg.go.dev/github.com/lyft/flyteplugins/copilot
awesome. 🙏 Thank you for the pointers @Ketan (kumare3). 🙂
we have been wanting to write a gosdk one-day
and eventually have a rust core data layer that all of them will reuse
Flyte will dominate the world
haha - will not mind it 🙂, but we just want to share a different way of thinkin