Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

I have another question. Let's say I want to create a plugin that is to execute Go tasks (it might be already there, i am still exploring Flyte). I am trying to figure out the changes i will have to make to realize this. any reference would be helpful.

there is no plugin to create go tasks - that is an SDK

it should be easy as the entire system is in go and hence writing go plugins should be easier as most of the machinery exists

flytectl is like a starting point of such a system

The easier way might be bundling the go code into binary and to docker image,  and then launch the task with that image?

Yes <@U042Z2S8268>. I had that as the first option. however, i was wondering about providing support like a python task.

Thanks a lot <@UNZB4NW3S>. :slightly_smiling_face: I will look into flytectl.

the beauty of gosdk might be super small images and fast runtime

we could also do some other optimizations

So basically <@U04JPEYTW2U> here is what you need to build a task acceptor -  1.  a CLI based invocation system - you control this when you generate the task definition. So in python we generate something like `pyflyte-execute …`
2. It will take a path to blob store for inputs and a prefix to put the outputs
Read this - <https://docs.flyte.org/en/latest/concepts/data_management.html#types-of-data>
and
Read - <https://pkg.go.dev/github.com/flyteorg/flyteplugins@v1.0.29/go/tasks/pluginmachinery/core/template#pkg-overview>

Also better than flytectl - copilot already implements some of this - <https://pkg.go.dev/github.com/lyft/flyteplugins/copilot>

awesome. :pray: Thank you for the pointers <@UNZB4NW3S>. :slightly_smiling_face:

we have been wanting to write a gosdk one-day

and eventually have a rust core data layer that all of them will reuse

haha - will not mind it :slightly_smiling_face:, but we just want to share a different way of thinkin