What is the recommended way to share curated flows...
# ask-the-community
f
What is the recommended way to share curated flows / tasks within an organisation? To add more perspective: In kubeflow we built a bunch of premade components (basically a bit of code + container) that we could load into any pipeline and use in any pipeline. Does something like this exist in Flyte? Best case still executable locally?
we are also developing something cool, happy to talk privately
f
Hmm, but then the pipeline can no longer be executed locally, correct? Mocking crucial steps is not a useful solution, since some of the steps we plan to share (such as trainings) make the pipeline pointless if those were to be mocked. What about this section here?
Copy code
Flytekit-Only Task Plugin
Flytekit is designed to be extremely extensible. You can add new task-types that are useful only for your use-case. Flyte does come with the capability of extending the backend, but that is only required if you want the capability to be extended to all users of Flyte, or there is a cost/visibility benefit of doing so.
Writing your own Flytekit plugin is simple and is typically where you want to start when enabling custom task functionality.
https://docs.flyte.org/projects/cookbook/en/latest/auto/core/extend_flyte/index.html#flytekit-only-task-plugin However, we could not find a clear example on how to write such a flytekit-only extension.
k
f
Awesome, thanks for sharing! 🙏
k
@Ferdinand von den Eichen if you think something can benefit the community do document or share
108 Views