With the way packaging and registration works with flyte, what is the recommended way to provide a common set of library methods across multiple flyte projects? (eg. our team wants to define a CustomTask that multiple projects want to use). Is there any example of this we can reference?
07/06/2022, 5:20 PM
you can use reference tasks to share pre-registered task definitions across projects and domains
07/06/2022, 5:26 PM
Ah ok, good to know, thanks!
I guess I am still curious about sharing more general purpose python utilities (not necessarily tasks). Is there any precedent for this?
Eg. say I want to provide a generalized setup_teardown, that enables experiment tracking on an external service and get multiple projects to use this function.
07/06/2022, 5:31 PM
could you build the utilities into a library and distribute that within your organization? if you're using the fast register pattern you can even bake that library dependency into your organization's base image
07/06/2022, 5:37 PM
Ok, I see. Thanks we can look into those options.
07/06/2022, 11:27 PM
Ya nothing stops regular python libraries, just reference task is further removing the dependence