Hi all, I hope this isn’t documented somewhere that I missed… I have a Python package that contains my tasks. If I build a docker image which contains all the dependencies of my tasks, I can use the

pyflyte package

+

flytectl register

flow to push the tasks to my flyte cluster. The tasks run fine, and the image does not need to include the actual task code. Now for reusability, I’d like to break my task package into several modules and define a workflow in a different module, thus I need to import the tasks in the workflow module. Is there a way to make this work with the protobuf registration? I’m getting errors that the task module could not be found, which makes sense since the image doesn’t contain that code. It’d be nice not to have to rebuild my docker image every time a workflow/task changes, since most of the “business logic” code (upon which the tasks depend) is fairly stable and doesn’t require updates in the image too often

Try pyflyte register. it will copy your code to s3, and task will download it when running.

Thanks! The getting started guide recommends

pyflyte package

+

flytectl register

for production: https://docs.flyte.org/projects/cookbook/en/latest/getting_started/package_register.html#productionizing-your-workflows which is why I was trying to go that route

In production, the best practice is put the code in the image, so the task doesn’t need extra time to download the code. pyflyte run and pyflyte register both will copy the code to s3, so you don’t need to rebuild the image. It makes development and iteration much easier

Got it, that makes sense!

no problem!