https://flyte.org logo
#ask-the-community
Title
# ask-the-community
b

Bryan Weber

02/15/2023, 8:31 PM
Hi all, I hope this isn’t documented somewhere that I missed… I have a Python package that contains my tasks. If I build a docker image which contains all the dependencies of my tasks, I can use the
pyflyte package
+
flytectl register
flow to push the tasks to my flyte cluster. The tasks run fine, and the image does not need to include the actual task code. Now for reusability, I’d like to break my task package into several modules and define a workflow in a different module, thus I need to import the tasks in the workflow module. Is there a way to make this work with the protobuf registration? I’m getting errors that the task module could not be found, which makes sense since the image doesn’t contain that code. It’d be nice not to have to rebuild my docker image every time a workflow/task changes, since most of the “business logic” code (upon which the tasks depend) is fairly stable and doesn’t require updates in the image too often
k

Kevin Su

02/15/2023, 9:04 PM
Try pyflyte register. it will copy your code to s3, and task will download it when running.
b

Bryan Weber

02/15/2023, 9:05 PM
Thanks! The getting started guide recommends
pyflyte package
+
flytectl register
for production: https://docs.flyte.org/projects/cookbook/en/latest/getting_started/package_register.html#productionizing-your-workflows which is why I was trying to go that route
k

Kevin Su

02/15/2023, 9:09 PM
In production, the best practice is put the code in the image, so the task doesn’t need extra time to download the code. pyflyte run and pyflyte register both will copy the code to s3, so you don’t need to rebuild the image. It makes development and iteration much easier
b

Bryan Weber

02/15/2023, 9:10 PM
Got it, that makes sense!
Thanks for the quick response!
k

Kevin Su

02/15/2023, 9:11 PM
no problem!
67 Views