https://flyte.org
Join Slack
I'm new to flyte and am planning to migrate code for a few different projects onto flyte. There are ...
# flyte-support
p
I'm new to flyte and am planning to migrate code for a few different projects onto flyte. There are a few different workflows, but i want them to share some tasks. Is there a way to set up my flyte project to make this possible? I'm planning to make a common/ folder for the shared tasks, and then project1/, project2/, etc. I'm assuming that the entire directory would be packaged and registered when I want to execute a run. Which folders would I run pyflyte init and pyflyte create in?
🚀 3
f
you do not need to run pyflyte init if you dont want to
if you want multiple workflows to share tasks and the code to be co-located then
Copy code
src/ 
   __init__.py
   proj1/
       __init__.py
   proj2/
       __init__.py
   common/
       __init__.py
And then you can register in
Copy code
root/src> pyflyte register proj1 common
and then proj2 .. Another option is to use https://docs.flyte.org/en/latest/user_guide/productionizing/reference_tasks.html#reference-tasks
p
Thats super helpful thanks! Is pyflyte create a required command? Also to use a reference_task, I would just write the function definition and leave the function itself empty?
f
Yup
You can also use flyteremote.fetch to retrieve the entity and not even create a rep
👍 1
Requirements though is that you will need network access
👍 1
4 Views
Open in Slack
PreviousNext
Powered by

Flyte

Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Powered by