<#4205 [Core feature] pyflyte run hybrid mode> Iss...
# flyte-github
a
#4205 [Core feature] pyflyte run hybrid mode Issue created by kumare3 Motivation: Why do you think this is important? There are times when it would be nice to execute some tasks remotely, while keeping other tasks local. With Flyte Agents we have already enabled this. Flyte agents have the capability of calling external services as long as "secrets" are configured correctly. I opine that this should be available for all task types. With Single task execution and Flyte remote this would be possible. This issue also proposes a UX for the same. Goal: What should the final outcome look like, ideally? A user could use
pyflyte run --hybrid module.py my_wf
and it should send certain tasks to the Flyte remote instance. If
--hybrid
is not specified then all tasks will be run locally if
--remote
is specified, then all tasks with workflow will be run remotely. This will also lend well into
@eager
workflows as they become more rampant. This relates to this issue flyteorg/flytekit#1833 IMO the solution proposed here
--hybrid
could be applied to eager as well! cc @cosmicBboy Describe alternatives you've considered NA Propose: Link/Inline OR Additional context Recommendation introduce a new decorator
@remote
which can be added to any task
Copy code
@task 
@remote 
def foo1(i: int) -> int:
   ...

@task 
def foo2(i: int) -> int:
   ...

@workflow
def wf(i: int):
   j = foo1(i=i)
   foo2(i=j)
Now if we invoke
pyflyte run wf.py wf
-> no deviation from current behavior
pyflyte run --remote wf.py wf
-> no deviation from current behavior
pyflyte run --hybrid wf.py wf
-> only task marked with
@remote
decorator is invoked remotely. in the above example
foo1
Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte