acoustic-carpenter-78188
10/11/2023, 4:25 PMpyflyte run --hybrid module.py my_wf
and it should send certain tasks to the Flyte remote instance.
If --hybrid is not specified then all tasks will be run locally
if --remote is specified, then all tasks with workflow will be run remotely.
This will also lend well into @eager workflows as they become more rampant.
This relates to this issue flyteorg/flytekit#1833
IMO the solution proposed here --hybrid could be applied to eager as well!
cc @cosmicBboy
Describe alternatives you've considered
NA
Propose: Link/Inline OR Additional context
Recommendation introduce a new decorator @remote which can be added to any task
@task
@remote
def foo1(i: int) -> int:
...
@task
def foo2(i: int) -> int:
...
@workflow
def wf(i: int):
j = foo1(i=i)
foo2(i=j)
Now if we invoke
pyflyte run wf.py wf -> no deviation from current behavior
pyflyte run --remote wf.py wf -> no deviation from current behavior
pyflyte run --hybrid wf.py wf -> only task marked with @remote decorator is invoked remotely. in the above example foo1
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyte