nice-airport-86898
12/08/2023, 3:38 AMmy_functions.py
) and I cannot change it. That is, I can't go into it and decorate the functions with @task
. Presently I've got a file that imports the necessary functions, creates tasks, and then builds an ImperativeWorkflow
. The tasks and workflows are created as shown in `my_workflow.py`:
from flytekit import task
from flytekit.core.workflow import ImperativeWorkflow
from my_functions import func1
func1 = task(func1)
workflow = ImperativeWorkflow(name="my_workflow")
workflow.add_task(func1)
However, when I try to run with this, it appears that Flyte is going and finding my_functions.func1
and trying to run it directly. Except that it isn't a PythonTask
object since in the main source code func1
is not decorated and it just grabs the actual function itself.
Is there a way to handle this? Can I create Tasks from functions in code I can't modify/add @task
decorators to?freezing-airport-6809
nice-airport-86898
12/08/2023, 2:23 PMfreezing-airport-6809
freezing-airport-6809
nice-airport-86898
12/08/2023, 4:43 PMfunc1
inside my_workflow.py
as a task
wrapped version of func1
imported from my_functions.py
. However the default resolver finds my_functions.func1
instead of my_workflow.func1
. After looking at the resolver, it looks like I could hack together my own resolver that addresses this, but I feel like the better solution is the task aliasing -- if there's a good way to do that. I'm interested to see what you're able to figure out.nice-airport-86898
12/08/2023, 4:43 PMfreezing-airport-6809
nice-airport-86898
12/11/2023, 7:47 PMfreezing-airport-6809
from orig import hello
from flytekit import task, workflow
hello_task = task(hello, cache=True, cache_version="1")
@workflow
def wf() -> str:
return hello_task(s="world")
if __name__ == "__main__":
wf()
in orig.py
def hello(s: str) -> str:
return f"hello {s}"
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
pyflyte run --remote --copy-all refer_orig.py wf
freezing-airport-6809
AttributeError: 'function' object has no attribute 'name'
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
"flytekit.core.python_auto_container.default_task_resolver"
19:"--"
20:"task-module"
21:"orig"
22:"task-name"
23:"hello"
freezing-airport-6809
freezing-airport-6809
freezing-airport-6809
hello_task
freezing-airport-6809
task
nice-airport-86898
12/12/2023, 2:27 PMnice-airport-86898
12/12/2023, 2:29 PM@task
is a pretty big restriction. I see the solution found here as more of a work-around than an official method of using Flyte. Just food for thought as the design progresses. But thanks so much for helping me with this!freezing-airport-6809
freezing-airport-6809