Can you dynamically load flyte tasks to run inside...
# ask-the-community
l
Can you dynamically load flyte tasks to run inside of a dynamic task? e.g. if I want a task to act as a wrapper
Copy code
@dynamic
def rerun():
    relative_func = "flyte.tasks.example_task"
    relative_mod, input_func = relative_func.rsplit(".", 1)
    import_mod = importlib.import_module(relative_mod)
    rerun_task = getattr(import_mod, input_func)
    inputs = [SOME STUFF]
    map_task(rerun_task, concurrency=8)(input=inputs)
getting errors like
Copy code
File "/usr/local/lib/python3.9/site-packages/flytekit/core/tracker.py", line 229, in extract_task_module
        name = f.lhs
      File "/usr/local/lib/python3.9/site-packages/flytekit/core/tracker.py", line 70, in lhs
        return self.find_lhs()
      File "/usr/local/lib/python3.9/site-packages/flytekit/core/tracker.py", line 96, in find_lhs
        raise _system_exceptions.FlyteSystemException(f"Error looking for LHS in {self._instantiated_in}")

Message:

    Error looking for LHS in __main__
k
I think it’s possible, might need some changes in flytekit. why don’t import task directly, like
Copy code
from relative_mod import input_func
l
bc the task is a user import so I wouldn't know what it is until it runs
k
ohh, I see, module name will be the task input. mind create an issue here, I think we need make some small changes in flytekit. [flyte-bug]
154 Views