Hello all I m having an issue when trying to use reference t Flyte #flyte-support

Hello all I'm having an issue when trying to use r...

dry-ability-69144

03/29/2023, 12:15 PM

Hello all I'm having an issue when trying to use reference tasks... When building image, I'm getting this error:

'ReferenceTask' object has no attribute '_task_function'

... Can someone help me? I would be very appreciated

dry-ability-69144

03/29/2023, 12:16 PM

I'm using some tasks to test This is an example:

Copy code

from flytekit import task

@task
def task_example(n: int) -> int:
    """
    Simple example task for library building.
    Parameters:
        n (int): name of the parameter for the task will be derived from the name of the input variable
            the type will be automatically deduced to be Types.Integer

    Return:
        int: The label for the output will be automatically assigned and type will be deduced from the annotation

    """

    return n * n

And I'm referencing like this:

Copy code

from flytekit import reference_task

@reference_task(
    project="flyte-tasks",
    domain="development",
    name="src.flyte_tasks.example_task.example_task.task_example",
)
def task_example(n: int)->int:
    ...

freezing-airport-6809

03/29/2023, 1:05 PM

Seems odd - do you have bigger stack trace

dry-ability-69144

03/29/2023, 2:42 PM

I think we understood what was the problem We are using a custom class as return in one of the referenced tasks. But I thought that it wont be needed to recreate in other project, if we are using reference class...

dry-ability-69144

03/29/2023, 5:21 PM

Even when we added the custom class, the error keeps happening...

tall-lock-23197

03/30/2023, 5:18 AM

@dry-ability-69144, if you could share the actual code snippet and stack trace, that'd be helpful to find the source of the error.

dry-ability-69144

04/04/2023, 1:14 PM

Sorry it took so long It is happening in a colleague's task Here it is:

Copy code

Traceback (most recent call last):
  File "/home/jbudke/miniforge3/envs/dedup/bin/pyflyte", line 8, in <module>
    sys.exit(main())
  File "/home/jbudke/.local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/jbudke/.local/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/jbudke/.local/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jbudke/.local/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/jbudke/.local/lib/python3.8/site-packages/click/core.py", line 1651, in invoke
    cmd_name, cmd, args = self.resolve_command(ctx, args)
  File "/home/jbudke/.local/lib/python3.8/site-packages/click/core.py", line 1698, in resolve_command
    cmd = self.get_command(ctx, cmd_name)
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/clis/sdk_in_container/run.py", line 607, in get_command
    entity = load_naive_entity(module, exe_entity, project_root)
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/clis/sdk_in_container/run.py", line 447, in load_naive_entity
    importlib.import_module(module_name)
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 843, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/jbudke/Área de Trabalho/image-intelligence/dsc-image-intelligence/src/flyte_tasks/workflows/image_workflow.py", line 13, in <module>
    def image_workflow(
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/core/workflow.py", line 739, in workflow
    return wrapper(_workflow_function)
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/core/workflow.py", line 734, in wrapper
    workflow_instance.compile()
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/core/workflow.py", line 614, in compile
    workflow_outputs = exception_scopes.user_entry_point(self._workflow_function)(**input_kwargs)
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/exceptions/scopes.py", line 198, in user_entry_point
    return wrapped(*args, **kwargs)
  File "/home/jbudke/Área de Trabalho/image-intelligence/dsc-image-intelligence/src/flyte_tasks/workflows/image_workflow.py", line 41, in image_workflow
    all_samples = map_task(
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/core/map_task.py", line 274, in map_task
    return MapPythonTask(task_function, concurrency=concurrency, min_success_ratio=min_success_ratio, **kwargs)
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/core/tracker.py", line 35, in __call__
    o = super(InstanceTrackingMeta, cls).__call__(*args, **kwargs)
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/core/map_task.py", line 58, in __init__
    _, mod, f, _ = tracker.extract_task_module(python_function_task.task_function)
  File "/home/jbudke/miniforge3/envs/dedup/lib/python3.8/site-packages/flytekit/core/python_function_task.py", line 156, in task_function
    return self._task_function
AttributeError: 'ReferenceTask' object has no attribute '_task_function'

tall-lock-23197

04/04/2023, 1:34 PM

Are you using the referenced task in a map task?

tall-lock-23197

04/04/2023, 1:35 PM

Is it possible for you to share a redacted code snippet?

dry-ability-69144

04/04/2023, 1:35 PM

Yes, the idea is to use the referenced task as a mapped task due to the size of the dataset it handles

dry-ability-69144

04/04/2023, 1:36 PM

I'll try to update the code as soon as I can

dry-ability-69144

04/04/2023, 1:37 PM

But the general idea is to use the referenced task, which uses a model to do some image predictions in a map task for better performance

tall-lock-23197

04/04/2023, 4:57 PM

Looks like a bug. cc @high-accountant-32689 Code:

Copy code

from flytekit import map_task, reference_task, workflow


@reference_task(
    project="flytesnacks",
    domain="development",
    name="core.flyte_basics.decorating_tasks.t1",
    version="AdGs5FM0Sh2WFM0T6rZemg==",
)
def t1(x: int) -> int:
    ...


@workflow
def wf(x: list[int] = [3, 2]):
    return map_task(t1)(x=x)

Error:

Copy code

Failed with Unknown Exception <class 'AttributeError'> Reason: 'ReferenceTask' object has no attribute '_task_function'

high-accountant-32689

04/04/2023, 5:39 PM

I agree. @tall-lock-23197, can you open a gh issue?

tall-lock-23197

04/05/2023, 4:56 AM

https://github.com/flyteorg/flyte/issues/3573

freezing-airport-6809

04/05/2023, 6:00 AM

I do not think this is a bug, I think this is a feature request

freezing-airport-6809

04/05/2023, 6:00 AM

And this is a hard one

176 Views

Open in Slack

Previous Next