Hey I m using `with overrides node name =f variable some tas Flyte #ask-the-community

Hey! I’m using `with_overrides(_node_name_=f"{vari...

Geert

07/10/2023, 2:07 PM

Hey! I’m using

with_overrides(_node_name_=f"{variable}-some-task")

to set a less cryptic name for my dynamic task (to have

{variable}-some-task

show in the UI instead of

d1-n-dn0

), and that works on first glance, but they get changed when the workload starts running; and stay that way after completion. Is this a bug or expected behaviour? Any way to get the name to stay?

Yee

07/10/2023, 7:38 PM

have you noticed this on non-dynamic tasks?

Geert

07/11/2023, 6:55 AM

I can check 👍

Geert

07/11/2023, 7:24 AM

For non-dynamic tasks it seems to work. Unfortunately, dynamic tasks is actually where this is needed, because Im using that to generate a bunch of tasks for which it would be nice to be able to differentiate them in the UI 😅

Yee

07/11/2023, 5:42 PM

@Jason Porter can you take a look at this? @Geert can you download the

futures.pb

file (it should be in the same place where you normally find the

outputs.pb

) for the dynamic task?

Yee

07/11/2023, 5:43 PM

also could you copy/paste the code for the dynamic task? (feel free to redact of course)

Geert

07/12/2023, 12:29 PM

Hmm the

futures.pb

contains some sensitive credentials, let me make a version that can be shared 👍

Geert

07/12/2023, 1:13 PM

Okay, took me a bit of time to repro and get down to a minimal example. If you have just 1 task created in the loop, I did not observe the issue. When I add another (

task_some_other_task

in this example), the issue occurs.

Copy code

from flytekit import Resources, WorkflowFailurePolicy, dynamic, task, workflow


@task(
    limits=Resources(mem="500Mi"),
)
def some_task(
    val: str,
):
    print(val)


@dynamic(limits=Resources(mem="500Mi"))
def base_workflow():
    # set up task promises
    for val in ["one", "two", "three"]:
        task_some_task = some_task(
            val=val,
        ).with_overrides(node_name=f"{val}-some-task")
        task_some_other_task = some_task(
            val=val,
        ).with_overrides(node_name=f"{val}-some-other-task")

        task_some_task >> task_some_other_task


@workflow(    failure_policy=WorkflowFailurePolicy.FAIL_AFTER_EXECUTABLE_NODES_COMPLETE,
)
def wf():
    base_workflow()

Geert

07/12/2023, 1:15 PM

Starts with the correct overridden names for all tasks, but after the first task

task_some_task

completes, the names of subsequent tasks get updated to random string:

Geert

07/12/2023, 1:18 PM

futures.pb file

futures(1).pb

Geert

07/12/2023, 1:30 PM

Documented in issue here: https://github.com/flyteorg/flyte/issues/3865

21 Views

Open in Slack

Previous Next