I seem to be running into the same bug as this iss...
# flyte-support
a
I seem to be running into the same bug as this issue opened up here: https://github.com/flyteorg/flyte/issues/4957 - It seems like there is a hint that you can use with_runtime_overrides, but I don't see that mentioned anywhere. Any ideas?
h
@ambitious-businessperson-90319, before we dive into
with_runtime_overrides
, can you talk a bit more about how you're registering the tasks/workflows? I'm thinking that maybe you're being hit by https://github.com/flyteorg/flytekit/pull/2151. Assuming that you're using
pyflyte
to register your code, can you pass
-v
and confirm that you're seeing that warning?
a
I am just using pyflyte run --remote. I am actually overriding the limit not the request by calling it like:
Copy code
my_task().with_overrides(limits=Resources(cpu="2", mem="4Gi"))
Calling pyflyte -v run --remote does not show that warning line
Hmm, actually I am not seeing it change regardless of where the task is defined. I think I am missing something obvious? Here is a more concrete example of what I am testing now: example_workflow.py
Copy code
from flytekit import task, workflow, Resources

from example.example_task import task_in_another_file


@task(
    requests=Resources(mem="1Gi"),
    limits=Resources(mem="1Gi"),
)
def task_in_same_file(
        word: str,
) -> str:
    return f"Example {word}"


@workflow
def simple_workflow(word: str) -> str:
    word_result = task_in_same_file(word=word).with_overrides(
        requests=Resources(mem="2Gi"),
        limits=Resources(mem="2Gi"),
    )
    word_result2 = task_in_another_file(word=word_result).with_overrides(
        requests=Resources(mem="2Gi"),
        limits=Resources(mem="2Gi"),
    )
    return word_result2
example_task.py
Copy code
from flytekit import task, Resources


@task(
    requests=Resources(mem="1Gi"),
    limits=Resources(mem="1Gi"),
)
def task_in_another_file(
        word: str,
) -> str:
    return f"Example {word}"
Command:
Copy code
pyflyte -v run -p mlplatform -d public  --copy-all --remote example/example_workflow.py simple_workflow --word my_word
Looking at the admin page for both versions of the task I see the non overriden values:
h
ok, there's another issue at play: https://github.com/flyteorg/flyte/issues/5589. There's nothing to do with overrides happening in another module. If you check the pods in k8s the resources will be overridden.
a
Ah, yep. If I check the pods they have the correct limits. Thanks!
👍 1