https://flyte.org logo
#ask-the-community
Title
# ask-the-community
f

Frank Shen

02/23/2024, 6:31 PM
Hello, Resource overrides works if the task foo() is defined in the same workflow file. However, if I move the task foo() to another package (so that it’s shared and re-usable task), with_overrides() like below will not be effective. Do you experience this, and what is the solution?
Copy code
@task(
    requests=Resources(
        cpu="3",
        mem="3Gi",
    ),
)
def foo():
    ...



@workflow
def my_wf():
    foo().with_overrides(
        requests=Resources(
            cpu="5",
            mem="5Gi",
        ),
    )
s

Samhita Alla

02/24/2024, 11:36 AM
seems like a bug. would you mind creating an issue? [flyte-bug]
f

Frank Shen

02/26/2024, 5:28 PM
Hi @Samhita Alla, Created. https://github.com/flyteorg/flyte/issues/4957 Please escalate this issue. We are trying to use Flyte in production and without this feature, productizing Flyte is NOT possible! Thanks! CC: @Kevin Su, @David Espejo (he/him)
k

Kevin Su

02/26/2024, 6:29 PM
@Frank Shen fixing it now
f

Frank Shen

04/11/2024, 7:58 PM
Hi @Kevin Su, @Samhita Alla, May I know if the fix has been in any Flyte version yet? Thanks!
s

Samhita Alla

04/12/2024, 6:18 AM
i don't think so, our team's a little busy at the moment, we'll get back to it. meanwhile, want to know if you've tried using
reference_task
and
with_overrides
?
f

Frank Shen

04/12/2024, 4:32 PM
Hi @Samhita Alla, I haven’t tried reference_task & with_overrides on that. Why do we need reference_task if I can import a task from another module that I own? The pod that runs the referenced task will be destroyed and will not be available when I reference it. How can this work?
@Samhita Alla, Could you please prioritize the fix to with_overrides not working if the task is defined in a different module? I am trying to promote Flyte’s re-usable task and workflow feature. And if that doesn’t work, it will fail the reputation of Flyte in our company. Thank you!
s

Samhita Alla

04/12/2024, 5:25 PM
i think it makes sense to use
reference_task
when using tasks available in a different domain or project-domain. else simple module import should suffice. i'm sorry, not sure about the team's availability. @Kevin Su is this an easy fix? @Frank Shen if you could contribute, that'd be amazing!
f

Frank Shen

04/12/2024, 5:26 PM
I am willing to contribute. However, I am new to Flyte code base. I need help on where and how to make the change. Thanks. @Kevin Su, @Samhita Alla