acoustic-carpenter-78188
05/06/2024, 6:55 PMcontainer_image
as an example for a task node override:
@task
def foo() -> str:
return "foo"
@workflow
def wf():
foo().with_overrides(
container_image="{{.image.default.fqn}}:" + "9410d0f0ed4a25577ab35a79bd3eb1119d8627d59c7a8fb947d42ca9fb46a61c"
)
If one runs this workflow with pyflyte run
, everything works as expected. If, however, one uses pyflyte register
and runs it manually from the UI, the default image is used.
* * *
@workflow
def wf():
foo().with_overrides(container_image="ubuntu:foo")
foo().with_overrides(container_image="ubuntu:bar")
Another problem is that even if one uses pyflyte run
, in this example both task nodes will use the image "ubuntu:bar"
.
Expected behavior
The reason for this behaviour is that only resource/extended resource overrides are treated as workflow-level task-node overrides which are registered separately from the task themselves and then overridden in the backend.
Other overrides such as container_image
override the task-level metadata before the registration process. These can only be used once as "the last override wins" and appear to not work with pyflyte register
.
The proper fix is to treat all overrides as task node overrides in flyteidl instead of overriding task-level metadata before registration.
* * *
☑︎ Container image
• #4858
• flyteorg/flytekit#2176
☐ ...
Are you sure this issue hasn't been raised already?
☑︎ Yes
Have you read the Code of Conduct?
☑︎ Yes
flyteorg/flyteacoustic-carpenter-78188
05/06/2024, 6:55 PMacoustic-carpenter-78188
05/23/2024, 2:06 PM