https://flyte.org logo
#ask-the-community
Title
# ask-the-community
d

Dan Farrell

11/01/2023, 3:42 PM
I'm working on a plugin to use imagespec to build from dockerfiles. https://github.com/flyteorg/flytekit/pull/1926 When I pyflyte run a workflow once, and then change my dockerfile, and then pyflyte run the same workflow I get the error:
Copy code
RPC Failed, with Status: StatusCode.INVALID_ARGUMENT
        details: task with different structure already exists with id resource_type:TASK project:"flytesnacks" domain:"development" name:"example_pyflyte_script.t1" version:"LMxQheYT9myAmWklNHYD6A=="
This seems like a bug, but it also seems like a design decision at the same time, so just wondering if this is intentional or not? -- also wondering what might be a workaround. I suppose it really depends how coupled you want tasks and images to be
it seems like when you submit a workflow, a hash (version) is built based on the python code and given to all of the nodes(tasks, workflows, etc), but it doesn't seem to take into account the imagespec https://github.com/flyteorg/flytekit/blob/master/flytekit/tools/translator.py#L171 (settings.version is derived from https://github.com/flyteorg/flytekit/blob/master/flytekit/remote/remote.py#L906C14-L906C14 )
n

Niels Bantilan

11/01/2023, 6:52 PM
Hi Dan, so the tag is being calculated here, so you’ll need to update calculate_hash_from_image_spec so that it includes the dockerfile content in its hash calculation
d

Dan Farrell

11/01/2023, 6:59 PM
@Niels Bantilan that's already taken into account by
ImageSpec.source_root
which I've co-opted as a dockerfile context variable
n

Niels Bantilan

11/01/2023, 8:37 PM
Ah, sorry is the issue that the version string
LMxQheYT9myAmWklNHYD6A==
should also take image spec into account? @Kevin Su @Eduardo Apolinario (eapolinario)
d

Dan Farrell

11/02/2023, 12:34 AM
I put my 'solution' in the PR.... but it is not very good. I don't want to make sweeping changes without discussing