Hi! I'm trying to understand how to best use `pyfl...
# flyte-support
e
Hi! I'm trying to understand how to best use
pyflyte package
in our deployments.. The setup: • using
flytekit==1.16.1
• tasks are using
ImageSpec
containers without specifying any of the
source_root
,
copy
or
source_copy_mode
args (i.e. using the defaults) When changing code or dependencies in one of the tasks,
pyflyte package
generates new image tags and rebuilds every
ImageSpec
container, even the ones not affected by the code/deps changes. What's the recommended strategy to ensure that the
ImageSpec
tag only depends on the relevant task code and dependencies?
r
can you just pyflyte package a specific workflow? you can pyflyte run just a specific workflow
e
I only have one workflow w/ multiple tasks, e.g.
Copy code
@task(container_image=image_spec_1)
def task1():
    ...

@task(container_image=image_spec_2)
def task2():
    ...

@workflow
def wf():
    task1()
    task2()
My problem is that changes to `task1`'s code or `image_spec_1`'s dependencies cause
image_spec_2
to be rebuilt as well and vice versa because the tag changes. I believe this is because
ImageSpec
hashes the entire code base to determine the tag. I wonder if there is a way to restrict which portion of the code base is hashed (e.g. via the
source_root
arg)
r
interesting, i have not seen this issue before but good to know it exists