Hi, I have a question about pyflyte. I run the following workflow with the command:

pyflyte run --env "FLYTE_CACHE_VERSION=123" --image myimage="..." --remote wf.py wf

The cache version, i.e., the cache_version of sample_task, is not set correctly in the following workflow. The output of the workflow, "sample_task()" does however return the correct value "123" If I change the type of "wf" to @dynamic instead of @workflow, it works though. Is this the intended behavior?

import os
from flytekit import task, workflow

def get_version():
    version = os.environ.get("FLYTE_CACHE_VERSION")
    if version is None:
        return "default cache version..."
    return version


config_cache = {
    "cache": True,
    "container_image": "{{.image.myimage.fqn}}:{{.image.myimage.version}}",
    "cache_version": get_version(),
}


@task(**config_cache)
def sample_task() -> str:
    return get_version()


@workflow
def wf() -> str:
    return sample_task()

Correct... It's important to understand the process by which flyte interprets your code... There is a compilation step that happens when you do call

run --remote

that loads your module and freezes the task and workflow specifications at that time. Anything you want to specify must be available at module load time... Dynamic workflows, as the name suggests, allow you to dynamically generate specifications at runtime.

--env

is only applied at runtime You can, instead, do this:

FLYTE_CACHE_VERSION=123 pyflyte run --image myimage="..." --remote wf.py wf

or on linux:

export FLYTE_CACHE_VERSION=123
pyflyte run --image myimage="..." --remote wf.py wf

Thanks for your quick reply, your suggestion works perfectly!