Channels
announcements
ask-the-community
auth
conference-talks
contribute
databricks-integration
datahub-flyte
deployment
ecosystem-unionml
engineeringlabs
events
feature-discussions
flyte-bazel
flyte-build
flyte-console
flyte-deployment
flyte-documentation
flyte-github
flyte-on-gcp
flyte-ui-ux
flytekit
flytekit-java
flytelab
great-content
hacktoberfest-2022
helsing-flyte
in-flyte-conversations
integrations
introductions
jobs
konan-integration
linkedin-flyte
random
ray-integration
ray-on-flyte
release
scipy-2022-sprint
show-and-tell
sig-large-models
torch-elastic
workflow-building-ui-proj
writing-w-sfloris
Title
o
Ophir Yoktan
01/02/2023, 7:02 PMMy use case is as follows:
the initial task of the flyte workflow extracts a dataset
* in production, I want to always use the latest data (or use cache with short expiration
* during development, runtime is more important then data freshness - so I prefer to use cached dataset
is there an option to control somehow the caching when launching a workflow (and not just when defining the workflow)
Cross posting:
https://stackoverflow.com/q/74984146/590335
k
Ketan (kumare3)
01/03/2023, 3:23 AMYes cache can be disabled per workflow execution
A better way would be to use a changeable input value, this might give finer control
o
Ophir Yoktan
01/03/2023, 5:03 AMCan you point me to a reference about the 1st option?
k
Ketan (kumare3)
01/03/2023, 5:18 AMIf you look at the executionCreateRequest - https://docs.flyte.org/projects/flyteidl/en/latest/protos/docs/admin/admin.html#executionspec - it has an
overwrite_cacheas you can see it can be passed when you launch an execution - https://github.com/flyteorg/flytekit/blob/0585b1394c6a6a90a35e8a337bc079bead6f7eb2/flytekit/remote/remote.py#L915
but it seems not from the ui yet
this PR is in progress - https://github.com/flyteorg/flyteconsole/pull/616