New RunLLM response generated: >>>*# Question* Hi, I am currently working on a Flyte POC. I am migrating one of our KF pipelines and I am trying to understand if I can run a workflow from step N and then provide another workflow run id to reuse the inputs from. I've been reading the docs and I can barely find any info about it. ChatGPT suggested using the recover function, which doesn't seem to exist in flytekit. Could anyone give me a hint how that could be achieved in Flyte, please? Our use case is not necessarily just recovering a failed run, but rather a rapid prototyping when DS people could run a heavy workflow end to end, and then, say, rerun only certain parts of the workflow later (model training with different hyperparams etc) reusing artifacts from the previous complete run (heave SQL + feature engineering). So, ideally they say I want to run from step ModelTraining reusing features generated in another run id 123. I want to do it programmatically from python/CLI Thank you! # Answer Flyte currently does not support running a workflow from an arbitrary step using inputs from a previous workflow run directly via the Python SDK (

flytekit

). The

recover

functionality, which allows reusing outputs from previous runs, is supported through the Flyte console and

flytectl

, but not yet in

flytekit

(source). For your use case, you can use caching to reuse outputs from previous runs. This involves setting up tasks with caching enabled, so that when you rerun the workflow, Flyte can reuse the cached outputs from previous successful executions (source). Would you like more details on how to set up caching or other potential workarounds? Sources: - https://github.com/flyteorg/flyte/issues/5049 - https://discuss.flyte.org/t/15715491/hey-folks-i-have-a-use-case-where-i-d-like-to-backfill-rerun/ Slack Link https://flyte-org.slack.com/archives/CP2HDHKE1/p1718684349.035799 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.