Hi all. Has there been any work on, or interest in, scheduling workflows based on the presence of input data and the absence of output data? I'm thinking of something similar to dagster's "asset reconciliation" where a scheduler is aware of the (possibly time-partitioned) inputs and outputs of each workflow and then it continually schedules a set of workflows as each workflow's input becomes available.

hi @Geoff Salmon firstly welcome to the community. Pleasure to meet you

thanks. "reactive workflows" led me to a Flyte OSS Sync Up video from July last year that discusses it. Do you know if there's any other public docs or info about reactive workflows? Is the latest thinking on it still leveraging Argo Events?

not sure if this is exactly what you are looking for

beat me to it!

haha ya

Yeah, it does seem like something that could be built on top. Figuring out a nice way for the user to represent the "assets" and declare their partitioning might be tricky. I don't think I know enough about how workflow inputs and outputs are represented currently.

hmm i can help

sorry, I've got to run now. I'll message you when I have more time. Thanks for the help and useful pointers.

absolutely - anytime