How would I implement "dynamic" workflows like: - ...
# flyte-support
f
How would I implement "dynamic" workflows like: • network training is running • I add new training data • the new data gets automatically integrated into the training-process or is this out of scope for pipelines and I have to work with my own events?
t
If you want to add training data during the execution run, you can do that via signaling:

https://www.youtube.com/watch?v=njNKBke5sQ0

.
f
@tall-lock-23197 There might be a misunderstanding here. What I want to achieve is to "always" listen for new data and reconfigure my dataset during runtime. I think "signaling" (at least in flyte terms) is not what I need since it always requires a pre-execution known static DAG.
So I need to implement this myself
t
Yes, you ought to have a flow defined prior to registering your DAG. The flow in itself can be dynamic if the structure can be determined only at run time.
151 Views