There’s a new use case that came up and we’re trying to figure how best to do this. I was wondering if I could some help with resources/ideas. So we have a bunch of tasks that are already registered and at runtime we want to be able to construct the graph. One thought was to basically create the workflow graph spec and submit a request to flyteadmin similar to how flytekit does. Then the newly created workflow can be launched. In short we want to be able to generate a workflow without needing to write python code for it. Just specify the proto spec. Thoughts? @acceptable-policeman-57188 @freezing-airport-6809 @thankful-minister-83577 cc: @prehistoric-pharmacist-43654 @acceptable-river-81601

can’t you fetch_task?

So it would kind of work but then you still have to generate the code for it at runtime. The idea is there is a UI that users can then move around to build their workflow and then the service would turn this UI to a graph. @acceptable-river-81601 feel free to add anything that I may have missed

you can yeah

yes basically

coool

Yeah it’ll have to be done in JS in some way. I don’t see why the core piece of it can’t be open sourced.

just to clarify, so we need to handle dag structure creation, but we’d be able to do it in such a way that the actual node dependencies (ordering, parallelism, etc.) are handled by flyte during the workflow execution?

if the dag is created correctly, then yeah the flyte execution backend will handle everything else

@glamorous-father-92994 you can actually use imperative programming model instead of going to the proto

Yeah but I think the goal here is ideally go to proto. The imperative model there still has to be a way of generating the dag and inputs and outputs dynamically so going straight to proto from UI would I think be a better solution

ya would we awesome

Hi all, sorry for digging up an old thread, but I wanted to circle back to this idea. For my current use case, we have our own definitions of nodes/workflows as well, so I’m thinking a conversion tool would make more sense (than, say, working directly in flyteconsole). However I was wondering if I could still keep some of you in the loop for support/tips in creating the protos. Also, any chance there are examples anywhere? (does flytekit or flyteconsole interact with flyteadmin this way?)

@acceptable-river-81601 lets make a channel, where we can collaborate. We support this

Just created #C03CYRW6M53!