👋 Hello, team! I hope this is the correct channel to ask. Context: As part of our CI/CD process I'm registering flyte packages on our test setup. I got a bunch of these .tgz files produced by our pipeline that I can register with "flytectl register". I have the following questions: • I noticed that

flytectl register

is not an atomic operation / transaction. It starts registering but aborts when failing on a workflow and ends up in this half deployed state. Is it possible for flytectl to rollback in that case? Or do first pre-checks and only register if all checks pass? • We are using reference tasks heavily and when registering all workflows, we'd need to make sure we register in the correct topological order as flytectl register will check that reference tasks are there. Is there a way to take the workflows just to be registered also into account or disable that check? • Is there a way to register multiple archives at once? • I see that

--force

can be used to forcefully set the version for all entities to be registered and that seems to also apply to reference tasks. Is there a way to also set project/domain on the referenced tasks? This is because we'd deploy for testing into a separate project/domain. I can solve these problems by: • Manually merging the tgz files for all my workflows • Topologically order the entries so that I register the workflows first that are referenced later as reference tasks • Overwrite the project/domain/version in the protos specifying called reference tasks manually But I was just wondering whether there's an easier way around that.

It should already be using topological sort. Reference workflows will not use topological sort, as it is not known to registration (by design) if referenced entities are already registered or they will be registered in this process. The best way to achieve rollback or roll forward is to use —activate launchplans which is guaranteed to be atomic and is the only way to schedule workflows. Registration on their own should have no side effects like stopping schedules Is the primary concern that referenced entities are not registered topologically

Let's say I want to deploy my current state of all workflows to a fresh test cluster or a new cluster. I have the docker files and a list of tgz packages that have been produced by "pyflyte package". I can merge these packages so I have one tgz archive. But then I do have to do topological sort. Ideally I just want flytectl register to only check in the end, and rollback if anything fails so as to not leave the new cluster in a half registered state, if that makes any sense.

aah @Sebastian Millius you are merging this is not ok