Is there any declerative approach to flyte stuff like gitopsy ?

Flyte is code declarative right? Question is what do you want to achieve

manage flyte via gitops

you mean things like quotas etc?

I've also been thinking about this. Would a CRD to create and archive flyte projects be useful?

That could be

I’d be willing to give it a go if anyone thinks it’s worthwhile. I can start with a small POC.

But it might be an overkill

At our org we try to manage most of our stuff through things like fluxcd/argocd so it would be nice to have it pulling from a CRD rather than us managing it through flytectl. Definitely just a nice to have at the moment, but would tie everything together

You don’t have to manage through flytectl right

I think this is referring to the entire "workflow" specification being able to be applied as a single CRD? This is an interesting tradeoff. We run data pipelines in Argo, & the

Workflow

/`CronWorkflow` manifests are easy to deploy/promote in a GitOps setting (ie. promotion involves copying the manifest from a staging namespace to a prod namespace, usually w/ some automation like a GitHub Action). We use Flux as well and it's a convenient pattern if you want to separate the repos where core workflow development vs deployment happen/enforce additional controls and disallow direct front-door access to the clusters the workload runs in

That would be amazing as well.

Please draft an rfc, this would be amazing to learn

@Sam Cox this is becoming more important for us too. Let's connect on this?

And this is absolutely ok if you create one

Out of curiosity @Ketan (kumare3), how did Lyft handle this with Flyte (if you can share)? Seems like more stringent access controls/promotion story would've been important in an org that large?

Ya we had a separate deployment system - using Jenkins

My current scenario is we define our “projects” in one repo with namespace, service accounts, cross plane (ie s3), etc and the Flyte project has to be applied through flytectl. The main flux kustomization controller applied all these specs/CRDs.

When users would Promote workflows and that would allow registration sto new Domains

I see - so this was handled at the CD-level. We're currently following this approach as well (our actions runners can talk to the cluster that we run flyte in). But like @Sam Cox said, it'd be nice to have a consistent approach for both model training and data ETLs

@Rahul Mehta for sure.

Also @Sam Cox there seem to be a few different concepts here. Think this can be broken up into CRDs for 1) management and 2) registration of entities/execution

Yes 1. Is what I was going after but 2 would be a game changer

Nice - 1 is a no-brainer for us in terms of the value-add, and I've got some thoughts re: 2

Yes let’s connect. I’m just getting started in Flyte so I’m sure I haven’t seen all the pitfalls and use cases

Cool, let's schedule something next week. Can post an invite in #flyte-deployment in case other folks are interested in gitops-ifying their flyte deployments

Sounds good to me

Ohh @Sam Cox pleasure to hear that. Thank you. One of my asks from the community is to share the word. Maybe write a blog or so

That sounds good to me. I’d be happy to

Cc @Tyler Su / @David Espejo (he/him) can you please point Sam towards the signup sheet. Sam I think next dates are in April or may

Hi folks. Was there any outcomes from the above meeting? We are interested in implementing a CRD/ operator for registering entities on a cluster