Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Question regarding the `limit-namespace`<https://docs.flyte.org/en/v1.0.0/deployment/cluster_config/flytepropeller_config.html#limit-namespace-string| option for propeller>:  I've tried to reverse engineer the semantics from the code, but I've wanted to verify. Is it possible to pass a list of allowed namespaces here, or is it limited to just `all` or a single namespace?

Based on how the information is used in `controller.go`<https://github.com/flyteorg/flytepropeller/blob/master/pkg/controller/controller.go#L516| , line 516>, I fear that only a single namespace is allowed, right? If so, would you consider using something like xns-informer (<https://github.com/maistra/xns-informer|Github>) to allow a list instead?

My use case: I'm trying to deploy multiple instances of the Flyte control plane in a single cluster for review during CI runs of a deployment configuration. Having multiple propellers configured with `limit-namespace=all` causes a race condition between them. Limiting each propeller instance to a single workflow namespace is too narrow OTOH, since in my understanding it precludes the use of k8s namespaces to separate Flyte projects (please correct me if I'm wrong here).

Thanks for your help! :pray:

Great question <@U04QMD7KS69>! Yes, currently the `limit-namespace` configuration option only allows a single value. However, you might want to look into <https://docs.flyte.org/en/latest/deployment/configuration/performance.html#automatic-scale-out|propeller manager>. Basically, this component is meant for managing deployments with multiple propeller instances, and rather than limiting the informer to only watch a certain namespace it uses label selectors to include / exclude certain Flyte project / domains.

So if you're able to partition the CI runs by project or domain you can either (1) use the propeller manager to automatically manage multiple propeller instances or (2) manually add include / exclude project / domain label selectors to each individual propeller deployment to fit your needs.

If this doesn't work, we certainly would be open to exploring other informer options. I suspect this would be a larger discussion required performance profiling / etc because Flyte relies on this informer quite heavily.

Thanks for your answer, <@U029U35LRDJ>! From skimming the page, propeller manager sounds exactly like what I was looking for! I'll try it out and see how far I can get :slightly_smiling_face: