Question about multi-cluster setups: In the <docs...
# ask-the-community
f
Question about multi-cluster setups: In the docs it says the following about configuring the dataplane propeller:
Copy code
configmap:
  ...
  catalog:
    catalog-cache:
      endpoint: <your-Ingress-FQDN>:443
      insecure: false
Is the assumption here that one modifies the ingress to also expose datacatalog routes? By default only admin is exposed, see here. Not a problem to do so, the docs are just vague here. How would authentication work here though? All auth is handled in admin.
About auth: Not a problem for me since all of it will be in privat networks, I just wonder what the “official way” is.
k
@Fabio Grätz with Union multi cluster is out of the box
d
@Fabio Grätz it would only make sense if the Ingress also exposes the datacatalog service as you said. Not sure if that counts as "official" but I don't see other way it would work on a multi cluster context where caching is enabled. Also, in terms of auth, it will still go through the
admin.admin.endpoint
f
I gave datacatalog a VPC internal load balancer type service for now and just do all the communication between dataplane and control plane via internal routes. Works well.
Also, in terms of auth, it will still go through the
admin.admin.endpoint
If I exposed datacatalog through the ingress, wouldn’t it just accept any connection from the internet? (Not a problem I need to solve, just curious.)
d
but, can someone query datacatalog without passing through flyteadmin? Not sure
f
If one needs to pass through flyteadmin, I wonder why the docs say to expose datacatalog 🤔
At least when developing flytepropeller locally, I always port-forward datacatalog to localhost so I assume that the communication is direct 🤔
g
Fabio, I'm just curious, if you wanted to update the docs to clarify them for this particular case, do you know how to push a docs PR all the way through? I ask because we have a pretty complicated docs build, and I'm not sure if we have done all we should to make it easy to accept docs contributions from the community. Feel free to DM me if you don't want to sidetrack this thread. Thanks.
f
Hey George, happy to contribute to the docs here. Have added to docs in the past, not a problem. But to me it’s still unclear what the recommended approach is for propeller->datacatalog networking in a multi-cluster setup. Would be good if someone from union could clarify how this is intended to be used 🙏 Then I’m happy to put that into the docs.
Let’s do it like this: I’ll bring this up in the next contributors’ sync meeting.
Maybe someone there knows this.
Will get back to you.
g
Ok thanks. I have absolutely no idea about the meat of your question. Just checking in on your perception of the docs contribution process.
f
Sometimes sphinx errors are difficult to understand but that’s just sphinx 🤷‍♂️
Not a blocker I’d say.
g
Sphinx is a well-named piece of software
f
g
Thanks Fabio. @Dan Rammer (hamersaw) you might want to take a look at Fabio's doc PR. I'm not sure how deep in the quagmire of multi-cluster config @Nikki Everett is at this point.