https://flyte.org logo
#flyte-deployment
Title
# flyte-deployment
s

Stefano

11/28/2023, 2:30 PM
Hi everyone, We currently have a multicluster deployment running. I am just wondering is there a way to split the controlplane and dataplane s3 storage from one another in the deployment files?
s

Samhita Alla

11/29/2023, 7:59 AM
cc @David Espejo (he/him)
d

David Espejo (he/him)

11/29/2023, 3:05 PM
@Stefano I think you can indicate different S3 buckets in your deployment files but still not sure if it's going to work. As of now, flyte-core doesn't allow to specify separate user/metadata buckets and if that would be the case, still the metadata bucket would be shared by the propellers on each dataplane
s

Stefano

12/01/2023, 10:19 AM
Thanks a lot @David Espejo (he/him). Yes, I noticed in the binary deployment that it was able to have different buckets between the user data & metadata. I did experiment changing the values file when doing a multicluster setup, but that did not work. Are you aware if the functionality for multicluster deployment is in the pipeline currently?
e

Ethan Brown

12/07/2023, 8:41 PM
@Stefano I have some similar requirements -- were you able to get anything working?
s

Stefano

12/07/2023, 9:09 PM
@Ethan Brown unfortunately not.
e

Ethan Brown

12/07/2023, 9:11 PM
Were you able to determine if this is an actual limitation of Flyte itself... or just in how the Helm charts are setup?
s

Stefano

12/07/2023, 9:19 PM
I played around with the values files and could not get it to work. I probably will deepdive properly next week, but since binary can do it I think it is only how the helm charts are setup.
e

Ethan Brown

12/07/2023, 9:25 PM
Cool - that was what I was hoping / thinking too - thanks for the info!
2 Views