I’ve deployed Flyte with the `flyte-binary` Helm c...
# flyte-deployment
e
I’ve deployed Flyte with the
flyte-binary
Helm chart on a GCP GKE cluster. I’ve been able to run workflows and tasks in the cluster. There were successful and failed executions. I can see that there are many pods created with status Error for those failed executions, and the pods stay there. If I delete those failed pods, the information about the executions seem to be vanished, and Flyte Console will complain that it “Failed to fetch data”. Is this an expected behavior that there are many pods created, failed and remained in my cluster? I believe it’s not, but I don’t really get how those failed pods are managed. Any enlightenment would really be appreciated
k
There is a configured garbage collection system. The default interval is 23 hours. Reason they are kept behind is for logs and debugging
e
@Ketan (kumare3) That explains it, thank you very much ☺️
k
You can adjust the interval too
m
I've wondered this for a while, thanks for explaining!
f
Hi @Erik Dao! I was curious which guide did you follow to deploy
flyte-binary
on GCP GKE ? I was able to deploy
flyte-binary
on EKS using the flyte-the-hard-way guide but it only has instructions for EKS - been looking for GCP GKE instructions but haven't found anything so far
k
Have you looked at the deployment guide
Cc @jeev do we not have gcp in single binary
j
we dont have docs for this yet afaict.
k
Ohh that’s a miss
e
@Faisal Anees There is no documentations for deploying Flyte to GCP (omitted since v1.0.0). What I did was to look at the source code of the Helm chart and build my own values.yaml file. In my deployment, I didn’t use GCS but MinIO deployed on the same cluster. It took a while to figure out everything. I’m not sure if I have deployed Flyte properly on GCP yet 😄 But so far, it’s been working for me.
f
@Ketan (kumare3) @jeev is there any chance that you can open source something like a
gcp-starter.yaml
values file ?
j
@Faisal Anees yes
k
Yes we will
I thought it already was
I have no idea how so many folks got it on gcp
Actually @Bernhard Stadlbauer and @Fabio Grätz both use it on gcp
j
should just be the storage and logging configs mostly that are different. and ingress if you are using GCE LB
k
Yup
It’s not much
j
I can help unblock you quickly if you have specific questions @Faisal Anees
f
We run on GCP but not with single binary. A bit of a custom setup with GKE ingress controller and istio service mesh
b
We’re also not using the single binary and doing ingress via an istio service mesh.
e
@Bernhard Stadlbauer May I ask if there is any reason for you to deploy the ingress via istio. I’ve not really worked with it but I want to know how it would be beneficial.
b
For is this was what we also use for our backend (non-flyte) service stack, so mostly for consistency reasons
100 Views