#3379 [BUG] Flyte webhook TLS certificate error Issue created by stephen37 Describe the bug Flyte certificates have a validity of 1 year, once this year has passed, the TLS certificate is then outdated which causes erorrs with Flyte-webhook

$ kubectl logs flyte-pod-webhook-58c7cf64c4-mbk74 

{"metrics-prefix":"flyte:","certDir":"/etc/webhook/certs","localCert":false,"listenPort":9443,"serviceName":"flyte-pod-webhook","servicePort":443,"secretName":"flyte-pod-webhook","secretManagerType":"Vault","awsSecretManager":{"sidecarImage":"<http://docker.io/amazon/aws-secrets-manager-secret-sidec|docker.io/amazon/aws-secrets-manager-secret-sidec>
2023/02/27 09:41:13 http: TLS handshake error from 10.96.2.17:32928: remote error: tls: bad certificate

You can check the validity of the secret by running

$ kubectl get secret flyte-pod-webhook -o yaml | yq '.data["ca.crt"]' -r  | base64 -d >/tmp/jep && openssl x509 -in /tmp/jep -text | head -n 15

Even after recreating a new secret, the issue still persists as the webhook the old

ca.crt

. Manual steps to fix the issue (thanks to @pradithya) • delete flyte-pod-webhook secret • delete flyte-pod-webhook deployment • create empty flyte-pod-webhook secret -> kubectl create secret generic flyte-pod-webhook • recreate flyte-pod-webhook deployment Without the empty secret the deployment will fail with

Warning  FailedMount  1s (x6 over 16s)  kubelet                                MountVolume.SetUp failed for volume "webhook-certs" : secret "flyte-pod-webhook" not found

Expected behavior New secrets should be used by the webhook so we don't have to delete the webhook and secrets to fix the issue. Additional context to reproduce I guess having Flyte deployed for a year Screenshots Are you sure this issue hasn't been raised already? ☑︎ Yes Have you read the Code of Conduct? ☑︎ Yes flyteorg/flyte