k

    Katrina P

    2 months ago
    Currently, I've got flyte at least mostly deployed on our k8s cluster, which is a self managed cluster with a traefik ingress provider. our ingress controller provides TLS certs if traffic flows through it for various traffic through our subdomains and have access to use https://cert-manager.io/ in our cluster. I have applied a manifest for the cert manager issuer, but the
    flyte-pod-webook
    won't come up due to:
    Failed to start webhook. Error: open /etc/webhook/certs/tls.crt: no such file or directory.
    Anyone have some insight into this?
    Haytham Abuelfutuh

    Haytham Abuelfutuh

    2 months ago
    Hey @Katrina P, Have you installed flyte through the flyte-core helm chart? This init container should take care of initializing the needed certs: https://github.com/flyteorg/flyte/blob/master/charts/flyte-core/templates/propeller/webhook.yaml#L41-L63
    k

    Katrina P

    2 months ago
    No I had to generate the manifests from the helm chart and install those since our k8s cluster admin doesn't support helm installation, but I did apply that manifest with that container
    There weren't much logs from the
    generate-secrets
    pod: `
    time="2022-07-06T21:17:16Z" level=info msg=------------------------------------------------------------------------
    time="2022-07-06T21:17:16Z" level=info msg="App [flytepropeller], Version [unknown], BuildSHA [unknown], BuildTS [2022-07-06 21:17:16.101877907 +0000 UTC m=+0.038773580]"
    time="2022-07-06T21:17:16Z" level=info msg=------------------------------------------------------------------------
    time="2022-07-06T21:17:16Z" level=info msg="Detected: 2 CPU's\n"
    Haytham Abuelfutuh

    Haytham Abuelfutuh

    2 months ago
    kubectl logs -n flyte deploy/flytepropeller-webhook -c generate-secrets
    is that what you are running?
    k

    Katrina P

    2 months ago
    And for the webhook: `
    time="2022-07-06T21:28:44Z" level=info msg=------------------------------------------------------------------------
    time="2022-07-06T21:28:44Z" level=info msg="App [flytepropeller], Version [unknown], BuildSHA [unknown], BuildTS [2022-07-06 21:28:44.093828325 +0000 UTC m=+0.029201245]"
    time="2022-07-06T21:28:44Z" level=info msg=------------------------------------------------------------------------
    time="2022-07-06T21:28:44Z" level=info msg="Detected: 2 CPU's\n"
    {"metrics-prefix":"flyte:","certDir":"/etc/webhook/certs","localCert":false,"listenPort":9443,"serviceName":"flyte-pod-webhook","servicePort":443,"secretName":"flyte-pod-webhook","secretManagerType":"K8s","awsSecretManager":{"sidecarImage":"<http://docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4|docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4>","resources":{"limits":{"cpu":"200m","memory":"500Mi"},"requests":{"cpu":"200m","memory":"500Mi"}}},"vaultSecretManager":{"role":"flyte","kvVersion":"2"}}
    {"json":{},"level":"fatal","msg":"Failed to start webhook. Error: open /etc/webhook/certs/tls.crt: no such file or directory","ts":"2022-07-06T21:28:44Z"}
    I have no idea why its referencing aws secrets manager there though; misconfigured something maybe?
    s

    Shahwar Saleem

    2 months ago
    @Katrina P Generating a certificate from Cloudformation and using its ARN in
    values.yaml
    solved my problem.
    Haytham Abuelfutuh

    Haytham Abuelfutuh

    2 months ago
    I think we are talking about different certs, @Shahwar Saleem and @Katrina P
    @Katrina P can you bump the logger level and restart the webhook deployment?
    there should be logs about it issuing certs… etc.
    k

    Katrina P

    2 months ago
    Yeah, I"m doing that now, what's the appropriate log level here? 6?
    k

    katrina

    2 months ago
    6 is debug so that should be fine
    k

    Katrina P

    2 months ago
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [storage] updated. No update handler registered.","ts":"2022-07-06T21:54:08Z"}
    {"json":{"src":"init_cert.go:50"},"level":"info","msg":"Issuing certs","ts":"2022-07-06T21:54:08Z"}
    {"json":{"src":"init_cert.go:61"},"level":"info","msg":"Creating secret [flyte-pod-webhook] in Namespace [flyte]","ts":"2022-07-06T21:54:11Z"}
    {"json":{"src":"init_cert.go:115"},"level":"info","msg":"A secret already exists with the same name. Validating.","ts":"2022-07-06T21:54:11Z"}
    Hmmm a conflicting secret 🤔
    Haytham Abuelfutuh

    Haytham Abuelfutuh

    2 months ago
    hmm not necessarily
    if no logs after that then it thinks everything is good..
    k

    Katrina P

    2 months ago
    The webhook:
    time="2022-07-06T21:57:08Z" level=info msg=------------------------------------------------------------------------
    time="2022-07-06T21:57:08Z" level=info msg="App [flytepropeller], Version [unknown], BuildSHA [unknown], BuildTS [2022-07-06 21:57:08.170414367 +0000 UTC m=+0.033019880]"
    time="2022-07-06T21:57:08Z" level=info msg=------------------------------------------------------------------------
    time="2022-07-06T21:57:08Z" level=info msg="Detected: 2 CPU's\n"
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.aws] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.aws.batch] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.sagemaker] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.bigquery] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.snowflake] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.catalogcache] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.k8s] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.logs] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.k8s-array] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.qubole] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.spark] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [plugins.athena] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [secrets] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:400"},"level":"debug","msg":"Config section [admin] updated. Firing updated event.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [event] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [catalog-cache] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [storage] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [propeller] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [propeller.admin-launcher] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [propeller.resourcemanager] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [propeller.workflowstore] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"viper.go:398"},"level":"debug","msg":"Config section [webhook] updated. No update handler registered.","ts":"2022-07-06T21:57:08Z"}
    {"metrics-prefix":"flyte:","certDir":"/etc/webhook/certs","localCert":false,"listenPort":9443,"serviceName":"flyte-pod-webhook","servicePort":443,"secretName":"flyte-pod-webhook","secretManagerType":"K8s","awsSecretManager":{"sidecarImage":"<http://docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4|docker.io/amazon/aws-secrets-manager-secret-sidecar:v0.1.4>","resources":{"limits":{"cpu":"200m","memory":"500Mi"},"requests":{"cpu":"200m","memory":"500Mi"}}},"vaultSecretManager":{"role":"flyte","kvVersion":"2"}}
    {"json":{"src":"server.go:96"},"level":"info","msg":"Starting profiling server on port [10254]","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"entrypoint.go:88"},"level":"info","msg":"Creating MutatingWebhookConfiguration [flyte/flyte-pod-webhook]","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"entrypoint.go:93"},"level":"info","msg":"Failed to create MutatingWebhookConfiguration. Will attempt to update. Error: <http://mutatingwebhookconfigurations.admissionregistration.k8s.io|mutatingwebhookconfigurations.admissionregistration.k8s.io> \"flyte-pod-webhook\" already exists","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"entrypoint.go:103"},"level":"info","msg":"Successfully updated existing mutating webhook config.","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"pod.go:143"},"level":"info","msg":"Registering path [/mutate--v1-pod]","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"entrypoint.go:55"},"level":"info","msg":"Starting controller-runtime manager","ts":"2022-07-06T21:57:08Z"}
    {"json":{"src":"webhook.go:119"},"level":"fatal","msg":"Failed to start webhook. Error: open /etc/webhook/certs/tls.crt: no such file or directory","ts":"2022-07-06T21:57:08Z"}
    Haytham Abuelfutuh

    Haytham Abuelfutuh

    2 months ago
    Can you validate the secret?
    kubectl get secret -n flyte flyte-propeller-webhook -o yaml
    does it have
    tls.crt
    do you also mind posting the manifest of the deployment?
    kubectl get deploy -n flyte flyte-propeller-webhook -o yaml
    k

    Katrina P

    2 months ago
    damn I don't have kubectl access to our cluster (we have a delivery platform between me and the cluster) -- I'll have to get our infra team to help tomorrow to run those
    Haytham Abuelfutuh

    Haytham Abuelfutuh

    2 months ago
    aha… np, what do you have access to?
    can you get me the manifest you applied?
    k

    Katrina P

    2 months ago
    yup one sec
    # Source: flyte-core/templates/propeller/webhook.yaml
    # Create the actual deployment
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: flyte-pod-webhook
      namespace: flyte
      labels:
        app: flyte-pod-webhook
    spec:
      selector:
        matchLabels:
          app: flyte-pod-webhook
      template:
        metadata:
          labels:
            app: flyte-pod-webhook
            <http://app.kubernetes.io/name|app.kubernetes.io/name>: flyte-pod-webhook
            <http://app.kubernetes.io/version|app.kubernetes.io/version>: v1.0.0
          annotations:
            configChecksum: "94a14941954f8d44256768b8600c65108c984e8ea369ed3f61b79ffbbebfc6b"
        spec:
          securityContext:
            fsGroup: 65534
            runAsUser: 1001
            fsGroupChangePolicy: "Always"
          serviceAccountName: flyte-pod-webhook
          initContainers:
          - name: generate-secrets
            image: "<http://cr.flyte.org/flyteorg/flytepropeller:v1.0.0|cr.flyte.org/flyteorg/flytepropeller:v1.0.0>"
            imagePullPolicy: "IfNotPresent"
            command:
              - flytepropeller
            args:
              - webhook
              - init-certs
              - --config
              - /etc/flyte/config/*.yaml
            env:
              - name: POD_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              - name: POD_NAMESPACE
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
            volumeMounts:
              - name: config-volume
                mountPath: /etc/flyte/config
          containers:
            - name: webhook
              image: "<http://cr.flyte.org/flyteorg/flytepropeller:v1.0.0|cr.flyte.org/flyteorg/flytepropeller:v1.0.0>"
              imagePullPolicy: "IfNotPresent"
              command:
                - flytepropeller
              args:
                - webhook
                - --config
                - /etc/flyte/config/*.yaml
              env:
                - name: POD_NAME
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.name
                - name: POD_NAMESPACE
                  valueFrom:
                    fieldRef:
                      fieldPath: metadata.namespace
              volumeMounts:
                - name: config-volume
                  mountPath: /etc/flyte/config
                  readOnly: true
                - name: webhook-certs
                  mountPath: /etc/webhook/certs
                  readOnly: true
          volumes:
            - name: config-volume
              configMap:
                name: flyte-propeller-config
            - name: webhook-certs
              secret:
                secretName: flyte-pod-webhook
          nodeSelector:
            role: flyte
    Haytham Abuelfutuh

    Haytham Abuelfutuh

    2 months ago
    Ah, I think you are hitting this: https://github.com/flyteorg/flytepropeller/pull/427 Can you upgrade to the latest flyte release? v1.1.0 ?
    k

    Katrina P

    2 months ago
    Ahh okay, sure thing, it'll take some time for the redeployment and for the pods to come up but I'll check back here if there's still any issues. Thanks so much for your help thus far!
    Good mornin! I upgraded to 1.1.0 but I'm getting the same logs, unfortunately. I'm going to double check my configs
    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        <http://artifact.spinnaker.io/location|artifact.spinnaker.io/location>: flyte
        <http://artifact.spinnaker.io/name|artifact.spinnaker.io/name>: flyte-pod-webhook
        <http://artifact.spinnaker.io/type|artifact.spinnaker.io/type>: kubernetes/deployment
        <http://artifact.spinnaker.io/version|artifact.spinnaker.io/version>: ''
        <http://cni.projectcalico.org/containerID|cni.projectcalico.org/containerID>: d7dd05a9a4e3e73642e66acc045a8e92c9a2ea631c09a1ece4593d49ea152ab8
        configChecksum: 94a14941954f8d44256768b8600c65108c984e8ea369ed3f61b79ffbbebfc6b
        <http://moniker.spinnaker.io/application|moniker.spinnaker.io/application>: flyte
        <http://moniker.spinnaker.io/cluster|moniker.spinnaker.io/cluster>: deployment flyte-pod-webhook
      creationTimestamp: '2022-07-06T22:51:06Z'
      generateName: flyte-pod-webhook-dcbfbdf9-
      labels:
        app: flyte-pod-webhook
        <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: spinnaker
        <http://app.kubernetes.io/name|app.kubernetes.io/name>: flyte-pod-webhook
        <http://app.kubernetes.io/version|app.kubernetes.io/version>: v1.1.0
        pod-template-hash: dcbfbdf9
      managedFields:
        - apiVersion: v1
          fieldsType: FieldsV1
          fieldsV1:
            'f:metadata':
              'f:annotations':
                'f:<http://cni.projectcalico.org/containerID|cni.projectcalico.org/containerID>': {}
                'f:<http://cni.projectcalico.org/podIP|cni.projectcalico.org/podIP>': {}
                'f:<http://cni.projectcalico.org/podIPs|cni.projectcalico.org/podIPs>': {}
          manager: calico
          operation: Update
          subresource: status
          time: '2022-07-06T22:51:06Z'
        - apiVersion: v1
          fieldsType: FieldsV1
          fieldsV1:
            'f:metadata':
              'f:annotations':
                .: {}
                'f:<http://artifact.spinnaker.io/location|artifact.spinnaker.io/location>': {}
                'f:<http://artifact.spinnaker.io/name|artifact.spinnaker.io/name>': {}
                'f:<http://artifact.spinnaker.io/type|artifact.spinnaker.io/type>': {}
                'f:<http://artifact.spinnaker.io/version|artifact.spinnaker.io/version>': {}
                'f:configChecksum': {}
                'f:<http://moniker.spinnaker.io/application|moniker.spinnaker.io/application>': {}
                'f:<http://moniker.spinnaker.io/cluster|moniker.spinnaker.io/cluster>': {}
              'f:generateName': {}
              'f:labels':
                .: {}
                'f:app': {}
                'f:<http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>': {}
                'f:<http://app.kubernetes.io/name|app.kubernetes.io/name>': {}
                'f:<http://app.kubernetes.io/version|app.kubernetes.io/version>': {}
                'f:pod-template-hash': {}
              'f:ownerReferences':
                .: {}
                'k:{"uid":"1098743a-7e43-449b-8d8e-5e02595325a7"}': {}
            'f:spec':
              'f:containers':
                'k:{"name":"webhook"}':
                  .: {}
                  'f:args': {}
                  'f:command': {}
                  'f:env':
                    .: {}
                    'k:{"name":"POD_NAME"}':
                      .: {}
                      'f:name': {}
                      'f:valueFrom':
                        .: {}
                        'f:fieldRef': {}
                    'k:{"name":"POD_NAMESPACE"}':
                      .: {}
                      'f:name': {}
                      'f:valueFrom':
                        .: {}
                        'f:fieldRef': {}
                  'f:image': {}
                  'f:imagePullPolicy': {}
                  'f:name': {}
                  'f:resources': {}
                  'f:terminationMessagePath': {}
                  'f:terminationMessagePolicy': {}
                  'f:volumeMounts':
                    .: {}
                    'k:{"mountPath":"/etc/flyte/config"}':
                      .: {}
                      'f:mountPath': {}
                      'f:name': {}
                      'f:readOnly': {}
                    'k:{"mountPath":"/etc/webhook/certs"}':
                      .: {}
                      'f:mountPath': {}
                      'f:name': {}
                      'f:readOnly': {}
              'f:dnsPolicy': {}
              'f:enableServiceLinks': {}
              'f:initContainers':
                .: {}
                'k:{"name":"generate-secrets"}':
                  .: {}
                  'f:args': {}
                  'f:command': {}
                  'f:env':
                    .: {}
                    'k:{"name":"POD_NAME"}':
                      .: {}
                      'f:name': {}
                      'f:valueFrom':
                        .: {}
                        'f:fieldRef': {}
                    'k:{"name":"POD_NAMESPACE"}':
                      .: {}
                      'f:name': {}
                      'f:valueFrom':
                        .: {}
                        'f:fieldRef': {}
                  'f:image': {}
                  'f:imagePullPolicy': {}
                  'f:name': {}
                  'f:resources': {}
                  'f:terminationMessagePath': {}
                  'f:terminationMessagePolicy': {}
                  'f:volumeMounts':
                    .: {}
                    'k:{"mountPath":"/etc/flyte/config"}':
                      .: {}
                      'f:mountPath': {}
                      'f:name': {}
              'f:nodeSelector': {}
              'f:restartPolicy': {}
              'f:schedulerName': {}
              'f:securityContext':
                .: {}
                'f:fsGroup': {}
                'f:fsGroupChangePolicy': {}
                'f:runAsUser': {}
              'f:serviceAccount': {}
              'f:serviceAccountName': {}
              'f:terminationGracePeriodSeconds': {}
              'f:volumes':
                .: {}
                'k:{"name":"config-volume"}':
                  .: {}
                  'f:configMap':
                    .: {}
                    'f:defaultMode': {}
                    'f:name': {}
                  'f:name': {}
                'k:{"name":"webhook-certs"}':
                  .: {}
                  'f:name': {}
                  'f:secret':
                    .: {}
                    'f:defaultMode': {}
                    'f:secretName': {}
          manager: kube-controller-manager
          operation: Update
          time: '2022-07-06T22:51:06Z'
        - apiVersion: v1
          fieldsType: FieldsV1
          fieldsV1:
            'f:status':
              'f:conditions':
                'k:{"type":"ContainersReady"}':
                  .: {}
                  'f:lastProbeTime': {}
                  'f:lastTransitionTime': {}
                  'f:message': {}
                  'f:reason': {}
                  'f:status': {}
                  'f:type': {}
                'k:{"type":"Initialized"}':
                  .: {}
                  'f:lastProbeTime': {}
                  'f:lastTransitionTime': {}
                  'f:status': {}
                  'f:type': {}
                'k:{"type":"Ready"}':
                  .: {}
                  'f:lastProbeTime': {}
                  'f:lastTransitionTime': {}
                  'f:message': {}
                  'f:reason': {}
                  'f:status': {}
                  'f:type': {}
              'f:containerStatuses': {}
              'f:hostIP': {}
              'f:initContainerStatuses': {}
              'f:phase': {}
              'f:podIP': {}
              'f:podIPs':
                .: {}
                'k:{"ip":"192.168.243.225"}':
                  .: {}
                  'f:ip': {}
              'f:startTime': {}
          manager: kubelet
          operation: Update
          subresource: status
          time: '2022-07-07T15:57:08Z'
      name: flyte-pod-webhook-dcbfbdf9-p5j6g
      namespace: flyte
      ownerReferences:
        - apiVersion: apps/v1
          blockOwnerDeletion: true
          controller: true
          kind: ReplicaSet
          name: flyte-pod-webhook-dcbfbdf9
          uid: 1098743a-7e43-449b-8d8e-5e02595325a7
      resourceVersion: '210870974'
      uid: abbad89c-13ef-410c-be39-bafb3a9db183
    spec:
      containers:
        - args:
            - webhook
            - '--config'
            - /etc/flyte/config/*.yaml
          command:
            - flytepropeller
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
          image: '<http://cr.flyte.org/flyteorg/flytepropeller:v1.1.0|cr.flyte.org/flyteorg/flytepropeller:v1.1.0>'
          imagePullPolicy: IfNotPresent
          name: webhook
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /etc/flyte/config
              name: config-volume
              readOnly: true
            - mountPath: /etc/webhook/certs
              name: webhook-certs
              readOnly: true
            - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
              name: kube-api-access-454tj
              readOnly: true
      dnsPolicy: ClusterFirst
      enableServiceLinks: true
      initContainers:
        - args:
            - webhook
            - init-certs
            - '--config'
            - /etc/flyte/config/*.yaml
          command:
            - flytepropeller
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  apiVersion: v1
                  fieldPath: metadata.namespace
          image: '<http://cr.flyte.org/flyteorg/flytepropeller:v1.1.0|cr.flyte.org/flyteorg/flytepropeller:v1.1.0>'
          imagePullPolicy: IfNotPresent
          name: generate-secrets
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
            - mountPath: /etc/flyte/config
              name: config-volume
            - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
              name: kube-api-access-454tj
              readOnly: true
      nodeName: ip-10-1-2-146.us-east-2.compute.internal
      nodeSelector:
        role: flyte
      preemptionPolicy: PreemptLowerPriority
      priority: 0
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext:
        fsGroup: 65534
        fsGroupChangePolicy: Always
        runAsUser: 1001
      serviceAccount: flyte-pod-webhook
      serviceAccountName: flyte-pod-webhook
      terminationGracePeriodSeconds: 30
      tolerations:
        - effect: NoExecute
          key: <http://node.kubernetes.io/not-ready|node.kubernetes.io/not-ready>
          operator: Exists
          tolerationSeconds: 300
        - effect: NoExecute
          key: <http://node.kubernetes.io/unreachable|node.kubernetes.io/unreachable>
          operator: Exists
          tolerationSeconds: 300
      volumes:
        - configMap:
            defaultMode: 420
            name: flyte-propeller-config-v001
          name: config-volume
        - name: webhook-certs
          secret:
            defaultMode: 420
            secretName: flyte-pod-webhook-v000
        - name: kube-api-access-454tj
          projected:
            defaultMode: 420
            sources:
              - serviceAccountToken:
                  expirationSeconds: 3607
                  path: token
              - configMap:
                  items:
                    - key: ca.crt
                      path: ca.crt
                  name: kube-root-ca.crt
              - downwardAPI:
                  items:
                    - fieldRef:
                        apiVersion: v1
                        fieldPath: metadata.namespace
                      path: namespace
    Ketan (kumare3)

    Ketan (kumare3)

    2 months ago
    Hmm
    @Katrina P did you get it deployed
    k

    Katrina P

    2 months ago
    I did, working through some last ingress / communication bugs but hopefully this is the home stretch
    Ketan (kumare3)

    Ketan (kumare3)

    2 months ago
    I am rooting for you - go!!!