Hey guys, I'm trying to configure a parameter in flyte propeller. specifically

create-container-error-grace-period

which people seemed to have no problem setting: https://discuss.flyte.org/t/22699844/hello-recently-our-flyte-cluster-v1-12-0-has-been-throwing-s. I set it to the best of my knowledge in my flyte propeller configmap, but it's not taking effect, even after restarting flytepropeller we're seeing tasks fail after the default

3m0s

even though we set it to

10m0s

. Here is exactly where we configure it: (based on this doc: https://www.union.ai/docs/flyte/deployment/configuration-reference/flytepropeller-config, see screenshot) (flyte-propeller-config configmap)

data:
  k8s.yaml: |
    plugins:
      k8s:
        create-container-error-grace-period: 10m0s

How do we troubleshoot this?

@high-park-82026 @average-finland-92144 any suggestions here?

one trick to make sure the config has been applied is:

kubectl port-forward -n flyte deploy/flytepropeller 10254

Then open the browser to

<http://localhost:10254/config>

You should see the setting there, if it's not then it wasn't properly passed. Happy to dig deeper if you've not solved this problem yet (I realize that was almost two weeks ago now)

Thanks a lot.. we’ll circle back

I checked the config and it was set right, but i incidentally glanced another config

image-pull-backoff-grace-period

which doesn't seem to be documented. but ill try to set that

oh, if the pods are failing to pull images in 3 minutes or less then yes, you could adjust that setting