Is this line in the docs correct? https://docs.flyte.org/en/latest/deployment/cluster_config/general.html

In the absence of an override, the global default values in the FlyteAdmin config are used.

https://github.com/flyteorg/flyte/blob/1e3d515550cb338c2edb3919d79c6fa1f0da5a19/charts/flyte-core/values.yaml#L35,L43 this looked like the resource requests/limits for the flyteadmin pods themselves, is flyte actually using those as the defaults for tasks? I just tried updating

task_resource_defaults

in the helm chart w/ new limits and I didn't see the change reflected in the configmap after applying the changes to the cluster

you mean the flyteadmin configmap didn't change after helm upgrading?

Yes, sorry -- the changes weren't reflected in the

flyte-admin-base-config

ConfigMap

ah i think you need to update: https://github.com/flyteorg/flyte/blob/1e3d515550cb338c2edb3919d79c6fa1f0da5a19/charts/flyte-core/values.yaml#L520

So that's what we're overriding right now: These are our overrides in

values.yaml

task_resource_defaults:
      task_resources:
        defaults:
          cpu: 1
          memory: 1Gi
          storage: 1Gi
        limits:
          cpu: 10
          memory: 50Gi
          storage: 2Gi

Yet this is what's reflected in the configmap after a helm upgrade:

task_resources:
  defaults:
    cpu: 100m
    memory: 500Mi
    storage: 500Mi
  limits:
    cpu: 2
    gpu: 1
    memory: 1Gi
    storage: 20Mi

the link you sent in the first message points to a different config block though

Sorry, think there are two separate issues I may have conflated: 1. The docs appear to suggest modifying the flyte admin configuration to set global task resource (which isn't correct as we're discussing) 2. I think we were able to resolve the issue actually, I had copied the eks values.yaml from the flyte repo, but our chart had a small bug

ah great! would you be interested in opening a docs PR to point to the correct configuration? 😄 if not totally okay, i'll file an issue to track

Yep happy to do so! I think a broader issue to think through is the fact that the helm upgrade appeared to succeed but failed silently due to the indentation problem

oh, that is certainly not great. i wonder if there's a helm "strict mode" or equivalent

Also the indentation issue also led to us having only 4 workers instead of 40 in propeller...fixing that is leading to much better perf

wow!

On my end 😅