Hello all. I am trying to set resources for a task using

@task(requests=Resources(cpu="8"))

But when I try registering the files I get the following error:

Requested CPU default [8] is greater than current limit set in the platform configuration [2]

How and where do I change the default platform configuration? I am running on an AWS EKS cluster with a minimum compute group node size of 2, desired size of 2, and maximum of 24. Each node is a 16 cpu EC2 instance

this is available in the admin config

Thank you, @Yee!

cc @Smriti Satyan can we please doc this

Yes Ketan

@Yee, one more question: When using map tasks within a dynamic task, I only see one pod running at any time. I was expecting all the map tasks to be spun up simultaneously. Is that not so? How do I achieve parallelization?

They should

Hey @Ketan (kumare3). I think I figured out what was happening. I had to increase the projectQuota values in the yaml file. I bumped up the projectQoutaCpu and projectQuotaMemory in my production env below:

# -- Configuration for the Cluster resource manager component. This is an optional component, that enables automatic
# cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain
cluster_resource_manager:
  # -- Enables the Cluster resource manager component
  enabled: true
  # -- Starts the cluster resource manager in standalone mode with requisite auth credentials to call flyteadmin service endpoints
  standalone_deploy: false
  config:
    cluster_resources:
      customData:
        - production:
            - projectQuotaCpu:
                value: "300"
            - projectQuotaMemory:
                value: "600Gi"
            - defaultIamRole:
                value: "arn:aws:iam::{{ .Values.userSettings.accountNumber }}:role/flyte-user-role"
        - staging:
            - projectQuotaCpu:
                value: "8"
            - projectQuotaMemory:
                value: "3000Mi"
            - defaultIamRole:
                value: "arn:aws:iam::{{ .Values.userSettings.accountNumber }}:role/flyte-user-role"
        - development:
            - projectQuotaCpu:
                value: "8"
            - projectQuotaMemory:
                value: "3000Mi"
            - defaultIamRole:
                value: "arn:aws:iam::{{ .Values.userSettings.accountNumber }}:role/flyte-user-role"

That I think should hopefully solve the problem

👍