https://flyte.org logo
#flyte-deployment
Title
# flyte-deployment
k

krishna Yerramsetty

06/28/2022, 9:30 PM
Hello all. I am trying to set resources for a task using
Copy code
@task(requests=Resources(cpu="8"))
But when I try registering the files I get the following error:
Copy code
Requested CPU default [8] is greater than current limit set in the platform configuration [2]
How and where do I change the default platform configuration? I am running on an AWS EKS cluster with a minimum compute group node size of 2, desired size of 2, and maximum of 24. Each node is a 16 cpu EC2 instance
y

Yee

06/28/2022, 9:51 PM
this is available in the admin config
you may also need to configure the section below that, which controls the namespace level quotas on the cluster.
this page may help as well
👍 1
k

krishna Yerramsetty

06/28/2022, 10:10 PM
Thank you, @Yee!
k

Ketan (kumare3)

06/28/2022, 10:33 PM
cc @Smriti Satyan can we please doc this
1
👀 1
s

Smriti Satyan

06/29/2022, 1:47 AM
Yes Ketan
k

krishna Yerramsetty

06/30/2022, 11:06 PM
@Yee, one more question: When using map tasks within a dynamic task, I only see one pod running at any time. I was expecting all the map tasks to be spun up simultaneously. Is that not so? How do I achieve parallelization?
k

Ketan (kumare3)

07/01/2022, 12:19 AM
They should
Can you share the snippet
k

krishna Yerramsetty

07/01/2022, 4:35 PM
Hey @Ketan (kumare3). I think I figured out what was happening. I had to increase the projectQuota values in the yaml file. I bumped up the projectQoutaCpu and projectQuotaMemory in my production env below:
Copy code
# -- Configuration for the Cluster resource manager component. This is an optional component, that enables automatic
# cluster configuration. This is useful to set default quotas, manage namespaces etc that map to a project/domain
cluster_resource_manager:
  # -- Enables the Cluster resource manager component
  enabled: true
  # -- Starts the cluster resource manager in standalone mode with requisite auth credentials to call flyteadmin service endpoints
  standalone_deploy: false
  config:
    cluster_resources:
      customData:
        - production:
            - projectQuotaCpu:
                value: "300"
            - projectQuotaMemory:
                value: "600Gi"
            - defaultIamRole:
                value: "arn:aws:iam::{{ .Values.userSettings.accountNumber }}:role/flyte-user-role"
        - staging:
            - projectQuotaCpu:
                value: "8"
            - projectQuotaMemory:
                value: "3000Mi"
            - defaultIamRole:
                value: "arn:aws:iam::{{ .Values.userSettings.accountNumber }}:role/flyte-user-role"
        - development:
            - projectQuotaCpu:
                value: "8"
            - projectQuotaMemory:
                value: "3000Mi"
            - defaultIamRole:
                value: "arn:aws:iam::{{ .Values.userSettings.accountNumber }}:role/flyte-user-role"
That I think should hopefully solve the problem
k

Ketan (kumare3)

07/01/2022, 5:47 PM
👍
26 Views