https://flyte.org logo
#ask-the-community
Title
# ask-the-community
d

Durgai Vel

09/29/2023, 12:15 PM
Hi all, I've deployed an on-premise kubernetes cluster and have installed flyte using helm to my cluster, now when I was trying to register my workflow through
pyflyte register
, it throws error saying,
Copy code
RPC Failed, with Status: StatusCode.INVALID_ARGUMENT
        details: Requested MEMORY default [20Gi] is greater than current limit set in the platform configuration [2Gi]. Please contact Flyte Admins to change these limits or consult the configuration
        Debug string UNKNOWN:Error received from peer ipv4:127.0.0.1:8089 {grpc_message:"Requested MEMORY default [20Gi] is greater than current limit set in the platform configuration [2Gi]. Please contact Flyte Admins to change these limits or consult the configuration", grpc_status:3, created_time:"2023-09-29T17:36:51.296741384+05:30"}
seems like there was a request limit in platform config, so I've updated the
values.yaml
file which contains the override values when installing via
helm install "" --values "values.yaml"
to,
Copy code
configuration:
  database:
    username: postgres
    host: postgres.flyte.svc.cluster.local
    dbname: flyteadmin

  storage:
    type: minio
    metadataContainer: "" 
    userDataContainer: ""
    provider: s3
    providerConfig:
      s3:
        region: "us-east-1" #Irrelevant for local but still needed
        authType: ""
        endpoint: "<http://minio.flyte.svc.cluster.local:9000>"
        accessKey: ""
        secretKey: "" 
        disableSSL: "true"
        secure: "false"

  inlineSecretRef: flyte-binary-inline-config-secret

  inline:
    plugins:
      k8s:
        inject-finalizer: true
        default-env-vars:
          - FLYTE_AWS_ENDPOINT: "<http://minio.flyte.svc.cluster.local:9000>"
          - FLYTE_AWS_ACCESS_KEY_ID: ""
          - FLYTE_AWS_SECRET_ACCESS_KEY: "" 

deployment:
  resources:
    limits:
      cpu: 64
      gpu: 16
      memory: 32Gi
    requests:
      cpu: 8
      gpu: 4
      memory: 16Gi

serviceAccount:
  create: true
got the
values.yaml
and
deployment steps
from @David Espejo (he/him)’s flyte-the-hard-way , now how can I update the request limit of the deployment to my desired range!? Need help with the configuration settings.
d

David Espejo (he/him)

09/29/2023, 1:08 PM
Hey Durgai The right place to configure platform resource defaults is under the
inline
section of the values file, as indicated here: https://github.com/davidmirror-ops/flyte-the-hard-way/blob/82fcb0ca1524f837df348b71058dfa6ea39160a4/docs/on-premises/local-values.yaml#L30-L35
d

Durgai Vel

09/29/2023, 1:30 PM
@David Espejo (he/him), still getting the same error, also I have a doubt, does it cache the inputs it's been passed with, coz previously I've changed the default bucked in minio to a value, but when I reverted back and started the cluster again, it uses the previous value as the default bucket inside the flyte-binary and throws
url unreachable
where the url is pointing to the previously used bucket name. Since this method didn't worked I used this file as reference and updated the
local-values.yaml
.
d

David Espejo (he/him)

09/29/2023, 1:40 PM
That section in the file indicates resources for the K8s Deployment that governs the flyte-binary Pod, that's not for Tasks. The error shows a request that's higher to the limit. Can you describe the
flyte-binary
configmap?
d

Durgai Vel

09/29/2023, 1:56 PM
@David Espejo (he/him), I hope by config map, you were saying the
values.yaml
which is used when we install the
flyte-binary
using helm, so
Copy code
configuration:
  database:
    username: postgres
    host: postgres.flyte.svc.cluster.local
    dbname: flyteadmin

  storage:
    type: minio
    metadataContainer: my-s3-bucket 
    userDataContainer: my-s3-bucket
    provider: s3
    providerConfig:
      s3:
        region: "us-east-1" #Irrelevant for local but still needed
        authType: "accesskey"
        endpoint: "<http://minio.flyte.svc.cluster.local:9000>"
        accessKey: ""
        secretKey: ""
        disableSSL: "true"
        secure: "false"

  logging:
    level: 7
    plugins:
      kubernetes:
        enabled: true

  inlineSecretRef: flyte-binary-inline-config-secret

configuration
  inline:
    plugins:
      k8s:
        inject-finalizer: true
        default-env-vars:
          - FLYTE_AWS_ENDPOINT: "<http://minio.flyte.svc.cluster.local:9000>"
          - FLYTE_AWS_ACCESS_KEY_ID: ""
          - FLYTE_AWS_SECRET_ACCESS_KEY: "" #Use the same value as the MINIO_ROOT_PASSWORD
    task_resources:
      limits:
        cpu: 64
        gpu: 16
        memory: 32Gi
      defaults:
        cpu: 8
        gpu: 4
        memory: 16Gi

serviceAccount:
  create: true
or do you mean the
config.yaml
which contains the endpoint details and used by
pyflyte
or
flytectl
when registering or running workflows, if so,
Copy code
admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///localhost:8089
  insecure: true
  authType: Pkce
console:
  endpoint: <http://localhost:8088>
logger:
  show-source: true
  level: 7
d

David Espejo (he/him)

09/29/2023, 2:01 PM
no. Please check
kubectl get cm -n flyte
Whatever you add to the `values`file, Helm renders it, validate it and ends up being applied to a configmap. I just can't remember if it's
flyte-binary-config
. Then, if you do a
kubectl describe cm <configmap-name> -n flyte
you'll see the actual config that the Flyte Pod is consuming
d

Durgai Vel

09/29/2023, 5:09 PM
@David Espejo (he/him) , sorry it was my fault, I’ve provided the wrong path for the file. It was on me. Also we are planning to use flyte for building a data pipeline, it would be more helpful we can discuss about some internal features and some best practices, which would help us to build a quality pipeline with the most out of it.
5 Views