Hi all, I've deployed an on-premise kubernetes cluster and have installed flyte using helm to my cluster, now when I was trying to register my workflow through

pyflyte register

, it throws error saying,

RPC Failed, with Status: StatusCode.INVALID_ARGUMENT
        details: Requested MEMORY default [20Gi] is greater than current limit set in the platform configuration [2Gi]. Please contact Flyte Admins to change these limits or consult the configuration
        Debug string UNKNOWN:Error received from peer ipv4:127.0.0.1:8089 {grpc_message:"Requested MEMORY default [20Gi] is greater than current limit set in the platform configuration [2Gi]. Please contact Flyte Admins to change these limits or consult the configuration", grpc_status:3, created_time:"2023-09-29T17:36:51.296741384+05:30"}

seems like there was a request limit in platform config, so I've updated the

values.yaml

file which contains the override values when installing via

helm install "" --values "values.yaml"

to,

configuration:
  database:
    username: postgres
    host: postgres.flyte.svc.cluster.local
    dbname: flyteadmin

  storage:
    type: minio
    metadataContainer: "" 
    userDataContainer: ""
    provider: s3
    providerConfig:
      s3:
        region: "us-east-1" #Irrelevant for local but still needed
        authType: ""
        endpoint: "<http://minio.flyte.svc.cluster.local:9000>"
        accessKey: ""
        secretKey: "" 
        disableSSL: "true"
        secure: "false"

  inlineSecretRef: flyte-binary-inline-config-secret

  inline:
    plugins:
      k8s:
        inject-finalizer: true
        default-env-vars:
          - FLYTE_AWS_ENDPOINT: "<http://minio.flyte.svc.cluster.local:9000>"
          - FLYTE_AWS_ACCESS_KEY_ID: ""
          - FLYTE_AWS_SECRET_ACCESS_KEY: "" 

deployment:
  resources:
    limits:
      cpu: 64
      gpu: 16
      memory: 32Gi
    requests:
      cpu: 8
      gpu: 4
      memory: 16Gi

serviceAccount:
  create: true

got the

values.yaml

and

deployment steps

from @average-finland-92144’s flyte-the-hard-way , now how can I update the request limit of the deployment to my desired range!? Need help with the configuration settings.

Hey Durgai The right place to configure platform resource defaults is under the

inline

section of the values file, as indicated here: https://github.com/davidmirror-ops/flyte-the-hard-way/blob/82fcb0ca1524f837df348b71058dfa6ea39160a4/docs/on-premises/local-values.yaml#L30-L35

@average-finland-92144, still getting the same error, also I have a doubt, does it cache the inputs it's been passed with, coz previously I've changed the default bucked in minio to a value, but when I reverted back and started the cluster again, it uses the previous value as the default bucket inside the flyte-binary and throws

url unreachable

where the url is pointing to the previously used bucket name. Since this method didn't worked I used this file as reference and updated the

local-values.yaml

.

That section in the file indicates resources for the K8s Deployment that governs the flyte-binary Pod, that's not for Tasks. The error shows a request that's higher to the limit. Can you describe the

flyte-binary

configmap?

@average-finland-92144, I hope by config map, you were saying the

values.yaml

which is used when we install the

flyte-binary

using helm, so

configuration:
  database:
    username: postgres
    host: postgres.flyte.svc.cluster.local
    dbname: flyteadmin

  storage:
    type: minio
    metadataContainer: my-s3-bucket 
    userDataContainer: my-s3-bucket
    provider: s3
    providerConfig:
      s3:
        region: "us-east-1" #Irrelevant for local but still needed
        authType: "accesskey"
        endpoint: "<http://minio.flyte.svc.cluster.local:9000>"
        accessKey: ""
        secretKey: ""
        disableSSL: "true"
        secure: "false"

  logging:
    level: 7
    plugins:
      kubernetes:
        enabled: true

  inlineSecretRef: flyte-binary-inline-config-secret

configuration
  inline:
    plugins:
      k8s:
        inject-finalizer: true
        default-env-vars:
          - FLYTE_AWS_ENDPOINT: "<http://minio.flyte.svc.cluster.local:9000>"
          - FLYTE_AWS_ACCESS_KEY_ID: ""
          - FLYTE_AWS_SECRET_ACCESS_KEY: "" #Use the same value as the MINIO_ROOT_PASSWORD
    task_resources:
      limits:
        cpu: 64
        gpu: 16
        memory: 32Gi
      defaults:
        cpu: 8
        gpu: 4
        memory: 16Gi

serviceAccount:
  create: true

or do you mean the

config.yaml

which contains the endpoint details and used by

pyflyte

or

flytectl

when registering or running workflows, if so,

admin:
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///localhost:8089
  insecure: true
  authType: Pkce
console:
  endpoint: <http://localhost:8088>
logger:
  show-source: true
  level: 7

no. Please check

kubectl get cm -n flyte

Whatever you add to the `values`file, Helm renders it, validate it and ends up being applied to a configmap. I just can't remember if it's

flyte-binary-config

. Then, if you do a

kubectl describe cm <configmap-name> -n flyte

you'll see the actual config that the Flyte Pod is consuming

@average-finland-92144 , sorry it was my fault, I’ve provided the wrong path for the file. It was on me. Also we are planning to use flyte for building a data pipeline, it would be more helpful we can discuss about some internal features and some best practices, which would help us to build a quality pipeline with the most out of it.