https://flyte.org logo
#ask-the-community
Title
# ask-the-community
a

Ariel Kaspit

06/04/2023, 9:40 AM
TLDR; Configuring cluster resources in Helm chart (on GCP) - specific for permissions with IAM service accounts Hi! I have some issues with configuring cluster resources for IAM permissions (in Console and CLI) I deployed Flyte helm chart on GCP and configured
cluster_resource_manager
as documented, but still getting permissions errors. I followed the documentation, specifically in here: https://docs.flyte.org/en/latest/deployment/configuration/general.html#cluster-resources This is my configuration in `values.yaml`:
Copy code
configmap:
  domain:
    domains:
      - id: development
        name: development
      - id: staging
        name: staging
  namespace_config:
    namespace_mapping:
      template: "{{ domain }}"
cluster_resource_manager:
  config:
    cluster_resources:
      customData:
        - development:
          - projectQuotaCpu:
              value: "5"
          - projectQuotaMemory:
              value: 4000Mi
          - gsa:
              value: <mailto:flyte@projectid.iam.gserviceaccount.com|flyte@projectid.iam.gserviceaccount.com>
        - staging:
          - projectQuotaCpu:
              value: "5"
          - projectQuotaMemory:
              value: 4000Mi
          - gsa:
              value: <mailto:flyte@projectid.iam.gserviceaccount.com|flyte@projectid.iam.gserviceaccount.com>
  templates:
    - key: aa_namespace
      value: |
        apiVersion: v1
        kind: Namespace
        metadata:
          name: {{ namespace }}
        spec:
          finalizers:
          - kubernetes
    - key: aab_default_service_account
      value: |
        apiVersion: v1
        kind: ServiceAccount
        metadata:
          name: default
          namespace: {{ namespace }}
          annotations:
            <http://iam.gke.io/gcp-service-account|iam.gke.io/gcp-service-account>: <mailto:flyte@projectid.iam.gserviceaccount.com|flyte@projectid.iam.gserviceaccount.com>
    - key: ab_project_resource_quota
      value: |
        apiVersion: v1
        kind: ResourceQuota
        metadata:
          name: project-quota
          namespace: {{ namespace }}
        spec:
          hard:
            limits.cpu: {{ projectQuotaCpu }}
            limits.memory: {{ projectQuotaMemory }}
In console, I don’t see any IAM and serviceaccounts assigned to the project (attached a screenshot). Using
pyflyte
, I’m trying to run the
hello-world
workflow (I use the basic workflow just for testing, it’s from
flytesnacks/cookbook/core/flyte_basics/hello_world.py
) - and I get 403 Permissions denied. Is there something I need to configure in the workflow itself / in
./flyte/config.yaml
?
d

David Espejo (he/him)

06/05/2023, 3:57 AM
@Ariel Kaspit can you check if the corresponding ConfigMap is in place with the correct info? it should be:
kubectl describe cm flyte-clusterresourcesync-config -n <your-flyte-namespace>
There's another way to specify the service account running workflows should use depending on project/domains: https://docs.flyte.org/projects/flytectl/en/latest/gen/flytectl_update_workflow-execution-config.html#id1 But it's not the same outcome as doing it from the Helm chart
a

Ariel Kaspit

06/05/2023, 8:34 AM
@David Espejo (he/him) Thanks for the answer. Yes, the configmap is in place with this configuration:
Copy code
Data
====
domain.yaml:
----
domains:
- id: development
  name: development
- id: staging
  name: staging

namespace_config.yaml:
----
namespace_mapping:
  template: '{{ domain }}'

cluster_resources.yaml:
----
cluster_resources:
  customData:
  - development:
    - projectQuotaCpu:
        value: "5"
    - projectQuotaMemory:
        value: 4000Mi
    - gsa:
        value: <mailto:xxx@projectid.iam.gserviceaccount.com|xxx@projectid.iam.gserviceaccount.com>
  - staging:
    - projectQuotaCpu:
        value: "5"
    - projectQuotaMemory:
        value: 4000Mi
    - gsa:
        value: <mailto:xxx@projectid.iam.gserviceaccount.com|xxx@projectid.iam.gserviceaccount.com>
  refresh: 5m
  refreshInterval: 5m
  standaloneDeployment: false
  templatePath: /etc/flyte/clusterresource/templates

clusters.yaml:
----
clusters:
  clusterConfigs: []
  labelClusterMap: {}
d

David Espejo (he/him)

06/06/2023, 6:35 PM
@Ariel Kaspit can you try using
defaultIamRole
Copy code
cluster_resources:
      customData:
      - development:
        - defaultIamRole:
            value: <mailto:xxx@projectid.iam.gserviceaccount.com|xxx@projectid.iam.gserviceaccount.com>
instead of
gsa
?
a

Ariel Kaspit

06/07/2023, 8:21 AM
Still get the same. no change
I got the issue! When using
pyflyte/flytectl
it uses flyteadmin in order to get the gcs bucket. The error I got is:
details: failed to create a signed url. Error: unable to sign bytes: googleapi: Error 403: Permission 'iam.serviceAccounts.signBlob' denied on resource (or it may not exist).
I use workload identity (Flyte is running on GKE) and the access to the bucket is using a signed url. but this does not work with workload identity! So I created a key.json to the IAM serviceaccount and added an env var called
GCLOUD_OAUTH_CREDS
which contains the path to the key.json file. Then,
pyflyte/flytectl
work perfectly!
d

David Espejo (he/him)

06/08/2023, 2:37 PM
@Ariel Kaspit That's great and thanks for sharing!
78 Views