https://flyte.org logo
Powered by
#ask-the-community
Title
# ask-the-community
s

Super Bo

06/18/2023, 3:49 PM
Hi, testing Flyte in EKS, I can register the workflows but can’t run it. When execution workflow, my pod returned this error. The docs saying something about Flyte User Role but I don’t know where to assign it. I used flyte-binary helm chart to deploy flyte.
Copy code
Failed to get data from \<ns3://meta-bucket/flytesnacks/development/HF4IUC4ZOJ77UBKLLP77H4SY24======/fastf>\na252ae25deeecc9b853db87173e84cb.tar.gz
          to /root/ (recursive=False).\n\nOriginal exception: Access Denied\n"
d

David Espejo (he/him)

06/18/2023, 4:01 PM
Hi @Super Bo and welcome to the Flyte community Typically on EKS, the recommendation is to use IRSA Here's a community-maintained guide that gives more details https://github.com/davidmirror-ops/flyte-the-hard-way/blob/main/docs/03-roles-service-accounts.md
s

Super Bo

06/18/2023, 4:20 PM
Hi @David Espejo (he/him), I already followed that guide. And the result is Flyte successfully uploaded metadata to that s3 bucket. However, when we run Workflow in another namespace (not the flyte namespace), pod in that namespace doesn’t attached to any service account, how can they (pods) know which info (role name, service account secret) will use to assume the IAM role to download artifact from that S3 bucket. As you can see in these lines, that role can only be assumed from 
flyte
namespace.
Copy code
"Condition": {
                "StringEquals": {
                    "<http://oidc.eks.us-east-1.amazonaws.com/id/<UUID-OIDC>:sub|oidc.eks.us-east-1.amazonaws.com/id/<UUID-OIDC>:sub>": "system:serviceaccount:flyte:flyte-backend-flyte-binary"
                }
            }
d

David Espejo (he/him)

06/18/2023, 5:21 PM
when you register you should be able to specify the SA to use: 
pyflyte register ... --service-account
or you could specify it as the default for the domain:
Copy code
flytectl update workflow-execution-config -p <your-project> --attrFile security_context.yaml

cat security_context.yaml >>
project: <your-project>
domain: ""
max_parallelism: 25
security_context:
  run_as:
    k8s_service_account: <your-SA>
The guide will be updated to reflect this necessary step, any contribution also to the Flyte docs is welcome
s

Super Bo

06/18/2023, 5:27 PM
thank you, I will try these steps tomorrow and report later. I tried setting default IAM via 
configuration.inline
like in this file https://github.com/flyteorg/flyte/blob/master/charts/flyte-binary/eks-production.yaml. But I think that doesn’t help much. Thank you!
Hi @David Espejo (he/him), I can change Service Account, but, how can I change IAM Role, it still using default role, which is admin Role, I think.
Hi @David Espejo (he/him), after setting security context, I can’t launch workflow anymore.
Copy code
Request failed with status code 500 failed to create workflow in propeller <http://FlyteWorkflow.flyte.lyft.com|FlyteWorkflow.flyte.lyft.com> "atsv5q7rp97m5fdg5kqs" is invalid: [metadata.annotations: Invalid value: "": name part must be non-empty, metadata.annotations: Invalid value: "": name part must consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyName', or 'my.name', or '123-abc', regex used for validation is '([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]')]
Hmm, annotate default service account also works. Appreciate your help.
Copy code
kubectl annotate serviceaccount -n flytesnacks-development default <http://eks.amazonaws.com/role-arn=arn:aws:iam::<account_id>:role/<role_name|eks.amazonaws.com/role-arn=arn:aws:iam::<account_id>:role/<role_name>>
d

David Espejo (he/him)

06/20/2023, 11:49 AM
right @Super Bo the service account to be used (especially in the context of IRSA) should be annotated with the corresponding IAM role
4 Views
Powered by