Hi, testing Flyte in EKS, I can register the workf...
# flyte-supportf
flat-minister-9355
06/18/2023, 3:49 PMHi, testing Flyte in EKS, I can register the workflows but can’t run it. When execution workflow, my pod returned this error. The docs saying something about Flyte User Role but I don’t know where to assign it. I used flyte-binary helm chart to deploy flyte.
Copy code
Failed to get data from \<ns3://meta-bucket/flytesnacks/development/HF4IUC4ZOJ77UBKLLP77H4SY24======/fastf>\na252ae25deeecc9b853db87173e84cb.tar.gz
to /root/ (recursive=False).\n\nOriginal exception: Access Denied\n"a
average-finland-92144
06/18/2023, 4:01 PMHi @flat-minister-9355 and welcome to the Flyte community
Typically on EKS, the recommendation is to use IRSA
Here's a community-maintained guide that gives more details
https://github.com/davidmirror-ops/flyte-the-hard-way/blob/main/docs/03-roles-service-accounts.md
f
flat-minister-9355
06/18/2023, 4:20 PMHi @average-finland-92144, I already followed that guide. And the result is Flyte successfully uploaded metadata to that s3 bucket. However, when we run Workflow in another namespace (not the flyte namespace), pod in that namespace doesn’t attached to any service account, how can they (pods) know which info (role name, service account secret) will use to assume the IAM role to download artifact from that S3 bucket.
As you can see in these lines, that role can only be assumed from
flyte Copy code
"Condition": {
"StringEquals": {
"<http://oidc.eks.us-east-1.amazonaws.com/id/<UUID-OIDC>:sub|oidc.eks.us-east-1.amazonaws.com/id/<UUID-OIDC>:sub>": "system:serviceaccount:flyte:flyte-backend-flyte-binary"
}
}a
average-finland-92144
06/18/2023, 5:21 PMwhen you register you should be able to specify the SA to use:
pyflyte register ... --service-account👍 1
average-finland-92144
06/18/2023, 5:24 PMor you could specify it as the default for the domain:
Copy code
flytectl update workflow-execution-config -p <your-project> --attrFile security_context.yaml
cat security_context.yaml >>
project: <your-project>
domain: ""
max_parallelism: 25
security_context:
run_as:
k8s_service_account: <your-SA>👍 1
average-finland-92144
06/18/2023, 5:25 PMThe guide will be updated to reflect this necessary step, any contribution also to the Flyte docs is welcome
f
flat-minister-9355
06/18/2023, 5:27 PMthank you, I will try these steps tomorrow and report later.
I tried setting default IAM via
configuration.inlineflat-minister-9355
06/19/2023, 1:31 AMHi @average-finland-92144, I can change Service Account, but, how can I change IAM Role, it still using default role, which is admin Role, I think.
flat-minister-9355
06/19/2023, 5:05 AMHi @average-finland-92144, after setting security context, I can’t launch workflow anymore.
Copy code
Request failed with status code 500 failed to create workflow in propeller <http://FlyteWorkflow.flyte.lyft.com|FlyteWorkflow.flyte.lyft.com> "atsv5q7rp97m5fdg5kqs" is invalid: [metadata.annotations: Invalid value: "": name part must be non-empty, metadata.annotations: Invalid value: "": name part must consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyName', or 'my.name', or '123-abc', regex used for validation is '([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9]')]flat-minister-9355
06/19/2023, 6:05 AMHmm, annotate default service account also works.
Appreciate your help.
Copy code
kubectl annotate serviceaccount -n flytesnacks-development default <http://eks.amazonaws.com/role-arn=arn:aws:iam::<account_id>:role/<role_name|eks.amazonaws.com/role-arn=arn:aws:iam::<account_id>:role/<role_name>>🙌🏽 1
a
average-finland-92144
06/20/2023, 11:49 AMright @flat-minister-9355 the service account to be used (especially in the context of IRSA) should be annotated with the corresponding IAM role
168 Views