Hey team, I am running tasks on AWS batch. I am wo...
# flyte-deployment
j
Hey team, I am running tasks on AWS batch. I am wondering do I always have to explicitly specify the IAM role? Is there a way to omit it? The default role to run the workflow is the same when it runs on batch.
k
By default Flyte is using the serviceaccount. It is not possible to convert from service accounts to role arn (afaik) - but you should be able to set a project level default to always use for every execution . Cc @Yee
g
What form of config or flytectl command does this "project level default" take? We were looking at something similar for a IAM role passed to SageMaker and were considering either workflow params set in a launchplan or a ConfigMap created in the cluster-resources with the role ARN populated via cluster-resource-attributes. Are we missing another option?
We're looking at how to serialize the workflow once and register it with flyte instances in multiple AWS accounts, so want parameterize things that will vary account to account.
y
sorry but this feature is not out yet. merging soon I hope. currently you can only set them on a project and domain basis - not for the whole project.
it’ll be a couple extra steps but it will do the same thing
project level to be out shortly
g
what does the current project+domain feature look like? Is this
cluster-resource-attributes
or something else?
y
Copy code
flytectl --config ~/.flyte/config.yaml update workflow-execution-config --attrFile wec.yaml
where wec.yaml looks like
Copy code
domain: staging
project: flytesnacks
security_context:
  run_as:
    k8s_service_account: default
ytong@argus:/opt/local
@Jacob Wang you can associate an eks service account with an iam role. take a look at the instructions here: https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts.html
g
hmm, maybe I misunderstood and co-opted this thread in the wrong direction. 🙂 I thought we were talking about setting an IAM role ARN that could be passed to AWS services by a workflow, not setting the ServiceAccount.
y
after you do that, you will see this
Copy code
apiVersion: v1
kind: ServiceAccount
metadata:
  annotations:
    <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: arn:aws:iam::xyz:role/development-service-flyte-userflyterole
  creationTimestamp: "2021-07-20T23:35:31Z"
  name: default
  namespace: flytesnacks-development
  resourceVersion: "45171"
  uid: 4ef4157b-4092-4247-a21a-147c2f9e5da5
secrets:
- name: default-token-8vxf2
there are cases where you can set both, but we internally set the service account and let EKS handle the auth.
i don’t entirely remember what happens if you try to set the iam role directly in that same flytectl command. @Ketan (kumare3)?
we recommend going through eks/aws for the auth.
k
hmm the IAM role is to pass to a downstream service like AWS Batch
I think we should support it in project/domain level defaults, if not already
cc @Geoff Salmon / @Jacob Wang
g
Are "project/domain level defaults" set by a flytectl command like
workflow-execution-config
or
cluster-resource-attributes
? I think I'm just not familar enough with all the config options to know if you're referencing a specific feature or not.
k
yes
sorry for the delayed response
162 Views