<@U06RKAMK0UC>, Flyteadmin error log {"json":{"src...
# ask-ai
v
@ancient-wolf-19325, Flyteadmin error log {"json":{"src":"service.go:142"},"level":"error","msg":"failed to create signed url. Error:%!(EXTRA *awserr.baseError=WebIdentityErr: failed to retrieve credentials\ncaused by: AccessDenied: Not authorized to perform sts:AssumeRoleWithWebIdentity\n\tstatus code: 403, request id: c6c61936-be49-473b-bbbe-4b9a214dc5bd)","ts":"2025-05-08T075055Z"}
j
This error means FlyteAdmin failed to assume the specified AWS IAM role using a web identity token, resulting in AccessDenied for sts:AssumeRoleWithWebIdentity. This typically happens if the Kubernetes service account used by FlyteAdmin is not properly annotated with the correct IAM role, or the IAM role's trust policy does not allow the service account to assume it. According to the documentation, you must annotate the service account with eks.amazonaws.com/role-arn and ensure the IAM role trust policy allows the OIDC provider and service account to assume the role (Union.ai Docs, Flyte on EKS Blog). Would you like step-by-step instructions to check and fix your service account and IAM role configuration? Sources: - Union.ai Docs - Flyte on EKS Blog 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.
v
@ancient-wolf-19325 yes
j
Step-by-step to resolve the sts:AssumeRoleWithWebIdentity AccessDenied error: 1. Check Service Account Annotation Ensure the Kubernetes service account used by FlyteAdmin is annotated with the correct IAM role ARN. Run:
Copy code
kubectl describe sa <flyteadmin-sa> -n <flyte-namespace>
You should see an annotation like:
Copy code
eks.amazonaws.com/role-arn: arn:aws:iam::<account-id>:role/<flyte-iam-role>
If missing, add it:
Copy code
kubectl annotate serviceaccount -n <flyte-namespace> <flyteadmin-sa> eks.amazonaws.com/role-arn=arn:aws:iam::<account-id>:role/<flyte-iam-role>
2. Verify IAM Role Trust Policy The IAM role must trust the EKS OIDC provider and the service account. The trust policy should include:
Copy code
{
  "Effect": "Allow",
  "Principal": {
    "Federated": "arn:aws:iam::<account-id>:oidc-provider/<oidc-provider>"
  },
  "Action": "sts:AssumeRoleWithWebIdentity",
  "Condition": {
    "StringEquals": {
      "<oidc-provider>:sub": "system:serviceaccount:<flyte-namespace>:<flyteadmin-sa>"
    }
  }
}
Update the trust policy in the AWS console if needed. 3. Check IAM Role Permissions The IAM role should have permissions for the required AWS actions (e.g., S3, STS). 4. Restart FlyteAdmin Pod After making changes, restart the FlyteAdmin pod to pick up the new credentials:
Copy code
kubectl delete pod -l app=flyteadmin -n <flyte-namespace>
For more details, see the Union.ai troubleshooting guide and Flyte on EKS blog. Sources: - Union.ai Docs - Flyte on EKS Blog 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.