e

    Edgar Trujillo

    3 months ago
    Hi - I noticed helm treated our
    accountNumber
    as a float in all annotation related lines in values-eks.yaml
    <http://eks.amazonaws.com/role-arn|eks.amazonaws.com/role-arn>: arn:aws:iam::{{ .Values.userSettings.accountNumber }}:role/iam-role-flyte
    I changed the lines to explicitly define the arn and can now schedule executions to run
    remotely
    But the console shows no actual executions and the role assigned is the
    default
    rather than 1 of the 2 roles created (
    iam-role-flyte
    ,
    flyte-user-role
    ) Doing a describe on the flytepropeller shows
    Environment:
          POD_NAME:                     flytepropeller-7d67df4c85-7sf4v (v1:metadata.name)
          POD_NAMESPACE:                flyte (v1:metadata.namespace)
          AWS_STS_REGIONAL_ENDPOINTS:   regional
          AWS_DEFAULT_REGION:           us-east-1
          AWS_REGION:                   us-east-1
          AWS_ROLE_ARN:                 arn:aws:iam::1.AWS_ACCTe+11:role/iam-role-flyte
          AWS_WEB_IDENTITY_TOKEN_FILE:  /var/run/secrets/eks.amazonaws.com/serviceaccount/token
    Any idea where the
    AWS_ROLE_ARN
    is being mounted from?
    Nastya Rusina

    Nastya Rusina

    3 months ago
    @Jason Porter
    e

    Edgar Trujillo

    3 months ago
    I resolved it by manually editing the deployment and specifying the
    AWS_ROLE_ARN
    env value. I saw a similar botched
    AWS_ROLE_ARN
    in the DataCatalog deployment and after updating that i now get
    E0621 17:56:06.090698       1 workers.go:102] error syncing 'flytesnacks-development/amp8cm9q2fbrdn84nvcd': failed at Node[start-node]. CausedByError: Failed to store workflow inputs (as start node), caused by: Failed to write data [61b] to path [metadata/propeller/flytesnacks-development-amp8cm9q2fbrdn84nvcd/start-node/data/0/outputs.pb].: PutObject, putting object: AccessDenied: Access Denied
    	status code: 403, request id: HXTEMC45YC2H49GF, host id: zkybrDtNFRW0TBe5hbBwiWjiSwdXQJIWAEG6al3gKOnCojFSEPmmY3/GB1H8QYbDgncp6xRpDfw=
    which is strange since these paths exists in the s3 •
    metadata/flytesnacks/development/amp8cm9q2fbrdn84nvcd/inputs
    metadata/flytesnacks/development/amp8cm9q2fbrdn84nvcd/user_input
    I realized the root error was in my
    flytepropeller
    deployment. While I manually added:
    - name: AWS_ROLE_ARN
      value: arn:aws:iam::ACCCOUNT:role/iam-role-flyte
    I forgot to add:
    - name: AWS_WEB_IDENTITY_TOKEN_FILE
      value: /var/run/secrets/eks.amazonaws.com/serviceaccount/token
    FWIW, I deployed this with
    tf
    and
    helm
    onto a private EKS and I think using
    tf
    may have caused the initial botched
    AWS_ROLE_ARN
    Ketan (kumare3)

    Ketan (kumare3)

    3 months ago
    Ohh @Edgar Trujillo can you please recommend a doc edit - cc @Marc Paquette