hi @average-finland-92144 I’ve followed the EKS deployment guide https://docs.flyte.org/en/latest/deployment/deployment/cloud_simple.html and managed to spin up a cluster (with tons of fixes) but again when I test the sample workflows with

pyflyte run --remote hello_world.py my_wf

I get:

Failed with Exception Code: SYSTEM:Unknown
RPC Failed, with Status: StatusCode.UNAVAILABLE
	details: failed to connect to all addresses; last error: INTERNAL: ipv4:127.0.0.1:8088: Trying to connect an http1.x server
	Debug string UNKNOWN:failed to connect to all addresses; last error: INTERNAL: ipv4:127.0.0.1:8088: Trying to connect an http1.x server {created_time:"2023-08-17T00:00:47.690344+02:00", grpc_status:14}

Any clue?

Hi Enrico please change the port to 8089

I think I did try that and got this error:

Failed with Exception Code: SYSTEM:Unknown
RPC Failed, with Status: StatusCode.INTERNAL
	details: failed to create a signed url. Error: WebIdentityErr: failed to retrieve credentials
caused by: ValidationError: Request ARN is invalid
	status code: 400, request id: 26d08b4c-11ce-4f9c-850d-ac0b9ab764fa
	Debug string UNKNOWN:Error received from peer ipv6:%5B::1%5D:8089 {created_time:"2023-08-16T23:59:27.12566+02:00", grpc_status:13, grpc_message:"failed to create a signed url. Error: WebIdentityErr: failed to retrieve credentials\ncaused by: ValidationError: Request ARN is invalid\n\tstatus code: 400, request id: 26d08b4c-11ce-4f9c-850d-ac0b9ab764fa"}

I’ll double check later

are you using IAM Roles for Service Accounts?

hi @average-finland-92144 with

endpoint: dns:///localhost:8089

I get:

_InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.INTERNAL
        details = "failed to create a signed url. Error: WebIdentityErr: failed to retrieve credentials
caused by: ValidationError: Request ARN is invalid

The recommended approach is to use IRSA which requires a trust relationship and association between the Service Account, IAM Role, Policy and the EKS cluster itself. I hope these instructions are helpful for you: https://github.com/davidmirror-ops/flyte-the-hard-way/blob/main/docs/03-roles-service-accounts.md

hi @average-finland-92144 yes that link should be useful! gratitude thank you I’ll restart my deployment again from scratch because those instructions look different than the official ones: https://docs.flyte.org/en/latest/deployment/deployment/cloud_simple.html Let me just triple-check with you before I set out on a new endeavour. Are the instructions at https://github.com/davidmirror-ops/flyte-the-hard-way/ the best way to deploy Flyte on K8s? Meaning, most comprehensive and tested? The reason I’m asking is I’ve been going in circles for a while trying to follow the official docs lolcry 🫠 🐬 After I get this deployed. How to deploy my own custom Agent (see code here)? In previous attempts I’ve followed these official instructions but at this point I’m not 100% sure that’ll work with the

flyte-the-hard-way

approach. Can you point me to the best way on how to deploy my custom agents? - @steep-jackal-21573 since you’ve authored the docs page on this topic, can you share your thoughts? Thank you so much folks!

Can you or anybody on your team help me closely in deploying Flyte on K8s?

I can help you, and sorry for the inconveniences so far. 🙂

@adventurous-van-8963, I'm sorry if the documentation is unclear. If you find any mistakes, could you please create PRs to update the docs? Thanks!

I don't exactly know how different docs are organized as I merely contributed to the agent page going together with a PR change, and https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/extend_flyte/agent_service.html#update-flyteagent looks correct to me.

And @average-finland-92144, could we include a link to "flyte the hard way" somewhere in the deployment guide? WDYT?

agree, will work on this