Hi folks, I am trying to launch the flytesnack task, which called

_bash.blastx

, but it displays below logs. Can somebody let me know what's going on here? It's status stays

unknown

forever.

c":"handlers.go:237"},"level":"debug","msg":"Running authentication gRPC interceptor","ts":"2022-05-30T16:01:59+09:00"}
{"json":{"src":"handlers.go:193"},"level":"debug","msg":"gRPC server info in logging interceptor [1a329842-ff6f-4268-8349-edd25113438c]method [/flyteidl.service.AdminService/CreateExecution]\n","ts":"2022-05-30T16:01:59+09:00"}
{"json":{"src":"execution_manager.go:808"},"level":"debug","msg":"Launching single task execution with [resource_type:TASK project:\"flytesnacks\" domain:\"development\" name:\"_bash.blastx\" version:\"v0.3.82\" ]","ts":"2022-05-30T16:01:59+09:00"}
{"json":{"exec_id":"a4jj4dgl6lk24nbn75vt","src":"execution_manager.go:381"},"level":"warning","msg":"Failed to fetch override values when assigning task resource default values for [resource_type:WORKFLOW project:\"flytesnacks\" domain:\"development\" name:\".flytegen._bash.blastx\" version:\"v0.3.82\" ]: Resource [{Project:flytesnacks Domain:development Workflow:.flytegen._bash.blastx LaunchPlan: ResourceType:TASK_RESOURCE}] not found","ts":"2022-05-30T16:01:59+09:00"}
{"json":{"exec_id":"a4jj4dgl6lk24nbn75vt","src":"execution_manager.go:385"},"level":"debug","msg":"Assigning task requested resources for [resource_type:WORKFLOW project:\"flytesnacks\" domain:\"development\" name:\".flytegen._bash.blastx\" version:\"v0.3.82\" ]","ts":"2022-05-30T16:01:59+09:00"}
{"json":{"src":"queues.go:43"},"level":"debug","msg":"refreshing execution queues","ts":"2022-05-30T16:02:33+09:00"}
{"json":{"exec_id":"a4jj4dgl6lk24nbn75vt","src":"queues.go:73"},"level":"warning","msg":"Failed to fetch override values when assigning execution queue for [{ResourceType:WORKFLOW Project:flytesnacks Domain:development Name:.flytegen._bash.blastx Version:v0.3.82 XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}] with err: Resource [{Project:flytesnacks Domain:development Workflow:.flytegen._bash.blastx LaunchPlan: ResourceType:EXECUTION_QUEUE}] not found","ts":"2022-05-30T16:02:33+09:00"}
{"json":{"exec_id":"a4jj4dgl6lk24nbn75vt","src":"queues.go:109"},"level":"info","msg":"found no matching queue for [{ResourceType:WORKFLOW Project:flytesnacks Domain:development Name:.flytegen._bash.blastx Version:v0.3.82 XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}]","ts":"2022-05-30T16:02:33+09:00"}
{"json":{"exec_id":"a4jj4dgl6lk24nbn75vt","src":"execution_manager.go:529"},"level":"info","msg":"getting the workflow execution config from application configuration","ts":"2022-05-30T16:02:33+09:00"}

Thanks in advance. P.S. I managed to launch the flyte on

on premise k8s cluster

using keycloak as auth backend, if it is relevant to anything.

Hi @Jake Yoon. Can you provide more complete logs from admin and propeller.

With your guidance I could find error from the propeller logs. Seems below portion is wrong.

E0530 10:47:08.166833       1 workers.go:102] error syncing 'flytesnacks-development/admjgn2s8rwn9xrtj544': Workflow[] failed. ErrorRecordingError: failed to publish event, caused by: EventSinkError: Error sending event, caused by [rpc error: code = Unauthenticated desc = token parse error [JWT_VERIFICATION_FAILED] Could not retrieve id token from metadata, caused by: rpc error: code = Unauthenticated desc = Request unauthenticated with IDToken]

Any idea on this? I attached logs from admin and propeller pod. Thanks.

You would need to enable auth from propeller to talk to admin using client secret .Can you check this doc https://docs.flyte.org/en/latest/deployment/cluster_config/auth_setup.html

Can you explain it in detail? I mean I think I've already done what I have to do according to the guide from the url that you gave me. Thanks.

Hi @Jake Yoon. thats strange did you restart flytepropeller.

It was flytepropeller. I debugged the code with 'dlv' and found out that when it send the request to flyteadmin, there was no authorization header. The only thing I could do was disable grpc authentication, using

auth.disableForGrpc

flag. Anyway, thank you for your concern.