Hi there does anyone have any tips on debugging why workflow

Question

Hi there does anyone have any tips on debugging why workflows are stuck in queue indefinitely

Yubo Wang · Answer

not an expert but have you tried `flytectl get execution p flytesnacks d development lt execution id gt details`

Ketan (kumare3) · Answer

in queue indefinitely can you talk more

Björn · Answer

I would check kubernetes resources used vs what the flyte project is allowed to consume

Yubo Wang · Answer

< Ketan kumare3 > FYI previously I had a task queued indefinitely was due to resource exceeded quota

Yubo Wang · Answer

``` start node SUCCEEDED 2022 10 15 03 11 44 931890319 +0000 UTC 2022 10 15 03 11 44 931890319 +0000 UTC n0 RUNNING 2022 10 15 03 11 44 950847963 +0000 UTC 2022 10 15 03 11 45 080042711 +0000 UTC Attempt 0 Task WAITING FOR RESOURCES 2022 10 15 03 11 45 075151304 +0000 UTC 2022 10 15 03 11 45 075151304 +0000 UTC Task Type python task Reason Exceeded resourcequota BackOffError The operation was attempted but failed caused by pods feaeaf1c97c574fd0aa8 n0 0 is forbidden exceeded quota project quota requested limits memory=500Mi used limits memory=3000Mi limited limits memory=3000Mi Metadata Generated Name feaeaf1c97c574fd0aa8 n0 0 Plugin Identifier container External Resources Resource Pool Info Logs ```

Ketan (kumare3) · Answer

i do agree that is the usual case

Ketan (kumare3) · Answer

for some reason our default helm chart had a very restrictive quota

Ketan (kumare3) · Answer

we have now removed it but that is only recent

Ketan (kumare3) · Answer

< Derek Yu> you might want to do `kubectl describe resourcequota`

Derek Yu · Answer

Thank you all for your help slightly smiling face and sorry for the lack of details We dug into it and it turned out to be that we needed a larger node with more memory than the ones we had available to run the task