Hi team.. I have a container task that shows as “Running” but if I click into it, it’s really “queued”. There is no pod in pending state and no info in flytepropeller logs. Not sure what’s going on and how to debug this

did the CRD get created?

kubectl get fly

on relevant namespace. what deployment are you running?

@Erich Shan I seem to be missing some permissions here.. can you try?

@Yee yes it was created

which helm chart are you using?

flyteorg/flyte-core, one moment, it might not be scheduling due to our high ephemeral storage request. we're testing it now

you mean flyte-core? so propeller is a separate pod than admin?

yeah i was checking the propeller logs all i had seen for that was

{
  "json": {
    "exec_id": "<project>:<environment>:flyte.workflows.<workflow>.workflow_wf",
    "ns": "",
    "routine": "worker-1"
  },
  "level": "warning",
  "msg": "Workflow namespace[]/name[<project>:<environment>:flyte.workflows.<workflow>.workflow_wf] not found, may be deleted.",
  "ts": "2024-06-04T20:10:49Z"
}

but that might just be a warning

We were able to resolve this

what was the issue?

Can you share what the issue was pls ?

we found that we had rwx access mode set for a gp3 block storage volume. it said nominated for scheduling and no useful errors