I ran into a task being scheduled but never starti...
# flyte-support
b
I ran into a task being scheduled but never starting due to a system problem in k8s blocking it (a failing mount) -- I can see there's the
timout
arg for
@task
decorators, but just making sure there isn't a system-wide default that can be configured?
I'm thinking along the way Buildkite does this for CI pipeline workflows https://buildkite.com/docs/pipelines/configure/build-timeouts
f
there is a systemwide default, per node
its in propeller config
a
@bland-dress-83134 it should be under
Copy code
propeller:
  node-config:
    default-deadlines:
      node-active-deadline: 0s
there are other parameters you can adjust: https://www.union.ai/docs/flyte/deployment/configuration-reference/flytepropeller-config/#node-config-confignodeconfig The actual location of the config depends on the Helm chart you're using
b
this is just a total time limit for tasks including queueing right? My situation was that the task was considering "running" from Flyte's perspective since the pod was created, but the pod just stayed in Pending indefinitely: I think what I was really looking for was a timeout from pod created -> pod running, but Flyte probably doesn't keep track this closely. 🤔
f
Aah yes, sad kubernetes problem
If you look at Flyte backend you will see we do support failure. But the timeout today s enforced after pod startup
We could add this new type of timeout / but I think it is infact some type of error
Look at the Flyte backend code and share what status you are seeing we can add support for it