jolly-florist-66728
10/01/2024, 1:58 PMretries work? Our spark tasks sometimes have transient failures but they are not retried after a "user error".
Then I tested with a python task, just raising an Exception, it's not retried either.
Is the intended behaviour?
# Answer
Flyte's retry mechanism differentiates between recoverable and non-recoverable errors. Recoverable errors are retried, while non-recoverable errors are not. By default, all user exceptions are considered non-recoverable unless they are a subclass of FlyteRecoverableException (source, source).
For transient failures in Spark tasks, ensure that the exceptions raised are subclasses of FlyteRecoverableException to trigger retries. For Python tasks, raising a generic Exception will not trigger retries unless it is wrapped in a FlyteRecoverableException.
Would you like more details on how to handle specific exceptions in Flyte?
Sources:
- Build Indestructible Pipelines with Flyte
- Flyte Task System Retry
Slack Link
https://.slack.com/archives/CP2HDHKE1/p1727791110.858489 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.