@ancient-wolf-19325 Is there a way to override interruptible-failure-threshold for a project or task?

Yes, you can override the

interruptible-failure-threshold

for a project or task. This configuration determines the number of failures a task can have while still being considered interruptible. If the number of failures exceeds this threshold, the task will be retried on a non-spot (regular) instance. According to this discussion, you can set the

interruptible-failure-threshold

to a higher number than the number of retries to ensure the task is only executed on spot instances. This configuration can be applied at the system level, but it can also be customized for specific projects or tasks. Would you like more details on how to configure this setting for a specific project or task? Sources: - Flyte Discussion on Interruptible Failure Threshold 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.

@ancient-wolf-19325 how do I configure that for a specific project or task

To configure the

interruptible-failure-threshold

for a specific project or task, you need to set it in the task definition using Flyte's Python SDK. This can be done by specifying the

interruptible

flag and setting the number of retries in the task decorator. However, the

interruptible-failure-threshold

itself is typically configured at the system level. Here's an example of how to set the

interruptible

flag and retries for a task:

from flytekit import task, workflow

@task(interruptible=True, retries=3)
def my_task() -> str:
    return "Hello, Flyte!"

@workflow
def my_workflow() -> str:
    return my_task()

For more granular control, such as setting the

interruptible-failure-threshold

specifically for a project or task, you would typically need to adjust the FlytePropeller configuration. This is done at the system level and affects how retries are handled globally. Would you like more details on configuring FlytePropeller for specific projects or tasks? Sources: - Flyte Discussion on Interruptible Failure Threshold - Flyte Documentation on Interruptible Tasks 👍 0 button 👎 0 button 💡 Hint: Mention @runllm in the thread for followups.