Hi, I am running interruptible workflows by setti...
# ask-the-community
t
Hi, I am running interruptible workflows by setting the interruptible flag at the workflow level, which causes all tasks in the workflow to be run as interruptible (even if they have no interruptible flag set) which is my intention. This is working, but I see an error associated with each task called by this workflow, referring to the retries param for the task:
Copy code
Requests overridden on node dn1 (<FlyteLiteral(NodeMetadata) name: "rf_v2_train_and_test_task" retries { retries: 2 }>) without specifying limits. Requests are clamped to original limits.
I have supplied the argument
retries=2
to my tasks, because I want to ensure that if the spot instance is reclaimed, it will be retried once more on a spot instance, and finally, if it is reclaimed again, it will be run on an on-demand instance. That is my understanding of these docs :
Copy code
If you set retries=n, for instance, and the task gets preempted repeatedly, Flyte will retry on a preemptible/spot instance n-1 times and for the last attempt will retry your task on a non-spot (regular) instance. Please note that tasks will only be retried if at least one retry is allowed using the retries parameter in the task decorator.
I don't understand the meaning of the error message, and what I should be doing differently. Thanks for any pointers!
s
seems like a bug.
would you mind creating an issue? [flyte-bug]
s
what's the flytekit version you're using?
t
I am using flytekit 1.12.0
@Samhita Alla I have filed an issue per your request: https://github.com/flyteorg/flyte/issues/5455