microscopic-furniture-57275
06/06/2024, 7:26 PMRequests overridden on node dn1 (<FlyteLiteral(NodeMetadata) name: "rf_v2_train_and_test_task" retries { retries: 2 }>) without specifying limits. Requests are clamped to original limits.
I have supplied the argument retries=2
to my tasks, because I want to ensure that if the spot instance is reclaimed, it will be retried once more on a spot instance, and finally, if it is reclaimed again, it will be run on an on-demand instance. That is my understanding of these docs :
If you set retries=n, for instance, and the task gets preempted repeatedly, Flyte will retry on a preemptible/spot instance n-1 times and for the last attempt will retry your task on a non-spot (regular) instance. Please note that tasks will only be retried if at least one retry is allowed using the retries parameter in the task decorator.
I don't understand the meaning of the error message, and what I should be doing differently.
Thanks for any pointers!tall-lock-23197
tall-lock-23197
user
06/07/2024, 9:19 AMtall-lock-23197
microscopic-furniture-57275
06/07/2024, 1:57 PMmicroscopic-furniture-57275
06/07/2024, 2:13 PM