sparse-advantage-22780
10/13/2023, 4:02 PMsparse-advantage-22780
10/13/2023, 4:02 PMWorkflow[flytesnacks:development:.flytegen.flyte_rf.master.run_rf_mpnn_esm_wf] failed. RuntimeExecutionError: max number of system retry attempts [11/10] exhausted. Last known status message: Failed to check Catalog for previous results: rpc error: code = Internal desc = Unable to read artifact data from location <s3://my-s3-bucket/metadata/flytesnacks/development/flyte_task-flyte_rf.mpnn_workflow.run_protein_mpnn/2.0-so0rb28S-ERl7whL3/267feb64-ab2e-4689-8679-e2fbd807c8c0/o0/data.pb>, err path:<s3://my-s3-bucket/metadata/flytesnacks/development/flyte_task-flyte_rf.mpnn_workflow.run_protein_mpnn/2.0-so0rb28S-ERl7whL3/267feb64-ab2e-4689-8679-e2fbd807c8c0/o0/data.pb>: not found
sparse-advantage-22780
10/13/2023, 4:08 PMflytectl demo
sandbox? but i'm not sure how I can delete that. So far I've teardown'd the cluster and pruned volumes and deleted ~/.flyte
but I think something must be missing somewhere. I'll try again though to be sure.high-accountant-32689
10/13/2023, 5:57 PMsparse-advantage-22780
10/13/2023, 6:12 PMdocker volume prune
(but now I see that flytectl demo teardown -v
would have also worked)
seems like the workflow should not fail if it is unable to find this though....