Flyte enables production-grade orchestration for machine learning workflows and data processing created to accelerate local workflows to production.

Flyte

Hi, <@U03TF7U3VCY> - can you provide a little more context? What does the error message say? Can you use the ID to check if the job is still actually running?

Hey, the error is just a aws batch job runtime error and the job goes to the failed state. So it is not running. But on the flyte console side the shows task is still running so I cannot know until I click the task detail in the flyte console

Interesting, it seems that Flyte is failing to report the error. Do mind filing an issue on Github? it helps to be as thorough as possible in the description (ie. exact error message, etc). We will pick this us ASAP.

cc <@USU6W5ATA> I know you've done some work on AWS batch - do you have any initial ideas?

Does this task rerun again? (task will rerun 30 times by default if propeller got retryable failure). if the task is rerunning, the status will be “running”

Hey, no the task didn’t rerun, ok I will fill this issue on GitHub

Sorry for the late reply since we are on different time zone

Ok it seems that the aws batch job is actually in *SUCCEEDED* state, but there’s exception captured by flyte and logged in cloudwatch. And also flyte console knows it is failed. So I guess this is related to flyte captured the exception so *no exception raised in aws batch job container runtime*. But flyte knows there’s an exception…

Do you think this is a bug?
If so, would be good if you send me an guideline about filling an issue in flyte’s github repo, if there’s any

Just file a bug at <https://github.com/flyteorg/flyte/issues/new/choose|here>. btw, mind sharing the code snippet, that will help debug.